Every developer who's been working for a while has been the main character in a very specific story.

Chapter 1: Someone has an idea. It could be you, a product manager, a C-level person, it doesn't really matter. But the idea goes, "Hey, I wonder if we could . . . ." And you think to yourself, "I don't really know for sure, but yeah. I could probably do that." So you put in a little extra time a few nights or on a weekend and you throw something together that does whatever that idea was.

Chapter 2: Everyone loves it. It's the new hotness. It could be an internal tool, or it could be an actual business idea for a startup, or it could be something in between. Doesn't matter. What matters is that you found your audience, and they love it. They want new features, which you very quickly add. Development is rapid, and everyone is very happy.

Chapter 3: Things break every once in a while, but people don't get that upset because this is all new and just a proof-of-concept anyway. They appreciate the work that's going into all of this because it's just all happening so fast.

Chapter 4: The Thing has been around for 6-8 months. People are using it daily and have come to rely on it. It's no longer the new hotness, and your team or customers aren't so charitable when things break. So management decides to invest your time (or your team's) into it officially. "Make it stable enough for production."

Chapter 5: Why is this taking so long? It already works! Just make it work better! This should just be like a 2-3 day project!

Chapter 6: Everyone is mad.

Why does this story happen so often? How can we prevent it?

The thought I'm going to explore in this article is that it happens because both companies and developers are unwilling to take out the trash. We need to incorporate garbage collection cycles into our human processes.

There are all kinds of people who will try to sell you solutions to this problem, but none of them are really correct. They fail to acknowledge that proofs-of-concept and MVPs are actually just learning mechanisms for developers to figure out a problem domain. That goes for traditional SDLC (i.e., waterfall), Agile, and Test-Driven Development. All of which try to solve this exact problem with exactly zero success.

Failed Solutions

Waterfall, Agile, and TDD are all different variations of the idea that you should "just build it right the first time." And my argument here is that is fundamentally the wrong approach and the wrong expectation. They are each very different in their approaches, but they are completely and totally wrong. Here's why:

• Waterfall says that you frontload the timeline by learning everything you need to know about the problem domain and the requirements. You don't just go out and build a thing based on a conversation. A) that's not reasonable, and B) there are tons of things you don't know about implementation and architecture details no matter how much research you do. There are many things you can't possibly know until you get into the weeds of writing the code.
• TDD says that if you build the test harness as you go, then you have—by definition—built the right thing. It's a fairly pedantic argument to say that a problem isn't really a problem because we've defined it as not-a-problem. But that's really what it does. In addition, TDD actually promotes this story at the top by sinking costs directly into the developer's learning process. You can't write effective tests without knowledge of the problem domain, and you often don't have that when you're writing an MVP. When you TDD a new project, you are essentially locking yourself into an undesigned architecture that's based on a completely naive understanding of the problem space. That is always going to end up putting Babe in the corner, and no one should do that. The test harness that you create along the way limits your options to rework the fundamental architecture because it's just so much work to redo all of that. It's not the point of a proof-of-concept.
• Agile says that if your process is agile, then you have—by definition—built it right the first time. It is the human process version of TDD. Working small ideas one at a time without any guiding design is a recipe for an absolutely terrible architecture, a codebase that sucks, and a fragile product that's unmaintainable. It's the equivalent of managing your logic with post-it notes all over the office walls or walking a random forrest.

So now that I've made pretty much everyone in the software developer world angry, what's next?

Take Out the Trash

We should acknowledge that every developer/team's first foray into a new thing is absolute garbage. That doesn't mean the developer is garbage. It only means that we acknowledge that software development is a Bayesian process. We start with some priors, and we update them as we go. Aspects of process like Agile and TDD slow down the updating mechanism or stop it altogether because we get entirely too sunk into our priors. Waterfall is an assumption that we can verify our priors and they will not need updating. TDD bakes the priors into the code, and Agile incorproates the initial priors into a process. All three approaches fundamentally defeat the learning process in one way or another.

Process is fundamentally bad for the exploratory and learning aspects of software development. Software Development is an inherently creative activity that needs freedom to learn from mistakes and adapt to them. Being free from constraints and able to experiment is a necessary step in designing good architecture.

But it is very difficult to get institutional buy-in for taking out the trash. So here are some strategies I've found to help make that more palatable:

• Write your MVP in a language that you don't know very well or a language that isn't generally supported by your company. Are you mostly a Python shop? Write your MVP in C# or Clojure. Are you a mostly functional language team? Write the MVP in Ruby. Or possibly write it in a meta language that can compile to multiple different langs like Nim. The possibilities here are endless, and you probably need to make sure you get the sign-off from your boss. The point is to do something that's guaranteed to get garbage collected. Plus, you get to learn more about a new language. If your boss' boss' boss' boss' boss is like, "I want this in production now!" and your boss is like, "Yeah, we don't have the ability to support brainfuck in prod right now. We need a v2 in common Lisp." That's a lot more likely to get prioritised than, "Well, yeah we could push that to prod, but we shouldn't for various technical reasons that you don't understand, Mr. CEO."
• Write integration tests as soon as you start seeing adoption. These don't require any specific implementation details at all. Just exercise the API. You can do this with make or bash if you want. You don't need anything fancy. It gives you some amount of stability, as well as a good starting point for v2.
• Speaking of, v2 should always be on your mind when you're writing your proof-of-concept. Always expect to want to have done things differently. When you actually get the go-ahead to work on it, notes and integration tests are a fantastic little love letter from your past self to your future self.
• Always have a deployment process. Bash script, Ansible playbook, Makefile, whatever. Always have a thing you can call in very simple terms that will get a new version deployed to users. You are going to have a situation where some user bumps into an edge-case and the fix is like one line of code, but it's been 3 months, and you don't remember the exact incantation to get it out there. Document it. Script it. With something.

I've bagged on almost every methodology that exists in this article, but that doesn't mean that I think they are all bad. Reality is nuanced. And this article is about a specific scenario revolving around MVPs or proofs-of-concept and how to get those into production.

But that doesn't mean I hate these methods of getting things done. In situations where developers really know the space, any of them could be real options. For example, I've implemented banking systems based on ISO-8583 twice, for two different companies, using two different programming languages. If someone came to me and asked me to do it a third time, first of all: no. But second, I would waterfall the hell out of that because I know what's needed and how sideways it is going to go. If someone asked me to work on, say, music composition software, I would TDD the hell out of that because I have the domain knowledge of a music theorist and composer already.

I'm not suggesting that these things are inherently bad. I'm saying that they don't account for the garbage that we create as we are learning, and that is bad.

We need to find our trash and take it out.