June 14, 2009

What problem are you trying to solve?

Anyone who has worked in IT for any period of time is going to be familiar with the dilemma of choosing one particular solution over another. Should I learn language X or language Y? Should I use framework A or framework B for my next project? Should I use the Foo Architecture or the Bar Architecture? Is the Widget application better than the Grommet application?

Since sometimes it seems that the choice you make can have big costs in terms of your project’s success (and your future employability), these dilemmas can result in a lot of stress. It doesn’t help that for each solution you have some proponent telling you that if you don’t use their approach you are stupid and ugly (see Linus Torvalds talking about his source control tool ).

So what’s the smart way to decide?

I think the first step is to realize that you have the wrong question. Technology people by definition tend to be interested in technology (tools and solutions) and we tend, as a result, to give short shrift to the problems themselves, especially when they fall outside our domain. As a result, solutions pile up faster than you can evaluate them, and problems often get shoehorned in a pet solution. So what is the right question?

Do you remember those word problems we used to have to do in school? Trains leaving the station at certain times and speeds, Mary and Bill trying to divide up a pie, and all those hypothetical problems?

The secret of those problems was often just figuring out what the problem was in the simplest terms. Once you rephrased the question, you often knew exactly which technique from that week’s lesson to apply to solve it. You could just whack the problem with all the techniques you had learned, hoping to hit the right one, but you would probably take too much time or might even get an “answer” that was totally off base.

To make this brute force approach even less likely to succeed, the teachers and textbook sometimes threw in irrelevant details that sounded important but had nothing to do with the real solution, in the same way that mystery writers throw in red herrings to make it harder to guess whodunit.

So the only reliable way to solve these problems correctly was to learn to filter out the extraneous blather and find the pithiest answer to the question “What problem am I really trying to solve?”

So how can we use this to solve our solutions dilemma? Let’s start by recognizing that the real problem is not what solution to choose but rather resolving the problem the solution is for. Trying to choose a tool without explicitly defining the real problem is exactly like trying to solve those word problems by throwing all the techniques you know at them. You may get lucky, but more likely you will make a mess.

If you realize, having thought about it, that the problem you are trying to solve is to avoid having others think you are ugly and stupid, then there is an easy solution to the problem: give up on technology altogether. In technology there is always someone telling you are ugly and stupid because you didn’t choose the same solution as them, so you might as well get used to it. If you want to go one step further, reject all forms of ideological thinking on principle, and craft your own rational set of criteria by which to judge things.

If you discover instead that there is some substantive problem you want to solve, spend some time trying to understand the problem. Ask yourself what the essential characteristics of the problem are. Identify some things that might normally be of interest but don’t really apply in this case or have less importance than normal. The truth is that no solution is perfect, all solutions require trade offs, and the secret of success is focusing on the must-haves while sacrificing the merely nice-to-haves or, more often, even the not-quite-so-must-haves.

Sometimes I find it very useful to start sketching out my own solution from scratch. At some point you realize either that you have a nice simple solution and don’t need to go any further, or that, now that you have a better feel for the problem and its challenges, that you have clear criteria upon which to base the choice of an external solution.

Anyone who wants to get good at software architecture has to be willing to throw out conventional wisdom and make choices that fit the problem at hand, not the problem that advocates of particular solutions tell them they should have. Sometimes just spending some quality time with your specific problem clarifies what you really need and melts the solutions dilemma away.