Some Thoughts on Malcolm X

I've been a fan of Spike Lee's movies for a long time, and I still remember seeing his "Malcolm X" in the theater.

Though it may seem like an odd thing for a white guy, I left that theater an admirer of the icon of Black Power who would later choose the name Malik El-Shabazz, and earn the honorific El-Hajj.  

And it wasn't just the universalist El-Shabazz I admired, the one who had given up his anger and hatred against me and my kind, but, in part thanks to the brilliance of Lee's movie-making, I also learned to admire, and appreciate the necessity for, his time as "Malcolm X", when he would have been perfectly happy to call me a "white devil" to my face.

When you believe passionately that diversity is a strength and an enhancement to the quality of your life, when you feel that you are "one of the good guys" reaching out in friendship, the venomous rejection that X and the Nation of Islam were spewing out is very hard to take, even if you recognize the justice of the grievances that inspired it.  It isn't easy being told "We don't want you here" when you feel you are on the same side.

But I learned through Spike Lee and Malcolm X that the "white devil" message was not about me, and not for me.  In a way, it was none of my business.

It was about purging the internalized sense of inferiority, shame, and envy that had been instilled by centuries of domination and abuse. The truth is that equality is not conferred by others: it needs to grow and become firmly established within first.  Once it is there, others can accept it or not, but then that is THEIR problem.

Centuries of collective, and decades of personal, anger and hurt had to be vented, processed, and transcended before El-Shabazz could emerge, equal not because someone else (or some document) told him he was, but because he just was, and knew it as surely as he knew his own freely-chosen name.

Another important thing I admired about Malcolm X, and the Nation of Islam, were the discipline and focus they brought to their activities, and while the message may have sounded vitriolic, they stopped short of actual violence.  This is not a trivial accomplishment, and it makes all the difference.

Recently, it has come to my attention that I need to relearn the lesson that Mr. Lee and El-Hajj El-Shabazz were kind enough to teach me over twenty years ago.

In my larger professional community of software developers, there are new groups with real historic grievances, and they are getting to the point where they too want to sit at the table without having to apologize for who they are. I've been fortunate to work in environments with a lot of diversity, and I welcome more of it enthusiastically. 

But some of these aggrieved groups are working through the anger and hate built up from their long suppression, from the mistreatment they have received from our industry and society at large. And sometimes what they say and do makes me uncomfortable, seems to target me or seems to contravene the values of diversity as I understand them.

As I said, this can be hard to take.  But maybe I have to remember the lesson of Malcolm X: it's not about me, and it is not for me. In a way, it's not even my business.

Rather than taking offense, or speaking up, maybe the best thing I can do is LET them hate my male, white, hetero, cis, middle-aged ass. 

If it gets too much, I can look away, ignore it, bite my tongue, live my values in some other way.  Short of actual violence, or the genuine threat of it, maybe I need to let them express themselves, however they like, until they've worked it through.

Diversity is one of MY most deeply-held values, and no one said that living your values was going to be easy (actually, I'm pretty sure that it being hard and doing it anyway is how you know it really is a value). I will continue living that value, even if sometimes it is rejected by those I'm reaching out to.

Perhaps the best I can do is to continue to be a SILENT ally, and to be ready, when they have worked through all that anger, when they are healed, to welcome them as the friends, colleagues and siblings in the universal family that I already know them to be.

Truth, Sales and Leadership

Salesmen, politicians and chief executives are all cut from the same mould. Their fundamental job is to impress and make others feel good about buying what they have to sell.
As someone who has spent his career in product delivery, and who is sceptical by nature, I’ve had a mixed relationship with the sales and presentation types. I have all too often ended up on the hook when one of them sold unicorns when what we had in the barn was donkeys.
But, unlike many technically oriented delivery specialists, I have a deep appreciation for the value and necessity of presentation and sales. I understand the symbiotic nature of our separate disciplines. In the end, things only work out well when both roles are filled competently, and both sides need to remember their dependence on the other if they want to succeed.

To Impress or to Be Impressive

If at this point you aren’t sure which side of this proposed divide is your natural home, let me propose a simple way to decide: which do you think is more important, to impress or to be impressive? If you don’t understand this distinction, let me rephrase: would you rather be famous and admired for something that you didn’t put a lot of effort or skill into, or be amazingly accomplished at something that no one knew about you?
Now some of you are I’m sure objecting that these are always or usually the same thing: accomplishment and skill, and recognition for that accomplishment and skill generally go hand in hand.
But the truth is that it is rarely the case that the two go together naturally. Are the best singers and actors the biggest stars, and vice versa? Are the most successful politicians most often the smartest and most qualified decision-makers? Is your average CEO the biggest expert in the business he is running?
This isn’t a complaint that the world isn’t fair. On the contrary, I think there is a good reason why things work this way. Impressing people requires that you present yourself in terms that they can understand, evaluate and appreciate. Being impressive requires honing your abilities and understanding beyond the experience and comprehension of most people, to make distinctions and to have perceptions beyond what is evident to a non-expert.

When Sales or Fulfillment Goes Bad

Each of these sides has its obvious pitfalls, and to understand these, all you have to do is think about the negative stereotypes of these two groups.
The bad salesman or politician is full of hot air, with plenty of promises that never get kept. The bad salesman is convinced he could “sell snow to the Eskimos”, the bad politician certain that he could finesse over the most egregious scandal and get away with breaking every promise.
The bad expert (let’s use software developer as an example) is disdainful of the buying public, mystified that they don’t see the deep technical skill that went into solving several difficult and subtle challenges but are instead impressed or put off by trivial details, such as the colour choices on the main screen. The bad software developer is convinced that good software will always sell itself, and sales and marketing are clueless bozos who soak up money and accolades that rightfully belong to the technical geniuses.
Both of these stereotypical bad guys are wrong: they desperately and inseparably need each other.

Commanding Officer and Executive Officer

To illustrate the symbiosis of these two callings, I will draw on an example that has literally been battle-tested over centuries. It is traditional in the military for there to be two command roles for every unit: Commanding Officer (CO) and Executive Officer (XO).
The CO is the external face of the unit. It is his job to represent the skills of his unit up the chain, to convince the command hierarchy to give his unit more resources, and to cheerlead and praise his men for their accomplishments. Ideally, he should be loved by his men, and they should feel proud and honoured to serve under him.
The XO is the internal manager and disciplinarian of the group. It is his job to push, prod, terrorize and manhandle his men to improve their skills and achieve their mission goals. He must be the first to observe and criticize any short-comings and ruthlessly punish any breach of discipline. He is the expert in building the best possible soldiers. Ideally, he should be respected and feared, at best grudgingly liked, and made fun of when the men are absolutely, positively certain that he can’t hear them.
With such radically different profiles, you would expect that these two officers should not work together very well, right?
But the truth is that, if both men actually understand their roles, they should be working in perfect collusion with each other. The XO should be feeding the CO with good things about the men that can be praised, and let him know when the men are achieving their best and can’t be expected to give anymore. The CO will let the XO know if any bad reports are coming in from outside the unit and share his fears of how the unit might fail in upcoming missions.
The two roles are distinct, and require different focuses, but they must be done in perfect concert to achieve maximum effectiveness in the situation. Such dual leadership roles can and often are performed by a single person in non-military leadership situations, but there is something to be said for the collaborative specialization of individuals dividing up the areas of responsibility. Even if you find yourself with both roles, you have to find some way to separate them in the minds of others so that they don’t contaminate and undermine each other.

Truth and Presentation

I think we would all agree that the ideal situation is the one where things both seem good and are good, but where the downside is also known and managed. Being aware that some aspects of the truth (possibly not the most important ones) need to be highlighted for the sake of presentation is important for experts to understand. Keeping in mind that finding and fixing negatives is essential for quality products and that it is in everyone’s interests to not diverge too far from the truth in presentation is vital for sales to understand.
A well-packaged truth can only come about when both perspectives work together and respect each other.

On Reviewing Old Code

I recently committed an old programming project to Github.

The project is several years old – older than Java 5 at least, since I had to add some generics to the code to get rid of some compiler nags before committing it.

It was one of my early TDD exercises. It was the last iteration of a quest I was on to develop a realistic world map generator that could be used, for example, in games such as Civilization or for any other purpose that might call for fictional worlds. Over the years I had tried several different algorithmic approaches and used several different programming languages: C, C++, Visual Basic, and finally Java.

When I was about to take a look at the code after all these years, I felt some trepidation. I’ve downloaded many open-source projects and I’m used to the shock that often accompanies the first peek at unfamiliar code. Clean code is not a universal value, and sometimes beautiful code is in the eye of the beholder. This code for the map generator was old enough to have become somewhat unfamiliar, and I was prepared for the worst.

On the whole, I was relieved to find that the code was reasonably clean. My experience of reading clean code is that you feel a kind of disbelief that code that seems so simple could be producing such complex behaviour. Well-factored, clean code looks, paradoxically, unimpressive because it is so easy to follow. To reverse a questionable old adage, it was harder to write so it could be easier to read.

What I saw was certainly not perfect. There were many conventions regarding unit testing that I evidently had not yet established for myself at the time the program was written, and it could still use some even more aggressive refactoring. As I recall, I simply ran out of steam toward the end of the project, so it’s possible I knew about these problems at the time but didn’t get around to fixing them.

The biggest thing that struck me upon reviewing this project was how far away it seemed. I had obviously been obsessed with map generation for many years, to the point of learning some GIS techniques and using it as a guinea pig project for new languages and new techniques. Even though I’m pretty self-motivated, in the end I ran out of steam. In the absence of a user community for this project, it became impossible to rationalize the effort anymore.

No matter how independent-minded I am, no matter how internal the art of programming is, in the end it seems that software is all about fulfilling the needs of others.

The Usefulness of Philosophy

I came across a comment on a blog recently where the author adjured the other participants to stop “philosophizing” so much and get down to the real matter at hand. This reminded me that, for many people, “philosophy” is a term of abuse. For them, philosophy is pointless blather among elitist twits with no practical consequence - the very opposite of anything practical and useful.

Given the name of this blog, you can guess that I don’t agree with this negative sentiment. I won’t deny there are some philosophers and some philosophies that I think are pointless blather, but to take them as our basic definition is to throw the baby out with the bath water.

I want to propose that philosophy is the study of mental models, and, as I said in my previous post, I think mental models are the basis of our competence. Since our competence determines how well we manage and how effective we are at realizing our goals, there is obvious practical importance in understanding how mental models work, getting used to taking them apart and building new ones.

As I explained in my very first post regarding my chosen name for the blog, I think software development is an eminently philosophical activity. It is all about constructing mental models of systems and manipulating those systems using the mental models. It doesn’t matter whether these systems are machines, protocols, teams, problem domains, programming languages, etc: how effectively you work with them depends on your ability to construct and manipulate good mental models of how they work.

Being unaware of your own mental models is a limit to your own effectiveness. We have all known people who thought they had found the perfect hammer and were busy nailing everything. Likewise, we have probably known someone who repeated the same dysfunctional pattern over and over again in spite of not getting the desired result.

Philosophy as the study of mental models can make you aware of the mental models underlying these behaviours and can give you the skills you need to improve them.

A word of warning though: as Socrates found out the hard way, people often get very upset when you question their cherished mental models, and this can happen even when we question our own. However, if you want to improve effectiveness and grow in competence, there is much to recommend the use of philosophy to take apart and rebuild our mental models.

The Importance of Mental Models

Many years ago, my wife and I were in Paris, staying in a quaint Left Bank hotel that did not provide an iron and ironing board. My mental model of a hotel is that it should provide these, free of charge, and preferably one to each room: I like having wrinkle-free clothes, even on holiday. However, as we will see, I’ve learned that my mental models are not always adequate representations of reality, and that in order to solve my problems, I may have to construct a new mental model. We decided that the solution to our problem was to buy a compact travel iron to solve the problem once and for all.

To Find an Iron in Paris: How Hard Can It Be?

At home in Toronto, the obvious place to buy a small electrical appliance would be at a large department store, so we thought that the first place to look for our quarry would be at a well-known Parisian department store a healthy walk from our hotel. When we got there, we wandered around a bit, but couldn’t see an appliances department, so we asked a saleslady where we could find such a thing. (We are both fluent in French, so we had a leg up on most tourists in such a situation.)

The saleslady was polite and helpful, but bewildered that we would be looking for an iron in her store: in her mind this clearly wasn’t the kind of place you shopped for such things. It was as if I had gone into a sporting goods store and asked if they had a fresh produce section.

Another faulty mental model: in spite of being two adults with years of experience fending for ourselves, able to speak the local language, familiar with Paris from previous trips, it dawns on us that we are simply lacking the competence to perform the simple task of buying a travel iron in Paris.

What the Heck Is “Darty”?

Luckily, when we asked our friendly saleslady where we might purchase an iron nearby, she offered one word: “Darty.” We weren’t sure if this was the name of a street, a neighbourhood, a local shop-keeper or a store, but she pointed us vaguely up the street, and off we went. Along the way, we found a small shop whose sign indicated that they sold electrical supplies, and we thought this might be what we were looking for, but in fact this store only sold light bulbs of every shape and variety, all stored in rows of wooden drawers mounted like a library’s old card-catalog on the walls. My mental model of the world did not contain the possibility of such a store, so I was mystified and delighted: if I ever need a light bulb in Paris, I will now know where to go.

After wandering around in circles through the figure eight streets, repeatedly asking reservedly helpful passersby for directions, and being vaguely pointed, sometimes in contradictory directions, we finally found a fairly large store, set back from the street with a sign proclaiming it to be “Darty”. At last!

Making a Purchase: What Could Be Easier?

Once we entered the store, it was clear we had found the right place. There were rows and rows of various kinds of home appliances and electrical gadgets. Not all of the logical groupings were readily apparent to me: for example there might be electric fans and electric razors in the same shelving island. Each item had a single display model, out of box, on a shelf with a small card with a number next to it. After some wandering around, we did find a small row of irons, one of which was a compact travel iron.

Now for our next challenge: how to buy one? There were no boxed models to pick up and take to the sales counter, just the floor model and the little number. We stood their looking and feeling clueless for a while, until a saleslady spotted us and asked if we needed help. Saved! Now, we thought, she will get us our item, process our transaction and our quest will be over.

We indicated to her the travel iron we had selected. She took out a little paper form, filled out the number of our item on it, handed it to us, and cheerfully bid us good day. Another perfectly good mental model crushed by a cruel Gallic world! I sheepishly asked where I was supposed to take the form. She pointed vaguely across the store, saying there was a counter.

Sure enough, we crossed the store and found a counter with several clerks standing around. We handed one of them our form, they processed our payment, gave us a new stamped form, and bid us a somewhat perfunctory good day. I waited for a moment, expecting our iron to appear, in spite of the fact that our clerk seemed to have completely lost interest in us. After a few moments, I asked where my iron was. They pointed vaguely in a new direction across the store, and we traipsed off, ending up among rows of televisions sets.

The sales guy there took pity on us, despite my mild irritation that I had already paid for my iron, but did not yet have it in my hand (another mental model). He explained that we actually had to leave the store, and walk half-way down the hall of the indoor mall it was in and we would be able to get our iron there. I’m starting to feel like a sucker: they’ve taken my money, but they are now telling me to leave the store with just a little stamped form and someone down the way will give me my item? Riiiight! Nonetheless, we followed his instructions, finding a little kiosk down the way, unmarked and unattended. After a few moments standing there, dejected and simmering, an attendant appeared, took our stamped form, and handed us our boxed travel iron. At last!

The Joys of Travelling

Now you could take this story as a mockery of the French way of doing things, but the American tourists I’ve seen loudly berating French workers for their bad customer service have got that angle covered.
In fact, I love these kinds of experiences, though they can be distressing at the time, and they are exactly part of the reason I travel. I want to have my mental models challenged by a different culture.

There is a perfectly good system at work there that Parisians navigate every day, I just didn’t understand it, but now I do, and having done so, I’m now just a little bit more competent at how to get needful things done in Paris.

Working with Systems Is Working with Mental Models

Though you could just read this as an amusing anecdote about travel, my real purpose in telling it is to apply it to thinking about useful systems. Doing software development, or managing a team, or running a business are all about navigating, manipulating and improving systems. To work with a system effectively, you need to have a good mental model of that system.

If you want to lead a team, one of your biggest challenges is to communicate your mental model of the undertaking you want the team to pursue. Only if all the members of the team share a mental model which is adequate to the task at hand can they work together to produced the desired end.

In fact, I would go so far as to say that to call yourself competent at some skill is to say that you have acquired an adequate mental model of that skill.

Humility Is the Path to Competence

As the iron-buying story illustrates, it can be frustrating and humiliating to be confronted by a foreign mental model. We are comfortable considering ourselves to be competent adults, knowing how to do things in the world, and being confronted with our own incompetence in the face of an unknown mental model can be painful.

This often leads us to disparage the people that have that mental model. For example, I’ve seen tech teams and sales teams run each other down behind their backs, each side thinking that what they do is complex and valuable, and what the other does is simple or over-valued. The fact is that each has invested a lot into understanding a complex mental model that underlies their respective competence, and it is easier to run down the other’s mental model than to accept their own lack of competence in the other’s domain.

My experience is that if I’m not getting the results I want, or if I’m having trouble communicating with someone else, the challenge is to discover the right mental model to make me competent at that task. The necessary ingredient, sometimes hard to practice, is the humility to abandon the comfortable mental model I’ve already mastered to be able to absorb the new mental model at which I am just a clueless newbie. Only by accepting my incompetence can I find the road to competence.

Captain Kirk Was a Lousy Tech Manager

Those of you who remember the original Star Trek series will remember the running trope of Captain Kirk asking Scotty how long something will take, and when Scotty responds something like “two days”, Kirk would respond with “You have two hours”, or some other ludicrously short period of time. Scotty would shake his head exasperatedly and go off to spin straw into gold (always making the deadline), while Kirk would get a smug, self-satisfied “There’s brilliant leadership at work” look on his face to let us know what a genius commander he was.

Years later on Star Trek: the Next Generation, Scotty made a guest appearance and confided to the engineer that he should never tell how long it would really take to do something: it turns out that the result of Kirk’s management style was to train Scotty to game the system.

Now sometimes good leadership requires pushing team members to pursue “stretch goals,” so that they continue to grow professionally and stay engaged with their jobs. But to make asking for the impossible into a routine part of every task assignment is just a bad idea. Scotty shows us why: it backfires, since the team member learns that honest estimations are punished.

The commanders in the next-generation Star Trek shows, Captains Picard, Sisko and Janeway, had much better leadership skills in this regard. They encouraged honest estimates and had open and respectful discussions with their team members about priorities and deadlines. It’s hard enough dealing with alien invasions and other calamities without introducing dysfunctional group dynamics into your own team through “heroic” Captain Kirk-style management.

The Diabolical Genius of C++

I have been feeling a strange pull to revisit C++ lately. Partly this is because C++ continues to be the language of choice for certain domains such as games and graphics programming that I find interesting. But more importantly, I have been curious to see what I would make of C++ if I took a fresh look at it all these years later, with the benefit of all the practical and theoretical expertise in programming and programming languages that I have acquired in the meantime.

I remember when the first edition of Effective C++ by Scott Meyers came out, and I was very tempted to buy it, but in those days computer books were ridiculously expensive, and I bought Bjarne Stroustrup’s equally fresh-off-the-press The C++ Programming Language 2nd edition instead, on the logic that it was the official reference and so its utility would stand the test of time. (Something you couldn’t count on with most technical books of the time.)

So given that Meyers’ book is still in print (in its 3rd edition) almost 20 years later, I figured it was the best place to go to reacquaint myself with C++.

Overall, I found Effective C++ to be a good book with good advice (though I might quibble here or there) and an excellent reminder of what programming C++ is like. What I rediscovered was that C++ is both a paragon and abomination of programming language design. It is in a way a poster child for the title of my blog, “Philosophy Made Manifest”, in the sense that it is so exquisitely the logical outcome of its philosophical premises. More particularly, it is the synthesis of two, quite different philosophies of software construction, and the extent to which it is a paragon or abomination is a direct result of the relative compatibility and incompatibility of these two different paradigms , in the Thomas Kuhn, The Structure of Scientific Revolutions sense.

To give you a sense of what these two philosophies are like, I can use my own early development as a programmer as an example. Like many programmers of my generation, my first language was a flavour of BASIC. BASIC was a good language to get your feet wet with programming, but too much of it was “magic”, in the sense that it buffered you from the real workings of the machine. It was fine for relatively simple programs, but once you got to more complex applications on a machine with memory measured in kilobytes, it didn’t really give you enough awareness or control of your environment to manage your resources.

The next step up from there was often assembly code or even raw machine language, and that looked like alphanumeric gibberish rather than comprehensible language.

So when I discovered C, it was a revelation. Here was a language that had a comprehensible syntax like BASIC but that really allowed you to specify exactly how you were using your resources. By this time, I had 1MB of RAM and clock speeds in the MHz, which seemed like a lot at the time, but for some of the applications I was interested in, you still had to juggle your resources to make this work, and C let you do that. With C, I went from being a dabbler in programming to being a programmer.

Moreover, C helped to give me an entrée into the world of assembly. The C compiler I used allowed me to generate assembly code as output, and I became very familiar with how my C code was translated to the machine. Sometimes I even wrote super-optimized functions in assembly for maximum speed and efficiency and called them from C.

But this was the first sign of trouble in paradise. Two things became apparent to me around this time.

The first was that the “C with assembly” approach wasn’t very portable or maintainable. Different versions of the 80x86 architecture, of DOS (and soon Windows), and of the compiler and associated libraries, made work done this way very fragile.

The second was that the low-level approach of C was very awkward for higher-level tasks such as GUI design, where what you wanted was reusable components that interacted on an event model. And this is the locus of the paradigm shift: from programmer as juggler of resources on a machine to programmer as designer of solutions in the problem space.

I think this dichotomy is a major one in software development, and it is not the exclusive domain of the C/C++ world. I’ve seen it play out in the Java world too.

Many technical people are (reasonably enough) oriented towards the technical side of things. They’re focused on the solution space. They have staked their professional mastery on understanding the arcane details of various languages, platforms, libraries and tools. This leads to a natural tendency to believe that the role of the programmer is to redefine the problem until it fits the bounds of the available solutions. They seek to turn the problem at hand into the proverbial nail for the hammer they have.

This is not a wholly bad phenomenon. In fact, in the kind of resource poor environment my C-programming self was contending with, this kind of shoe-horning was a necessity for being able to accomplish anything of value. And since resources are never completely unlimited, some of this thinking always ends up being essential on a technical project of any scope.

But the world changed as more computing resources became widely available, and richer, more user-friendly GUIs and application came to be the norm. Customers for software products and the software teams that built them started to want a more problem-focused approach to software. Instead of reworking the problem to fit the technical solution, there was more value in squeezing the technical solution into the mould of a more natural, conceptual model of the problem space being addressed.

And this is where Object-Oriented Programming (OOP) came in. The classes and objects that were the ++ to C gave developers a new set of abstractions to capture such a model in the source code. In principle, OOP was supposed to remove the focus from the implementation details of the application and place it on the modelling of the entities and activities associated with what the user wanted the application to do. Again, in principle, the programmer of an OOP language was supposed to cede control over some of the low-level resource allocation issues to the language implementation, and focus on the world according to the user. Some OOP languages did go quite a way down this road.

C++, however, made a different choice. It decided to try to fulfill both paradigms. Want to micro-manage resource usage? No problem: C++ includes C. Want to work at the higher-level abstraction of OOP? No problem: C++ has all the OOP features you want.

In some ways, C++ has been wildly successful in its goals. It succeeds in having all the features of both so that the developer is free to choose his approach. But it is exactly this blend that makes it such a nightmare from a design perspective.

As I was reading Meyers’ book, I was struck by how often he says of some C++ feature “there is a very simple rule for this in C++, with a few specific exceptions”, and the exceptions turn out to be mind melting and highly unintuitive to someone trying to avail himself of the “high-level” approach to their application. The language is designed to “help” you by doing certain things automatically, but it often doesn’t do the thing that seems to be most reasonable, since it doesn’t want to conflict with the freedom of the programmer to operate at a lower-level of control.

I think this illustrates a general principle of design: simple solutions can only exist for focused design philosophies. When you try to be “all things to all people” you necessarily end up with complicated, hard-to-manage solutions.

Having identified the “original sin” of C++, I think we still need to give it its due. Decades later, it is still going strong in application domains such as bleeding-edge games and graphics and in device-embedded controllers – domains where the spirit of scarce resources is still alive and well, and where the need exists for both the large-scale organizing principles of OOP and the “down-to-the-bare metal” optimization of resources.

Given that the pendulum in popular programming languages has swung back to the spirit of BASIC (interpreted languages that are “fun” and where resource allocation is mostly “magic”), I wonder if the end of Moore’s Law spells a return to the spirit of C++. If so, any successor to C++ will have to start where it left off and learn from both the bad and the good of its diabolical genius.

Lessons from Confucius for Software Development

In my previous post, I talked about lessons from Confucius that I think are still useful in the modern world, but there is a field of endeavour that I think can particularly benefit from understanding the Confucian worldview: software development.

At first glance, it seems highly unlikely that Confucianism might have some application to software development. The Confucian values of respect for the past, harmonious social relations, decorum and moral leadership don’t have an obvious affinity for an industry that is known for its focus on what is new and shiny, and which is stereotypically populated by raging individualists with a disregard for social standards.

But it is worth considering that the Confucians represent one of the earliest groups of knowledge workers in human history. They were trained in specialized knowledge for specialized tasks in a complex society, and they had a sense that their specialized knowledge made them an elite group in society.

An important concept in Confucian texts that bears on this is junzi. Etymologically, it means “ruler’s son, prince”, but already in Confucius’ time it is used more metaphorically, often translated into English as “superior man”, “gentleman”. The Yiddish term “mensch” has a similar ring. It represents what every Confucian was striving to be.

In spite of its etymology, junzi is one of the earliest-known notions of elite status that is not derived from the happenstance of one’s birth, such as being the literal son of a ruler, or a free man of Athens. The kind of “superior man” we are talking about here is defined entirely by his knowledge and skills, and his savoir faire in using them. Confucianism is a truly meritocratic system of thought, and a modern IT specialist can easily relate to such an ethic.

A second consideration is the fundamentally social nature of software development. If it ever truly existed, the age of the lone genius changing the world with his software is over. Any non-trivial software development these days involves a whole team of people with different specialties and knowledge, and building reliable and maintainable applications requires that these people work together as an effective and harmonious community.

The Confucian junzi has a sense of noblesse oblige, a sense that the status conferred on him by his knowledge and skills requires that he use them for the betterment of his society and to achieve collective goals. He is willing to lead and mentor new members of the fraternity of knowledge workers, and does this not by pontificating, but by providing an example through good practices.

The very fact that these kind of values are not what most of us think of when we consider software development suggests that there is still much benefit and advantage to be gained by nurturing them in software teams, and a team lead could do worse than to study the Analects of Confucius to prepare for the challenges they face.

After all, Confucius and his followers have several centuries of experience organizing and training knowledge workers to draw on.

What problem are you trying to solve?

Anyone who has worked in IT for any period of time is going to be familiar with the dilemma of choosing one particular solution over another. Should I learn language X or language Y? Should I use framework A or framework B for my next project? Should I use the Foo Architecture or the Bar Architecture? Is the Widget application better than the Grommet application?

Since sometimes it seems that the choice you make can have big costs in terms of your project’s success (and your future employability), these dilemmas can result in a lot of stress. It doesn’t help that for each solution you have some proponent telling you that if you don’t use their approach you are stupid and ugly (see Linus Torvalds talking about his source control tool ).

So what’s the smart way to decide?

I think the first step is to realize that you have the wrong question. Technology people by definition tend to be interested in technology (tools and solutions) and we tend, as a result, to give short shrift to the problems themselves, especially when they fall outside our domain. As a result, solutions pile up faster than you can evaluate them, and problems often get shoehorned in a pet solution. So what is the right question?

Do you remember those word problems we used to have to do in school? Trains leaving the station at certain times and speeds, Mary and Bill trying to divide up a pie, and all those hypothetical problems?

The secret of those problems was often just figuring out what the problem was in the simplest terms. Once you rephrased the question, you often knew exactly which technique from that week’s lesson to apply to solve it. You could just whack the problem with all the techniques you had learned, hoping to hit the right one, but you would probably take too much time or might even get an “answer” that was totally off base.

To make this brute force approach even less likely to succeed, the teachers and textbook sometimes threw in irrelevant details that sounded important but had nothing to do with the real solution, in the same way that mystery writers throw in red herrings to make it harder to guess whodunit.

So the only reliable way to solve these problems correctly was to learn to filter out the extraneous blather and find the pithiest answer to the question “What problem am I really trying to solve?”

So how can we use this to solve our solutions dilemma? Let’s start by recognizing that the real problem is not what solution to choose but rather resolving the problem the solution is for. Trying to choose a tool without explicitly defining the real problem is exactly like trying to solve those word problems by throwing all the techniques you know at them. You may get lucky, but more likely you will make a mess.

If you realize, having thought about it, that the problem you are trying to solve is to avoid having others think you are ugly and stupid, then there is an easy solution to the problem: give up on technology altogether. In technology there is always someone telling you are ugly and stupid because you didn’t choose the same solution as them, so you might as well get used to it. If you want to go one step further, reject all forms of ideological thinking on principle, and craft your own rational set of criteria by which to judge things.

If you discover instead that there is some substantive problem you want to solve, spend some time trying to understand the problem. Ask yourself what the essential characteristics of the problem are. Identify some things that might normally be of interest but don’t really apply in this case or have less importance than normal. The truth is that no solution is perfect, all solutions require trade offs, and the secret of success is focusing on the must-haves while sacrificing the merely nice-to-haves or, more often, even the not-quite-so-must-haves.

Sometimes I find it very useful to start sketching out my own solution from scratch. At some point you realize either that you have a nice simple solution and don’t need to go any further, or that, now that you have a better feel for the problem and its challenges, that you have clear criteria upon which to base the choice of an external solution.

Anyone who wants to get good at software architecture has to be willing to throw out conventional wisdom and make choices that fit the problem at hand, not the problem that advocates of particular solutions tell them they should have. Sometimes just spending some quality time with your specific problem clarifies what you really need and melts the solutions dilemma away.

Functional Programming is Not All or Nothing

I’ve noticed a flurry of discussion around the blogosphere lately about what is and is not a functional programming language and what is or is not stateful. After being interested in functional programming (FP) for many years, I’m glad that it is finally getting some mainstream attention. I think FP has some valuable lessons for all programmers, and that it provides a better default mindset for programming than traditional imperative programming.

However, a lot of the discussion seems to make the mistake of assuming that functional programming and statefulness are all-or-nothing properties, that either you have a total purity of statelessness or you have opened Pandora’s Box and let all the evils of state into your program or programming language. But the truth is that functional programming is best viewed as a style of programming with different default assumptions about state. And statefulness is best viewed as a continuum that varies along at least two dimensions — locality and monotonicity — both of which I will explain shortly.

What is state?

Before I do that, I had best describe in simple terms what state is. The easiest way to understand state is to imagine that I have given you a distinctive-looking box. If each time you open the box you find exactly the same contents, then the box can be said to be stateless. (This is also known as referential transparency). If each time you open the box the contents vary — the box could be empty sometimes, but full later, it could have different items in it or the same items with some new ones added — then it is stateful.

As you can see from this description, most boxes that we use in the real world are stateful, since it is very unusual for their contents to remain the same throughout their lifetime. However, if you are doing something complicated, say keeping financial records from past years in banker’s boxes, you might impose a discipline on your boxes, such as designating a particular box to contain only those financial records from April 2002, so that you can reliably find the records for a given month when you are looking for them.

Two styles of programming

Programming is like this as well. “Normal” imperative programming uses variables that can have changing values throughout the life of the program, which matches our understanding of boxes in the real world. FP takes the banker’s view of boxes. Since programs are very complex, and you want to be assured of finding what you expect, then the default assumption is that the contents of boxes will be the same throughout the life of the program.

So the key difference between a functional and an imperative style of programming is not whether there is any state at all in the program but where you start from. With imperative programming you start by assuming total changeability of all values you work with and have to add disciplines to get that state under control, whereas with FP you start from a place of statelessness and strategically loosen the reins in small amounts when you need to. The great strength of the FP approach is that it is much easier to maintain control by letting the genie out of the bottle slowly than by starting with the genie out of the bottle and trying to jam it back in.

Locality of State

The first dimension of state I want to discuss, locality, should be the easiest for programmers to understand but, for some reason, often causes confusion. Locality for state is just like scope for names in that it ranges from totally private to global, depending on how widely in your program it can be seen. A well-known property that makes state more local is encapsulation, when it is used to limit the visibility of a particular bit of state to a small part of the program. For encapsulation to accomplish this, though, it must actually hide the effects of the state from the parts of the program outside its scope.

To explain this, it is helpful to consider a common objection to functional programming languages by people new to the subject: how can a stateless program be built on a computer, which is an inherently stateful machine? The answer is that so long as the state is completely localized “under the hood”, it isn’t state at the next level up.

If we go back to the box analogy, let’s say that you have a very high-tech box that has the same contents every time you open it, but that, instead of the contents just being there when the box is closed, it actually disassembles the item when you close it and re-assembles it when you open the box again. The box is stateless because its contents never change from your point of view, even though its contents change when you can’t see inside.

So you can reduce the state in your program by ensuring that its various components look to each other as if they are stateless, even if internally they are implemented with large amounts of state, say for efficiency.

Monotonicity of State

Monotonicity of state is a little bit more subtle and probably less consciously familiar to programmers. Monotonic is a fancy way to say that once you add something, you can’t take it away. Using our boxes, the simplest example of monotonic state would be if, when you first open a box, you might find either nothing or something, but, as soon as you open the box and find something there, you must find the same something in the box every time you look in it thereafter. Upon finding the box empty, you could stop what you are doing and wait for something to show up in the box, after which you could be sure that it was not going to change, or you could take the opportunity to fill the box with something you want to fill it with, confident that someone else won’t be able to change it later. This is how dataflow variables work.

A more complex example of monotonic state would be a stream.
Once you have asked for the first value from the stream, it is as if you have filled the first box in the sequence, even if the box for the next item in the stream might still be empty.

With non-monotonic state, items can be added and taken away from the box at will, so that the box might have one thing in it the first time you open it, nothing in it the second time, and something totally new the third time. This is the kind of state that we normally think about when we talk about default imperative statefulness.

You can see that, though monotonic state is still state, it is much more predictable and manageable then full non-monotonic state, since you can always tell how much has changed from the last time you checked and you can be assured that what you found before will still be there. Furthermore, there is an explicit order to the changes in state that allows you to understand the history of operations, and thereby to make better automated decisions about what to do.

This is why the kind of message-passing concurrency, such as is found in Erlang is less stateful and relatively safer and more scalable than full shared-state concurrency, since an asynchronous message queue can be thought of as a monotonically-stateful stream.

Conclusion

While it would be much easier to talk about functional programming if statefulness were a nice black and white property that could be assigned to a particular language or program, I hope it is clear now that things are not quite so simple, and that degrees of state and “functionalness” could be present in any language or program. Obviously some languages and programs are going to make reduced state the default, and thus easier to implement, but the functional perspective can be applied to any language or program.

And making use of this perspective will help tame the complexity of software, especially in the face of concurrency.

Metaprogramming still needs programming discipline

This post will be a little bit more technical than previous posts, but I’ll try to keep it comprehensible to those who don’t already know what I’m talking about.

I want to talk about meta-programming. Meta-programming, in the most general sense, means making programs that produce programs or that change what existing programs do by altering the environment in which they operate. Writing an interpreter or a compiler for a programming language can be considered an example of meta-programming. Macros, monkey-patching and code generation are also examples. I would argue that the XML “configuration” files that haunt many Java frameworks, such as Spring, Hibernate, Struts, etc., are a form of meta-programming.

Now meta-programming is a very important idea, and I would go so far as to say that everyone who is at all serious about programming should learn about and understand meta-programming, even if only at a basic level. Meta-programming is often considered an advanced topic, and there certainly are advanced forms of it and advanced ideas that fall under its domain, but I think that anyone who is smart enough to program at all is smart enough to understand and perform basic meta-programming.

Now, since meta-programming is very important, has deep things to say about computation, and is intellectually stimulating, many people find it very exciting and even beautiful.

I will confess: I am one of those people. I have spent a non-trivial chunk of my leisure time throughout my adult life studying the semantics of programming languages, and all the supporting theories and math. And though knowing this stuff has been directly and indirectly useful in my software development career, I really did it because I enjoy it, and because I think it is beautiful and exciting.

Having said all these great things about meta-programming, whenever I see someone using meta-programming in a project, I get queasy. And that is because people often forget that just because meta-programming seems to let you go beyond the rules of “mere” programming, it still requires all the same discipline you would apply to any other kind of programming. For example, you still need to remember source code discipline and the enforcement of locality principles, such as modularity, the single-responsibility principle, encapsulation, don’t-repeat-yourself (DRY), and many others.

Take DRY for example. I have seen many programmers who would never tolerate cut-and-paste boilerplate in the source code blithely create megabytes worth of XML “configuration” files, or use a source code generator to do the same thing, even if there might be more acceptable alternatives with a bit of creativity.

Moreover, I think there are two kinds of locality that meta-programming should have that wouldn’t apply to single-level programming. First, meta-programming level code should be modularized away from the programming level code, and second, any domain-specific-languages (DSLs) or language variants created by the meta-programming should be clearly demarcated from the “normal” programming language (in their own source files if possible).

The reason for this is simple. Imagine if I started writing this post by alternating between English and some other language, say French. Some sentences or phrases in one language, some in the other. First of all, any member of the audience who is not fluent in both will probably be lost immediately. For those who are bilingual, there are many words that are spelled the same or very similarly in both languages, but may not have the same meaning, either subtly or not so subtly. So even if you are fluent in both, aside from the difficulty I’m adding by making you code switch, I may also cause you to misinterpret what I’m saying with similar or ambiguous words.

Just as the user of a programming library only wants to have to think about the interface to that library and not have to understand the gory details of how it actually works, the consumers of meta-programming “enhancements” need to be insulated from having to understand the details of how it works under the hood. This is where non-local meta-programming, such as reckless global monkey-patching, can really mess things up.

So anyone who wants to keep meta-programming beautiful should use the same judgment, taste and discipline that an experienced programmer would apply to keep any other type of programming beautiful. Otherwise, it can morph into something very, very ugly.

Dancing Monkeys

Many years ago, before I started my software development career, I had a job call-center representative for a national automobile association. This included a twelve-hour shift by myself on Sundays.

The Sunday shift was a bit of a grind. It could be a long, silent, boring wait for calls that never came, or if the weather was inclement somewhere on the other side of the country, it could be very busy handling a spate of calls all alone.

During the normal workweek, when there were a bunch of us working, we would pass the slow times doing paperwork. By Sunday, when I was often the only person in the building, the paperwork was usually all done, so to pass the time I took to listening to the radio. As soon as the phone rang I would shut the radio off and pick up. There was no doubt in my mind that I was doing my job fully and faithfully, in spite of the discomfort and difficulty of the job.

One Sunday, the Vice President of the company decided to come to the office to catch up on some work. At some point, he wandered back to the call-center where it was a slow day, and I was sitting there by myself listening to the radio. He may have greeted me, or asked me how busy I was, but he didn’t stay long and he didn’t say anything significant.

However, on Monday I heard from my boss that he had complained that I was listening to the radio instead of working.

This was my first significant experience of a phenomenon that I call “Dancing Monkeys”: the tendency of arms-length leadership to prefer situations where everyone “looks busy” or is “hopping to”, even when the expected effort would have no additional effect or might even be counterproductive.

There are many reasons for this phenomenon some pernicious and some benign. The pernicious ones that we will all have seen at some time or another is pure ego-tripping: “I’m the top monkey here, so you lesser monkeys start dancing!”

A common, more benign reason is lack of understanding of the work domain under observation. Software development is particularly prone to this one.
For example, more than once I’ve heard some executive complain that he stopped by the development team area and no one was typing, with the implication being that no work was getting done, since as every non-technical person knows programming is all about lines per minute of code typed.

Another common scenario is that inevitable day when, under some unforeseen deadline pressure, some executive asks the development manager when he is going to institute overtime to help meet the deadline. Because of course, as any non-programmer knows, there is no degradation to the quality of software development when the team is exhausted, since typing lines of code is a purely mechanical process.

To come back to my call-center job, effective execution of my duties required that when I got a run of long, stressful calls, I had the energy and focus needed to solve the clients’ problems efficiently and effectively. Anyone who has ever made a service call to a droning zombie service rep knows what happens when someone doing that job lets their energy reserve run down.

Listening to the radio helped me recharge between calls, kept my morale up on quiet days and improved my performance. Doing mindless, unnecessary busy-work would have sapped that energy and morale. Asking my boss to find work for me to do that was meaningful but would not interrupt my real work would have just increased her workload without increasing the efficiency or effectiveness of our department.

So that Vice President had to ask himself: was making himself feel better by getting a dancing monkey really in the best interest of his company?

Explaining through Skillful Means

In Buddhist philosophy, there is a concept whose Sanskrit name is upaya. This is often translated as "skillful means", though, as with many other ancient philosophical terms, it is hard to translate exactly and has many interpretations.

To illustrate one useful interpretation, I'll start by explaining a little bit about Buddhism. If you boil Buddhism down to a one-sentence summary, it is about reducing your suffering (and that of the world) by living a moderate life and practicing meditation.

As simple as this sounds, there is a lot of theory behind it, covering — for example — what it means to live a moderate life, why that helps reduce suffering, what good meditation is and how it works, and a host of other questions.

The theory and practice of Buddhism is in fact so complex that it is usually taken for granted that it takes years of study and practice to really understand it. To make matters worse, there are many different opinions about what kind of study and practice, and how much.

But most of the concepts can be understood at first in a simplified way, and since all students of any complex skill have to start somewhere, people often start with these simplified forms that aren’t quite right to a more advanced student.

For example, there is an uncomplicated form of Buddhism called Pure Land, that more or less believes that if you faithfully chant a particular mantra, you will be reborn in a special paradise. You can still see in this belief some of the elements in my one-sentence summary above: chanting a mantra is a form of meditation, being reborn in a paradise is a reduction of suffering, and faithfully practicing a good habit is a rudimentary form of living a disciplined life.

So even though a “more advanced” student of Buddhism might roll their eyes at the simplicity of this approach, they would have to admit that practicing it was still an advancement in the direction of the aims of Buddhism, and therefore better than no understanding of Buddhist principles.

This, in a nutshell, is what upaya means: it is OK to let someone have a “wrong” idea about something you are explaining to them, so long as it takes them one step closer to understanding.

What does this have to do with software development?

Software development, boiled down, is about solving practical problems by producing reliable applications with maintainable code bases. (All of these things, incidentally, also reduce suffering. ;-) )

There is a complex skill set and complex methodologies that must be learned to accomplish this task. It is taken for granted (at least by some people) that mastering them takes years of study and practice, but there are many different opinions about what kind of study and practice, and how much.

But many concepts have a simplified version, and since all learners have to start somewhere, you often see versions of good practices that a more advanced practitioner would see as not quite right.

For example, a developer with less experience — or perhaps just different experience — might chant a mantra such as “always document”, “never document”, “always design before you program”, “never design before your program”, or “paradigm/language/library/framework X is always better than Y”. These mantras might strike you as over-simplified or incorrect based on your experience (especially when you favour an opposite version of the chant. ;-) ).

You can still see that they share the core concern of solving the practical problem reliably and maintainably. Maybe it is OK to let them be “wrong” for now, so long as they are engaged with the ultimate concerns of software development and are using the mantra to get one step closer to understanding.

When mentoring developers as a team lead or when introducing new ideas, techniques and methodologies to your peers or to an on-line community, it can be hard to determine when to hold a hard line, to nitpick or to qualify and when not to. Often it is best to introduce a simplified form of the idea and let them run with it for a while, trusting that if they share the same goals they will come to a deeper understanding on their own and that they will ask for more input when they need it.

Sometimes the best policy is to let others be "wrong" for the time being.

Be Willing to See Failure

There is an old saying that I quite like: "If you’re never failing, you aren't trying."

The simplest way to read this adage is as an exhortation to take more risks, to stop playing it safe and undertake bigger and scarier projects.

An equally valuable second reading is to take this as a salve to failure -- it is OK to have failed or had a set back because it means you were trying something challenging.

But there is a subtler reading that I want to explore that comes into play when you are trying, and you doseem to be succeeding, that addresses a more dangerous form of failure: the failure to perceive failure.

Anyone who has spent any time doing development projects (for software or otherwise), has probably heard of, if not actively participated in, a project that went along just fine for months, only to burst into flame as some final deadline approached. And by going along just fine, I mean that people were trying: meetings were held, decisions were made, mountains of code were written, documents filled the shelves, metrics were monitored.

But mysteriously, as the looming deadline approached, all kinds of critical problems seemingly popped out of nowhere, delaying delivery, running up costs, creating a sudden, unforeseen crisis.

(Sounds a lot like our current world-wide financial crisis, doesn't it?)

So what went wrong?

Rarely do problems just pop out of nowhere. More often, we simply become aware of them suddenly. And sometimes that’s because we didn’t want to be aware of them, because we wanted to focus on the success instead of the failure.

Unfortunately, I think this is a major challenge in our North American cultural temperament. (People from other locales can assess this for their own situation.)

We tend to value optimism, the power of positive thinking, the positive spin to the point where any talk of real risks or real failings tends to sound like pessimism or cattiness. We tend to assume that those who question a person, project or process that seems to be succeeding are guilty of sour grapes or defeatism.

So a sub-prime mortgage crisis can "sneak up on us" and bring down an Alice-in-Wonderland financial system, and a project that was "going along fine" can suddenly go off the rails.

The solution to this problem is to accept that failures and problems are, as the adage suggests, a necessary side effect of trying to accomplish something, and that success doesn’t come from ignoring problems but from finding creative and effective solutions to solve or mitigate them. And once we’ve accepted this, we can change our team culture and practices to help flush problems and failures out of the bushes sooner.

In software development, there are a number of techniques that are gaining wide acceptance, such as iterative delivery, open communication between clients and teams, continuous integration, test-driven development, and others, that solve this very problem.

These techniques all work by creating more frequent opportunities to put a project through its paces, so that weaknesses, misunderstandings and oversights are spotted sooner, and so that successes have really proven themselves, and can be safely built upon.

A good rule of thumb is that if you are aren’t routinely finding unexpected problems or failures when using such techniques, you probably aren’t using them rigorously enough.

Which leads us back to the opening adage, but maybe in this context it should be changed to: "If you’re never failing, you aren't looking."

Minimize Locally, Succeed Globally

My first full-time programmer position was in the research institute of a nationally-known hospital, building database applications for research data and administration.

I have often joked about that time that I was a one-man development cycle, since I had was responsible for the process from beginning to end.

I was the "sales guy", drumming up business with internal clients. I was the business analyst, modeling the often complicated problem domain. I was also the programmer, the system administrator, and the tech support guy.

To complicate things further, there were a fair number of projects, some small, some dormant, but also some that were quite large, complex and active, and which grew more so over time. Eventually, I also added the roles of project manager and dev team lead when a family of the projects coalesced into a behemoth.

As you can imagine, having so many roles on so many projects was a bit of a tightrope walk. To survive, I had to constantly ask myself: "What is the simplest and most reliable way I can do this?"

As the "sales guy" (we did internal billing), there was no benefit to making grandiose promises to drum up more sales than I could safely deliver. I could only handle so many projects anyway. And failing to deliver on a couple projects would quickly have poisoned the well.

As the business analyst, I had no reason to nurture technicolor dreams for a particular project. I was going to have to implement and support it as well, so I had to just solve the priority problems elegantly and efficiently, with no frills.

As the programmer, I had no motivation to demonstrate my genius with complex algorithms and impenetrable code, or by layering on the latest tech toys. I needed my code to be easy to understand and fix, because I wasn't going to have a lot of time to build it or maintain it. Simplicity and clarity also helped with stability and reliability, which was good, since if the application wasn't stable and reliable, I was going to generate more tech support pain than I could deal with.

And since I didn't have time to be a tech writer and software trainer as well, I had to sacrifice a slick look or gee-whiz features to make simple, clearly-labeled interfaces that the most technophobic administrative assistant would be able to figure out reasonably well in 10 minutes or less. Otherwise, I was going to add the role of professional hand-holder to my roster.

Of course, in spite of my best efforts, none of this worked out 100% of the time, and I dreamed of the day when I would be using the heavy-duty tech on the big projects with the big boys, where most of my roles would be fulfilled by other people. And I got my wish soon enough.

But I discovered a hidden danger with larger projects where most of the participants don't have sight of the whole process: that each role maximizes itself to the detriment of the outcome.

The sales rep makes unfulfillable promises to get his sales. The business analyst gold-plates and over-specifies the requirements. The programmers over-engineer with too many libraries, frameworks, cool features and clever code. The tech support guys, who, to be fair, do the most suffering on the front lines when things go awry, try to toss some of their woes back over the wall to the developers.

And if this tendency isn't actively countered, instead of building bigger, better software, they start to build software that costs more, is less reliable, and less usable.

So now that I've seen both sides, I think maybe necessity had put me on to something way back then: being minimalist with the requirements, efforts and costs at each step in the process in order to optimize the workings of the whole.

And I think an organization that manages to foster that ethic through good communication and good leadership will produce the best software for the lowest cost in time and money.

Software is "Philosophy Made Manifest"

What does the name "Philosophy Made Manifest" have to do with software development? Here is the answer:

Though I have worked most of my adult life in software development, my original academic background is in Linguistics and East Asian Studies. Since this seems like a bit of a leap, I have often been asked what, if anything, I gleaned from those studies that helped me in my career.

My answer is: a surprisingly large amount. In fact, sometimes I think these things give me a big advantage over computer science or software engineering graduates.

The influence of Linguistics on my thinking about software development probably deserves its own post, but to explain "Philosophy Made Manifest", the most relevant area of my past studies was the philosophy portion of East Asian Studies: Buddhism, Taoism, Confucianism and others.

When you first dip into the surface of these systems of thought, some of the stuff sounds pretty far out there, especially from a modern western point of view, but once you get past the surface, it turns out that these philosophies are the end result of a long line of astute observers and thinkers who were concerned with very real and practical problems: how to live harmoniously with other people, how to set up effective government, how to live a happy and productive life, and many others.

In order to succeed as a student of these philosophies, you have to get good at figuring out the how these people in far away times and places understand the world. And to make their ideas relevant, you have to get good at seeing beyond the foreign trappings of the ancient vocabulary and forgotten social customs, to get at the heart of what they have to say about the unchanging realities of human life.

Once you've mastered this, you can then translate these ideas into more immediately applicable forms, ones that speak to the vocabulary and social customs of the here and now.

Which, perhaps surprisingly, leads us to software development.

Effective software development is not only about the clever use of technology, it is a profoundly human activity that requires an understanding of how people work together and how users, business leaders, developers and others think about the world.

At its core, software development is synthesizing a philosophy of some problem domain, based on these different world-views, and implementing it as a concrete, practical tool that supports and enables the activities of that problem domain.

This is how software development is "philosophy made manifest".