This post will be a little bit more technical than previous posts, but I’ll try to keep it comprehensible to those who don’t already know what I’m talking about.
I want to talk about meta-programming. Meta-programming, in the most general sense, means making programs that produce programs or that change what existing programs do by altering the environment in which they operate. Writing an interpreter or a compiler for a programming language can be considered an example of meta-programming. Macros, monkey-patching and code generation are also examples. I would argue that the XML “configuration” files that haunt many Java frameworks, such as Spring, Hibernate, Struts, etc., are a form of meta-programming.
Now meta-programming is a very important idea, and I would go so far as to say that everyone who is at all serious about programming should learn about and understand meta-programming, even if only at a basic level. Meta-programming is often considered an advanced topic, and there certainly are advanced forms of it and advanced ideas that fall under its domain, but I think that anyone who is smart enough to program at all is smart enough to understand and perform basic meta-programming.
Now, since meta-programming is very important, has deep things to say about computation, and is intellectually stimulating, many people find it very exciting and even beautiful.
I will confess: I am one of those people. I have spent a non-trivial chunk of my leisure time throughout my adult life studying the semantics of programming languages, and all the supporting theories and math. And though knowing this stuff has been directly and indirectly useful in my software development career, I really did it because I enjoy it, and because I think it is beautiful and exciting.
Having said all these great things about meta-programming, whenever I see someone using meta-programming in a project, I get queasy. And that is because people often forget that just because meta-programming seems to let you go beyond the rules of “mere” programming, it still requires all the same discipline you would apply to any other kind of programming. For example, you still need to remember source code discipline and the enforcement of locality principles, such as modularity, the single-responsibility principle, encapsulation, don’t-repeat-yourself (DRY), and many others.
Take DRY for example. I have seen many programmers who would never tolerate cut-and-paste boilerplate in the source code blithely create megabytes worth of XML “configuration” files, or use a source code generator to do the same thing, even if there might be more acceptable alternatives with a bit of creativity.
Moreover, I think there are two kinds of locality that meta-programming should have that wouldn’t apply to single-level programming. First, meta-programming level code should be modularized away from the programming level code, and second, any domain-specific-languages (DSLs) or language variants created by the meta-programming should be clearly demarcated from the “normal” programming language (in their own source files if possible).
The reason for this is simple. Imagine if I started writing this post by alternating between English and some other language, say French. Some sentences or phrases in one language, some in the other. First of all, any member of the audience who is not fluent in both will probably be lost immediately. For those who are bilingual, there are many words that are spelled the same or very similarly in both languages, but may not have the same meaning, either subtly or not so subtly. So even if you are fluent in both, aside from the difficulty I’m adding by making you code switch, I may also cause you to misinterpret what I’m saying with similar or ambiguous words.
Just as the user of a programming library only wants to have to think about the interface to that library and not have to understand the gory details of how it actually works, the consumers of meta-programming “enhancements” need to be insulated from having to understand the details of how it works under the hood. This is where non-local meta-programming, such as reckless global monkey-patching, can really mess things up.
So anyone who wants to keep meta-programming beautiful should use the same judgment, taste and discipline that an experienced programmer would apply to keep any other type of programming beautiful. Otherwise, it can morph into something very, very ugly.
May 3, 2009
Good to Great
I finally got around to reading the book Good to Great. I first heard about it in early 2002 in the introductory speech of the incoming president at an organization I was in the process of leaving. I was quite impressed with what he had to say about leadership and organizational greatness and made a note to read the book as soon as it came out in paperback. (Hardcovers use up too much shelf space, which is at a premium in my home, so I tend to avoid buying them.) Since it is still in hardcover all these years later, I gave up and borrowed a copy from my brother.
There is a lot of criticism of the book online (here, here and here), but I think most of it misses the point. It doesn’t matter if you are happy with the experimental model the book was based on, or if the companies profiled didn’t stay good stock picks forever after the book came out, or if the advice Collins gives boils down to “obvious” suggestions.
The best business books tell you in an organized and compelling way what you already know to be true from your own experience. No business book can predict the future or give you a sure fire recipe for success, and Good to Great doesn’t pretend to anyway, so why do the critics expect it to?
What I did find in this book was a well-written and thoughtful explanation of the only reliable strategy I know of to accomplish any undertaking: for a disciplined group of people to pursue a focused goal in a determined manner, while being willing to see failure.
Now one can say “Hey, that’s obvious!”. But unfortunately, the obvious is more often observed in the breech than in the practice. The path of least resistance in many organizations is to let ineffective or obstructive people stick around way too long, to let the day-to-day crises and tempests-in-a-teapot derail their long-term goals, and to rationalize or ignore failures.
If Good to Great manages to inspire people by reminding them to struggle against these tendencies and to set a higher standard toward greatness, I think it deserves its place on the best-seller list.
There is a lot of criticism of the book online (here, here and here), but I think most of it misses the point. It doesn’t matter if you are happy with the experimental model the book was based on, or if the companies profiled didn’t stay good stock picks forever after the book came out, or if the advice Collins gives boils down to “obvious” suggestions.
The best business books tell you in an organized and compelling way what you already know to be true from your own experience. No business book can predict the future or give you a sure fire recipe for success, and Good to Great doesn’t pretend to anyway, so why do the critics expect it to?
What I did find in this book was a well-written and thoughtful explanation of the only reliable strategy I know of to accomplish any undertaking: for a disciplined group of people to pursue a focused goal in a determined manner, while being willing to see failure.
Now one can say “Hey, that’s obvious!”. But unfortunately, the obvious is more often observed in the breech than in the practice. The path of least resistance in many organizations is to let ineffective or obstructive people stick around way too long, to let the day-to-day crises and tempests-in-a-teapot derail their long-term goals, and to rationalize or ignore failures.
If Good to Great manages to inspire people by reminding them to struggle against these tendencies and to set a higher standard toward greatness, I think it deserves its place on the best-seller list.
May 2, 2009
Dancing Monkeys
Many years ago, before I started my software development career, I had a job call-center representative for a national automobile association. This included a twelve-hour shift by myself on Sundays.
The Sunday shift was a bit of a grind. It could be a long, silent, boring wait for calls that never came, or if the weather was inclement somewhere on the other side of the country, it could be very busy handling a spate of calls all alone.
During the normal workweek, when there were a bunch of us working, we would pass the slow times doing paperwork. By Sunday, when I was often the only person in the building, the paperwork was usually all done, so to pass the time I took to listening to the radio. As soon as the phone rang I would shut the radio off and pick up. There was no doubt in my mind that I was doing my job fully and faithfully, in spite of the discomfort and difficulty of the job.
One Sunday, the Vice President of the company decided to come to the office to catch up on some work. At some point, he wandered back to the call-center where it was a slow day, and I was sitting there by myself listening to the radio. He may have greeted me, or asked me how busy I was, but he didn’t stay long and he didn’t say anything significant.
However, on Monday I heard from my boss that he had complained that I was listening to the radio instead of working.
This was my first significant experience of a phenomenon that I call “Dancing Monkeys”: the tendency of arms-length leadership to prefer situations where everyone “looks busy” or is “hopping to”, even when the expected effort would have no additional effect or might even be counterproductive.
There are many reasons for this phenomenon some pernicious and some benign. The pernicious ones that we will all have seen at some time or another is pure ego-tripping: “I’m the top monkey here, so you lesser monkeys start dancing!”
A common, more benign reason is lack of understanding of the work domain under observation. Software development is particularly prone to this one.
For example, more than once I’ve heard some executive complain that he stopped by the development team area and no one was typing, with the implication being that no work was getting done, since as every non-technical person knows programming is all about lines per minute of code typed.
Another common scenario is that inevitable day when, under some unforeseen deadline pressure, some executive asks the development manager when he is going to institute overtime to help meet the deadline. Because of course, as any non-programmer knows, there is no degradation to the quality of software development when the team is exhausted, since typing lines of code is a purely mechanical process.
To come back to my call-center job, effective execution of my duties required that when I got a run of long, stressful calls, I had the energy and focus needed to solve the clients’ problems efficiently and effectively. Anyone who has ever made a service call to a droning zombie service rep knows what happens when someone doing that job lets their energy reserve run down.
Listening to the radio helped me recharge between calls, kept my morale up on quiet days and improved my performance. Doing mindless, unnecessary busy-work would have sapped that energy and morale. Asking my boss to find work for me to do that was meaningful but would not interrupt my real work would have just increased her workload without increasing the efficiency or effectiveness of our department.
So that Vice President had to ask himself: was making himself feel better by getting a dancing monkey really in the best interest of his company?
The Sunday shift was a bit of a grind. It could be a long, silent, boring wait for calls that never came, or if the weather was inclement somewhere on the other side of the country, it could be very busy handling a spate of calls all alone.
During the normal workweek, when there were a bunch of us working, we would pass the slow times doing paperwork. By Sunday, when I was often the only person in the building, the paperwork was usually all done, so to pass the time I took to listening to the radio. As soon as the phone rang I would shut the radio off and pick up. There was no doubt in my mind that I was doing my job fully and faithfully, in spite of the discomfort and difficulty of the job.
One Sunday, the Vice President of the company decided to come to the office to catch up on some work. At some point, he wandered back to the call-center where it was a slow day, and I was sitting there by myself listening to the radio. He may have greeted me, or asked me how busy I was, but he didn’t stay long and he didn’t say anything significant.
However, on Monday I heard from my boss that he had complained that I was listening to the radio instead of working.
This was my first significant experience of a phenomenon that I call “Dancing Monkeys”: the tendency of arms-length leadership to prefer situations where everyone “looks busy” or is “hopping to”, even when the expected effort would have no additional effect or might even be counterproductive.
There are many reasons for this phenomenon some pernicious and some benign. The pernicious ones that we will all have seen at some time or another is pure ego-tripping: “I’m the top monkey here, so you lesser monkeys start dancing!”
A common, more benign reason is lack of understanding of the work domain under observation. Software development is particularly prone to this one.
For example, more than once I’ve heard some executive complain that he stopped by the development team area and no one was typing, with the implication being that no work was getting done, since as every non-technical person knows programming is all about lines per minute of code typed.
Another common scenario is that inevitable day when, under some unforeseen deadline pressure, some executive asks the development manager when he is going to institute overtime to help meet the deadline. Because of course, as any non-programmer knows, there is no degradation to the quality of software development when the team is exhausted, since typing lines of code is a purely mechanical process.
To come back to my call-center job, effective execution of my duties required that when I got a run of long, stressful calls, I had the energy and focus needed to solve the clients’ problems efficiently and effectively. Anyone who has ever made a service call to a droning zombie service rep knows what happens when someone doing that job lets their energy reserve run down.
Listening to the radio helped me recharge between calls, kept my morale up on quiet days and improved my performance. Doing mindless, unnecessary busy-work would have sapped that energy and morale. Asking my boss to find work for me to do that was meaningful but would not interrupt my real work would have just increased her workload without increasing the efficiency or effectiveness of our department.
So that Vice President had to ask himself: was making himself feel better by getting a dancing monkey really in the best interest of his company?
April 2, 2009
Ideology vs Principles
Ideologies are strangely seductive.
Ideologies promise a sure-fire recipe to solve all problems. They often give you a single value to maximize, and equally often tell you who to blame.
Got a problem? Easy. Just trust the free market. Or increase government spending. Or blame the wealthy. Or blame the poor.
Whichever solution an ideology offers, that is supposed to be the first solution to apply whenever a problem arises, and if that doesn't work you're supposed to do apply it again until it does work.
Given the complexity of some of the problems that face us in our daily lives as individuals and even more so in our collective life as a society, it is easy to see the allure of straightforward solutions you don't have to think about too much.
But history has shown us over and over again that ideologies are fairly poor at solving problems in the short run, and in the long run usually create worse problems of their own.
If you want effective problem solving, you're better off with principles rather than ideology.
A set of principles is like an ideology in that it expresses values upon which to base decisions, but principles can't be applied unconditionally. Principles compete with each other, forcing you to make trade-off decisions. The exact trade-offs have to be worked out differently for each instance of a problem.
Ideology is like the imaginary fifties family where everyone listens to Dad and the kids never compete for attention. Principles are like a real family, where you love everybody, but you can't always make everybody happy.
The amount of work required to implement a decision is about the same for both ideological and principled approaches is. The real difference is in the amount of certainty you have, both before you choose a solution, and after you've made a choice.
In a principled approach, you always have to wonder if you could have gotten the balance of competing values better, and you may have to constantly rejig the balance as new situations develop. With an ideology, you always have it right, even if you aren't getting the results you want.
As nice a feeling as it is to be certain, if you really care about outcomes, uncertainty has the benefit that it keeps you more closely in touch with the realities of your problem. It makes you listen more attentively to the feedback from your solution. You are much more likely to succeed in the end, even if paradoxically you are less sure at any one point in time that you have found the right solution.
Abandoning certainty leads to greater confidence of real success.
This idea has so many applications to software development, project management, government, and other governance and systems thinking domains, I'll leave elaboration to future posts.
Ideologies promise a sure-fire recipe to solve all problems. They often give you a single value to maximize, and equally often tell you who to blame.
Got a problem? Easy. Just trust the free market. Or increase government spending. Or blame the wealthy. Or blame the poor.
Whichever solution an ideology offers, that is supposed to be the first solution to apply whenever a problem arises, and if that doesn't work you're supposed to do apply it again until it does work.
Given the complexity of some of the problems that face us in our daily lives as individuals and even more so in our collective life as a society, it is easy to see the allure of straightforward solutions you don't have to think about too much.
But history has shown us over and over again that ideologies are fairly poor at solving problems in the short run, and in the long run usually create worse problems of their own.
If you want effective problem solving, you're better off with principles rather than ideology.
A set of principles is like an ideology in that it expresses values upon which to base decisions, but principles can't be applied unconditionally. Principles compete with each other, forcing you to make trade-off decisions. The exact trade-offs have to be worked out differently for each instance of a problem.
Ideology is like the imaginary fifties family where everyone listens to Dad and the kids never compete for attention. Principles are like a real family, where you love everybody, but you can't always make everybody happy.
The amount of work required to implement a decision is about the same for both ideological and principled approaches is. The real difference is in the amount of certainty you have, both before you choose a solution, and after you've made a choice.
In a principled approach, you always have to wonder if you could have gotten the balance of competing values better, and you may have to constantly rejig the balance as new situations develop. With an ideology, you always have it right, even if you aren't getting the results you want.
As nice a feeling as it is to be certain, if you really care about outcomes, uncertainty has the benefit that it keeps you more closely in touch with the realities of your problem. It makes you listen more attentively to the feedback from your solution. You are much more likely to succeed in the end, even if paradoxically you are less sure at any one point in time that you have found the right solution.
Abandoning certainty leads to greater confidence of real success.
This idea has so many applications to software development, project management, government, and other governance and systems thinking domains, I'll leave elaboration to future posts.
Subscribe to:
Posts (Atom)