Thursday, December 21, 2006

Openness, transparency and efficiency in IT

In Open Source - More Than Code Bill Barr discuss if you really can run IT efficiently while maintaining full transparency to stakeholders. I have also been looking at the effects of transparency.

The spirit of SOX for example seem to require that business have full visibility into everything going on, down to a very technical level. In my opinion business should. First of all, they are paying for the IT systems and their development. Secondly, the more involved they become the bigger is the opportunity that the IT system serves the business need. Following SOX and other rules can then just be seen as another minor reason for driving transparency. Extra paperwork and meetings is often seen as a main drawback, and with the sad state of documentation I have seen on a number of cases this is most certainly true, but not in my opinion valid, since there is usually some value in improving documentation anyway.

The problem with transparency and involvement starts when business stakeholders interpret this as a mandate to lead and manage the IT development. This often result in micromanagement, having everything grind to a halt. The problems may be small or big, but in the end they all depend on the personalities and driving forces of the stakeholders you are working with. Keeping these stakeholders in the dark is often seen as a solution, but that can create even more serious antagonism further down the road. I think that a better solution is to make them aware of other stakeholder views and needs, in terms of costs and business risks, and the fact that they may not necessarily have all the facts or the knowledge required.

I would not say that transparency or openness need to be in conflict with efficiency, but there is a good chance that they will be. In a deep hierarchical "chain of command" type organization where departmental silos are not discussing with each other I think you can have either openness, transparency or efficiency. Achieving any two in any combination would come at an extremely high price, and all three is probably next to impossible achieving.

That said, I still consider involvement of individuals, openness for change and transparency to what is being done as the key to efficiency. The individuals in a lean organization, once set with a task, may well know how to achieve a solution as long as the organization is truly open to this kind of involvement. Transparency guarantees that if some initial condition change or some theory is later found wrong, there are plenty of opportunity spotting and correcting the error. The key i think is that everybody need to agree on what the actual problem that is to be solved is and listen to each other.

Improving efficiency is easy to if you don't need to consider any side effects. In the end, IT department efficiency should not be considered as the ultimate goal, and trying to optimize on a project or code development level can in worst case only reach a local optimum. Optimizing for business efficiency would be far better in many cases.