Are "best practices" really any good?

Why did somebody pick the term "Best Practices" to describe something very often constituting a local optimum in efficiency space? I find it very hard to believe that all approaches described as "best practices" are equally good or even fulfill the label "best practice" at all. Lets for example examine two approaches that seems to be fairly popular, and seem to at least think they represent "best practices". The first, patterns, is fairly technical in its offering and close to the code while the second, ITIL is more management related.

Patterns have a clear area where they are applicable limiting their scope. Patterns are can be used, copied and most importantly improved by anyone. Patterns are in a sense open for improvement. Patterns seems to have spread rapidly to use into real life applications, some patterns obviously more than others. Embracing patterns has been made fairly easy. Inefficient code can be reimplemented using patterns by a technique called refactoring. Some tools are even claimed to automatically refactor for you at the press of a button.

ITIL is described as a collection of best practices, but is it really as efficient as it can be? I would state that it is too early to say although ITIL has been around for some years. ITIL is however only in the process of being broadly adopted. ITIL seems more proprietary and does not seem to welcome feedback to the same extent. This limits the people working on improving the offering to only those that work for the agency. In stead ITIL seems more focused on providing training and certification at a price. Migration to using ITIL processes is left to the organization perusing excellence. ITIL also seems fairly vague in terms of limiting applicability.

A "best practice" should logically be something that is coming fairly close to being as efficient as possible. Many patterns fulfill this requirement. ITIL i would say is a good practice, at least in some respect, but in others seem fall short of being a "best". I don't think the difference in technical dept between the two examples i picked is fully responsible for all differences.

Some are often baffled by the fact that a bureaucratic approach like ITIL is even considered a good approach. Everything however depend on the problem you have at hand. If you need to only manage a single server used for a non critical application used by a single user to serve his own blog entries ITIL is probably just as overkill as writing this application using every existing pattern ever documented. For a major fortune 500 enterprise in Europe ITIL may however be a very good approach to fulfill legally mandated governance of financial applications.

What i mainly disappointed in is the failure of "best practice" promoters to recognize and efficiently communicate the known limits. One major key to efficiency in IT lies in knowing what is suitable. In a similar way one of the major reasons for failure is belief that one approach would be suitable when it in reality is not. Marketing seldom gets the limit across. In the mean while until there is a non subjective declaration of content for each approach the best solution is to be skeptical of all "best practices" and carefully evaluate their merits. And yes, i would prefer that the "best practice" promoters would in stead start to talk about good operating practice.

Efficiency in IT

Defining efficiency in IT is harder than defining inefficiency. Inefficiency could for example be defined as when resources/time/effort is wasted in IT functions.

There is nowadays much talk about efficiently aligning IT with business. IT functions should be getting the right things done in an agile manner as requested by the business. There is some merit to this, but mostly its just empty talk. If business have no clue what they want, or if business have no way of communicating what they need you will not either get any efficient aligned IT function for your business.

You must to a large extent know what to do from all stakeholders point of view in a balanced way. Some unfortunately tend to understand that business alignment only mandate implementation of the often arbitrary requested features. Truth is that a successful efficient system must fulfill needs raised by all stakeholders regardless if they are business representatives, technical deployer or help desk support professional. Some requirements are more important than others, but letting one part decide on priorities is not wise.

This does not mean that ALL functional or nonfunctional requirements should be implemented. It is equally important to be able to disregard all not well thought out requirements. Featuritis is a dicease where often business are forced to prematurely state all requirements up front (as done for waterfall projects) to be sure that they don't need to be involved in any cumbersome change management process intended to limit changes more than fulfilling any other need.

So how do you know what to do? One thing you could try is to ask me, but ill just answer that I don't know... The key, if there is any, is to involve everybody in your quest for efficiency. This has been shown to work in factory settings, so it should work in any setting. The people doing the work that will be affected by the change should know what can be done better and where it will lead, but getting the ideas out into the open will take a bit more than just asking for comments in a email message sent to all. An other thing to rely on is experience. If you don't have experience you have a good opportunity to acquire some by trial and error.

An useful guide to efficiency is to keep the spirit of Occam's razor in mind, or try tp do everything as simple as possible, but no more simple. Excessive complexity leads to waste, or aspects that will never be used that will be in the way of further development.

Nature has a couple of important lesson for us to learn. Always start small and evolve from there. Kill of bad ideas often and early, before they become multimillion failures bringing the whole company down. Everything does not need to be perfect, just good enough will do. Interaction with stakeholders and suggestions for small changes should be welcomed at every opportunity. The ability to test out new small features in a cheap way without substantial risk can be one source of efficiency. Doing small nondisruptive changes quickly is why agile movement is pushing. Note however that any agile approach by itself will not guarantee efficiency.

What is a limit?

According to common sense it is simple to state that one thing is better than something else, at least as long as there is some aspect of the thing that can be measured. Stating that one approach to software engineering is better than an other does however not belong to the category of things that can be analysed in a simple way and does not seem to lend itself to any common sense approach. The problem is that there is not only one aspect to measure from which you can draw conclusions, but lots and lots of different aspects playing together, either making an approach good or bad. There are also lots and lots of "parameters" that can be changed and changing one thing will influence the others, even in unpredictable ways. Further more, changing one parameter any way possible may improve efficiency.

Theoretically there must exist a very best way to achieve something, but we don't know how good it is or where in the “parameter space” it is at. With any selected approach you can do better or worse, and finding the best way to archive a goal can be done by trial and error.

For example, by obtaining the latest development tools and making tool usage mandatory to a team of developers used to notepad / vi, delivery is delayed and amount of shipped bugs increase. The task for the team has become learning in stead of creating software. After the price is paid, and the team have done the learning and got used to the new tools the situation is totally different. Mundane tasks are automated by the tools, and tool specific quality assurance measures reduce the number of bugs. In this example you could say that by using notepad /vi the efficiency was trapped in a local maximum value. Getting out of this local maximum could have been difficult, but once done the efficiency increased. There are presumably many other things that could have been done, costing more or less, but before trying them out you can not really know if they would have worked better than the selected approach. There is neither any guarantee that the same approach will work on the next project.

Comparing to when scientists calculate the 3-D conformation of macro molecules like proteins and smaller molecules that bind to proteins, there is a simulation named Monte Carlo method being used, where you by repeatedly and randomly altering energy levels can get a good estimate of what the overall lowest energy conformation of the macro molecule system is. Repeated optimisation cycles and random alteration to the parameters will not guarantee that all possible conformations are evaluated but it does a pretty good job estimating the best conformation.

I don't know of any fully successful approach that would lend itself to study software engineering where you could experimentally or computational control the environment. We only do things once under normal conditions, and afterwards you have learned some more. We can't repeatedly create the same software over and over again altering just one factor and analysing the efficiency. In RAD Races you have different teams competing to create something working and useful, but the teams are constituted by individual members whose skill and motivation can vary by a large extent. Statistical approaches with many parties, contributing results over a long period of time could also be a good possibility, but getting objective and impartial data would be difficult. Further more the vast number of possible values for the vast number of parameters is way too large for any kind of exhaustive analysis.

If productivity is rising exponentially over time, it can be very difficult to pinpoint the most efficient approach. Getting stuck in a local optimum is very easy, and that is in my opinion the biggest problem for many organisations today.

A. A.

What is efficiency anyway?

Efficiency can be defined as the ratio of output to input for any given system. A better explanation could be skillfullness of avoiding waste. Google has a lot of other definitions as well.

The strictly economic aspect of efficiency is not always appropriate, although there may actually exist people who think so. See for example Efficiency in Art as an illustration of what i mean. I would argue that we need to take a broader view of what constitutes efficiency in software engineering. Clearly economic aspects are important, but suitability of functionality to the need as it changes over time should be considered more important.

An other problem with stating what is efficient in software engineering is that it is very difficult to draw the line of what is the boundary when you measure input and output. Some waste today could turn out to reduce waste later on.

If you draw your boundary too tight, and just focus on the next object or line of code you need to write, you can easily get stuck in local optima, which probably is very far off from the efficiency limit. If you make your boundary too wide, and try to embrace the world you will not either end up with a very good end result.

A. A.

Is there a limit to what software can do?

Grady Booch wrote an article for Rational Edge a few years ago explaining the physical limitations of what software can do. The article is still available, but now on IBM developerWorks (http://www-128.ibm.com/developerworks/rational/library/2082.html).

To kind of summarize, there are some things in software engineering that are possible to achieve and some that are impossible. Of the possible ones we currently know how to achieve some while we have no clue about the rest. Of the ones we know how to achieve, some we can actually afford to, while other are too expensive to achieve, leaving in between a grey area where we can either make a reasonable estimate or we will be able to afford in a year or two. We are also likely to be restricted by our management and organisations further by imposed rules and regulations to be followed, some for a well motivated reason, others for more obscure beliefs.

Further more, we unintentionally add errors and magnify any uncertainties in our understanding of the requirements. Some of the tasks in SW engineering are also are less motivating than others, further making the end result more uneven.

I somehow don't think that the hard limits to software engineering efficiency are anywhere near. I feel that the way we go about doing things are not anywhere near the level we could achieve

My intention is to further describe what is limiting our ability to solve IT related problems
as efficiently as possible in upcoming posts to this blog.