Dealing with a complex environment

We have considered where conventional, Industrial Age business thinking might be inadequate to cope with the Internet and how it might lead to wrong attitudes for competing in the environment of e-commerce. This begs the questions: "What is a better way of thinking and what other strategies might be more appropriate?". To answer these questions, the temptation is to try to get a perspective by looking at the big picture: looking to find some superior strategy that will ensure success. But, this is the method of deduction used in the Industrial Age: looking at the general in order to deduce the particular.

With complexity, looking at the big picture is impossibly vague and confusing. The inability, to be able to comprehend the big picture of a complex environment makes it unlikely that a good strategy could be seen even if it were there. A strategy might be found that works for a while, but, it is almost certain that change and competition will render that strategy ineffective very quickly. Successes, quickly attract the herd and on the Internet the herd can move very fast.

Take a moment to consider what it means when something is described as a complex environment. It doesn't just mean that there are too many things to know or remember. Complexity describes a system of interrelated parts, where each part has a direct or indirect effect on all other parts. Interacting with a complex environment is not possible without changing that environment and some of its parts through the very process of the interaction itself. Trying to get a fix on a complex environment is very difficult; snapshot pictures are no help at all. The only way to get to grips with a complex environment is to become part of it and then to see what happens when you do something in it.

When Apple brought out the Macintosh computer in 1984, it heralded a new era in computer programming: the age of object oriented systems. Besides introducing a more friendly user interface, the Macintosh included an operating system which provided a novel environment - an object oriented environment - within which developers could create applications. It was a complex environment where complex systems could be created and controlled.

At that time, in the early 1980's, most conventional computer programs were designed specifically for a single special purpose. They were highly structured, designed top down and hierarchically organised. They were algorithmic and deterministic, in the sense that every command resulted in a rigidly defined sequence of events. It was almost inconceivable at that time for programs to be anything other than dedicated and well regimented systems. This restricted them to being inflexible: neither re-configurable nor adaptable

Because of its friendly user interface, the Macintosh was immediately perceived as a computer for the masses; a computer where very little knowledge or training was needed to be able to use it. When the computer programmers of the day examined the way in which the operating system was designed - as a collection of independent little modules - they viewed it in much the same way as the interface: an artificially created environment that had been designed to be simple to use.

They interpreted the ready made modules as a design strategy that would allow novice programmers to knock up quick and dirty applications. With its functions all separated out into independent modules which could be pieced together in any order, the Macintosh operating system seemed to be designed more along the lines of a Lego set than as a programming environment where professional programmers could create serious applications.

What the programmers saw, when they examined the Macintosh operating system, was many built-in functions that could be used by any applications that had a need for them. These functions could be called into use simply by sending simple messages to them: nothing more complicated than a message name and perhaps a few parameters to specify how the function should perform. In this way, much of the programming a developer needed for an application was already available, in the form of modules. All that was needed to use them was a communication infra structure - an organisation - to link selected appropriate modules together.

In the beginning, many experienced programmers were completely thrown by this. It seemed to be the wrong way to go about things: starting with the objects and then adding the organising structure afterwards. Yet, despite this apparent back to front strategy of design, it was found to work remarkably well. In fact it worked so well and was so superior to the structured, top down conventional approach to design that MicroSoft was forced to come up with a similar system.

This example is a good place to start looking for clues as to how to handle and control complexity. The Macintosh operating system is typical of a complex environment to which we can address the question "How did the first developers handle the problem of designing within this operating system?". This inductive approach might lead us to some useful conclusions.

An important clue comes from the way the programmers regarded the four massive technical manuals that Apple provided for application developers. A cynic once described them this way: "To understand any part of these manuals, you would first need to have understood all the rest of what is in the manuals". This, in many ways, sums up the attitude necessary to understand complex systems. No single part can be understood without reference to all other parts.

Quite obviously, programmers did succeed in getting to understand and control the Macintosh operating system, despite the seeming impossibility of being able to make any sense of the manuals. This is evidenced by the wealth of useful and sophisticated programs that have emerged over the years. What isn't so obvious is the way they eventually came to understand and control this complex system: when the manuals were so incomprehensible.

The trick was that they didn't even attempt to sit down with the manual to try to understand them. They used them to provide solutions. Whenever they wanted their applications to do something they looked in the manual to see if they could find some module that would do it for them. In this way, they could create software is a modular form. Much of it consisting of the pre-designed modules present in the operating system to which they then added their own custom built modules to provide any function the existing modules couldn't provide. All modules could then be loosely coupled together within an organising framework which simply routed messages between the modules.

In this way, the developers conquered the complexity of the Macintosh environment to create complexities of their own. Some of the independent, individual developers didn't even need a detailed plan, they could start with one module and then add other modules, one at a time, creating and extending a message routing infrastructure as they went along.

This was a radically new way to design software, adding new modules into an environment as and when they are needed. Such a strategy could allow an evolutionary design approach, where software products grow and evolve from the bottom up. This proved to be a highly effective way in which to approach design in the complex environment of the Apple Macintosh and suggests that this same strategy could be used by e-commerce traders to make effective and imaginative use of the Internet.

Such an approach to dealing with complexity is not startlingly new or revolutionary. It is the way natural systems have evolved to cope with competition and complex environments. There are numerable examples, at all levels of biological organisation, which exhibit a similar bottom up, modular approach to design; even, such approaches can be found genetically programmed into the human mind.

Take the case of language. Like the Macintosh operating system, it consist of a myriad of small modules: words. To a non English speaking person, a dictionary of English words is as confusing as that set of original Macintosh computer manuals were to the developers. All words in a dictionary are described by other words, so, a cynic could easily make the observation that no word can be understood without knowing all the other words.

As everyone knows, it is impossible to learn a language simply by looking at a dictionary. Language is a complex environment and cannot be understood in terms of a "directory" (or a search engine). Each of thousands of words can be combined and configured in an almost infinite number of ways to describe almost every tangible and non tangible entity and thought in existence. To an alien, trying to observe the way in which humans communicate with each other (looking at the big picture), it would appear to involve an awesome complexity which is seemingly impossible to unravel. Yet, a two year old child doesn't have a problem with this at all. It starts off by learning one or two words, then adding a few more, then puts a few of them together in combination. Within a very short time the child is talking and before the age of ten has probably mastered the complexity sufficiently well to have a reasonably intelligent conversation with an adult.

This process is similar to that of the Macintosh developers. They started out by learning what a few of the modules did; then stringing them together. Their way of progressively finding new modules to use and incorporating into a growing design is almost identical to the way in which a small child builds up and uses a vocabulary of words. It is the process of starting with the particulars and building up into the general: the process of induction.

Now if we look at this process in the abstract and apply it to any kind of complex environment, we can see how complexity can be mastered and made to grow and evolve in a controlled way. It is not necessary to understand the whole system as long as you can think in terms of small modular units that interact and send messages to each other within a communication environment. It is not necessary to design any modules if they are already in existence. The only time new modules will be needed is if the system is being developed or expanded in a totally new and novel direction.

Applying this abstraction to the Internet and the World Wide Web would see a communication network already in place and a host of versatile modules available and already connected up (the myriad of software applications and components being offered for use in e-business and e-commerce solutions). As an e-commerce strategist, looking to set up a system for e-commerce the problem is no different from that of the application developers for the Macintosh. There is a complex environment in place, which is filled full of handy little modules that can be put together to create any kind of customised system required. E-commerce business people don't have to understand the whole system, or, most parts of it. All they need do is to select the modules most appropriate for the application they have in mind and connect them together by messages.

Certainly, there will be small gaps in such systems where a suitable module cannot be found, but, this serves to inspire design: perhaps revealing an opening for a new commercial product or service which then becomes another available module in the ever growing sea of modules. In this way a multitude of different kinds of module becomes available for anyone to use, manipulate and organise as they see fit.

Let's now look at what these modules need to consist of. At first thoughts, the modules will comprise principally of software. It is through the software that a module will be able to receive and process messages. Through software the module will be able to carry out and perform its function. The function itself will be an operation performed by software. It is tempting then, to think of the modules available to an e-commerce trader as being solely software.

However, upon reflection, the modules consist of much more than just software. The software is actually only a part of a wider organisation which includes humans. In this wider concept of a module, the software appears as just the visible part of the module's system, much as a mound of ice protruding from the sea is the visible manifestation of an iceberg.

That human part is the design and development, the back-up, the service, the marketing and all manner of different personnel that might be involved in the module's creation, function, efficiency and future performance. The actual software itself is merely a commodity. The part of the module that an e-commerce trader is really dealing with is the software module's human part .

If we begin to extrapolate out from this system of modules, which include humans, we can begin to get a fleeting glimpse of what the whole of the Internet is about. It is a gargantuan sea of modules that are interacting with each other in a myriad of different ways. These modules are available, usually at a price, for any e-commerce trader to use and combine for any purpose whatsoever. However, the utilisation of the modules will involve direct or indirect communication between humans because the invisible supporting system for the module is a fundamental part of the module's value within a system.