Is it stigmergy?
by Peter Small
If you search for references to stigmergy or stigmergic systems on the Web, you'll find many articles about social insects, e.g. ants and ant colonies. You'll find articles on building robots and creating autonomous software agents. There will be references to agents and robots communicating with each other, cooperating and swarming.
This is all very interesting stuff, but how many of these articles and references are really about stigmergy?
Stigmergy is the phenomenon of order and organization emerging as a result of individuals interacting with and changing a landscape or environment. If the environment does not play a dominant role, or, if the environment is not changing dynamically, then it isn't stigmergy.
Open or closed stigmergic system?
Stigmergy, in its original form, applied to open systems. This is where the environment or landscape is separate from and independent of the individuals changing it.
In recent years, the concept of stigmergy has been extended to include environments that are created by the individuals themselves, without interacting with any external environment. This is a closed stigmergic system.
The difference is important because these are totally different kinds of system.
For Web based applications, only the original, open stigmergic system is valid. This is because the environment is the Web site and the individuals changing it - the visitors to the site - are separate and independent.
Limitations of closed systems
Stigmergy was extended to include closed systems because they can be created in their entirety by computer programs. Although this has led to many useful applications, it has blurred the line between what is stigmergy and what is not. For example, robots or agents communicating directly with each other is not necessarily stigmergy. It may be incorporated in a stigmergic system, but stigmergy is about individuals doing something to the environment, not to each other.
Similarly, although swarming is associated with stigmergy, strictly speaking it is not stigmergy in its original sense. Swarming is quite a separate phenomenon, specific to closed systems.
As a result of this blurring of the concept, you'll find people claiming to be working on stigmergic systems who are concentrating solely upon the activities and interactions of individuals - the equivalent of the worker ants. They'll design simulated version of ants; they'll give these ants functions and processing power; they'll provide a means for them to communicate and cooperate with each other. They'll then put these agents to work on information. But, this alone is not stigmergy. This is an object oriented environment. Certainly there can be complexity and emergent outcomes, but essentially they are closed systems where the ultimate performance of the system is determined by algorithms.
This is not to say these kind of applications are not valid in their own right. For example, working with autonomous agents, that use neural nets or sophisticated algorithms for cognitive intelligence, can provide very many useful functions. Mostly though, these are based upon pattern recognition techniques, where the objective or the required solution has to be clearly defined.
To go further than this - to solve for problems where the solutions are ambiguous, or, where the information content is controversial, unpredictable, contains uncertainty, unknowns or unknowables - requires an organic system that is open to interact with a human environment.
Here is where the difference between open and closed stigmergic systems becomes critical; a closed system is computer generated and cannot be organic because of the inherent limitations of computer technology.
Overcoming the limitations of computers
The difference between an open organic system and a closed computer generated system is best explained by considering the limitations of digital computers.
In theory, computers should to be able to model any system. In practice they can't because they are limited by their design philosophy to being controlled by algorithmic programs. Without humans in the loop, it is impossible for a computer program to evolve and adapt to an external environment in the way biological organisms can.
Ever since computers first came into existence, they have been compared with biological systems. Initially, computers were associated with the working of the human brain. This gave birth to the field of artificial intelligence. Everyone had high expectations that this would lead to computers becoming intelligent. But, even after decades of research and countless millions of dollars of investment, these hopes were never realized in any practical way.
Why did these efforts fail? What was the mysterious missing element preventing computers being able to emulate the efficiencies of biological systems?
Computers can be made to remember, analyse and process huge amounts of data. Their computing power can far exceed that of humans. But, they are totally dependent upon programs supplied by humans, which inhibits them from exhibiting any real intelligence of their own.
Certainly computers can be programmed to make sophisticated decisions, but, they cannot deal with surprises, unpredictable events and unknowns that have not first been anticipated and allowed for by a human designer. Similarly, computer systems do not have a freedom to evolve their own activities and functionality. They cannot change and adapt to changing conditions of their own volition.
These deficiencies are not due to a lack of programming expertise. They are fundamental limitations imposed by the design philosophy of computers - the von Neumann architecture and the Turing machine - which limits computers only to following instructions.
Although this does not detract from the fact that the computer is a massively powerful and indispensable tool, computers have never been able to acquire the subtleties of organization found in most biological systems.
However, we are now in a position to correct for this deficiency - not by changing the design of computers or by inventing any revolutionary new programming techniques - but by placing computers into external conceptual frameworks. These frameworks - one of which is stigmergy - can enhance the power of computing by superimposing the self-organizing characteristics of biological systems.
The role of agents
It was not until the last two decades of the twentieth century that it was realised biological systems function quite differently from computers. It was found they don't rely upon imposed order for their structure and organization, but by emergent changes of state that arise spontaneously within dynamic systems. Such changes cannot be explained by logic, only by the theories of chaos and complexity.
The key to understanding how order and organization can be teased out of chaotic, complex systems is in being able to appreciate the role of emergence. This is not easy. Emergence cannot be planned or designed: it just happens. This makes it very difficult to give credence to the idea that such an unpredictable phenomenon can be considered as a reliable component of a design strategy. How can you trust in a system, when it seems to rely only on hope and faith?
This is why the concept of stigmergy has become so important. It provides the proof and a rational explanation as to how emergence can bring about self-organization and order, without it having to be imposed by explicit human instruction. The route to this understanding is in being able to visualize the computer as being a component of a wider system, as opposed to creating a system within the computer itself.
In other words, stigmergy is a phenomenon that happens outside of the computer environment. The computer simply provides the mechanisms that allow it to occur.
When stigmergy is used on the Web, a computer is used to present a dynamic landscape in the form of a Web site. The structure is not fixed: it is flexible. Visitors shape the structure and content by interacting with it. They do this through agents that make this interaction possible.
The most important thing to understand here is that the self-organization and emergent order that emerges is not a result of anything the agents do. The agents are merely the tools provided for the visitors to use to enable the stigmergic, self-organization to emerge. We call these facilities tools and agents; in biological systems they are called genes.
Note: The tutorial "Harnessing the power of stigmergy" explains how this is put into practice.