How Stigmergy Is Applied

In the 'Crowd-sourced Info Towns' Project

The essence of a Crowd-sourced Info Town is the bringing together - in one place - all the best pages on the web that provide knowledge relating to a particular subject area.

To fully appreciate the thinking behind this project, it may help to be aware of some of the theory upon which it is based.

Learning from Nature

Biological systems have evolved ingenious solutions to the same kind of problems encountered in information technology.

Understanding how nature solves these problems allows us to use these solutions ourselves, to create highly efficient informational websites.

Most applicable of nature's clever solutions is the concept of 'Stigmergy', the strategy used by ants and termites to build complex nests and create efficient information systems.

Stigmergy as a strategy

Stigmergy is not like a product, a service or an application. It isn't even like a computer program. It is a very clever strategy used by nature to get colonies of insects to self-organize, tell each other where to find food, create sophisticated messaging systems and build complex architectural structures.

We can use this same clever strategy on the Web, to get people to collaborate in the exchange and sharing of information. The strategy works well for brainless insects, but it works infinitely better for intelligent humans equipped with computers.

What are stigmergic systems?

Stigmergic systems combine information technology with stigmergy and other ingenious computing structures found in nature - to create self-organizing Web sites and low cost knowledge management systems.

Stigmergy alone is not sufficient because it can lead to uncontrolled growth.

Nature counteracts this by using the techniques of reproduction and selection, which control the evolution of organic systems to make them increasingly efficient and continuosly adaptable to whatever challenges and changes are thrown up by any dynamic environment.

We can use a similar controlling mechanism to optimise an information system.

Current project

The "Crowd-sourced Info Towns" project has been created to provide a 'proof of concept' demonstration of a stigmergic system in action. This aims to create collaborative information sources in the form of towns that hold pointers to web-sites using crowd-sourcing techniques.

To fully appreciate how nature's clever solutions are embedded in this project, we need to go deeper into the theory.

Systems - an insight

Seeing beyond the trees

For untold generations, the tendency for some types of toadstool to grow in rings was a mystery: an unexplainable phenomenon. It is easy to see how this was associated with elves and faires. What else could explain this structural organization?

Of course, we now know the reason. The toadstools aren't separate organisms, they are the fruiting bodies of a single, unseen fungus that lives beneath the soil. The circle marks its outer perimeter, from where the fruiting bodies (the toadstools) emerge.

Systems are often like this. You can see the interfaces, where the system interacts with people, but you cannot see the invisible organization that lies beneath.

During the development of the "Crowd-sourced Info Towns" project, many people commented on the parts they could see. They would exclaim:

"What's special about this? It's something that any high-school student can get together with a simple database program".

One critic commented, in an early review of the site:

"It's a very neat, simple idea, but it is dressed up in a plethora of unnecessary technical jargon to make it look important".

They had been looking at the toadstools and were oblivious of the complex organism hidden from view

The point is that the town, streets and facades are the very simple database-like stuctures that form the visible parts of an intangible organization that has no visible form. Of course it is easy to create these structures in a database, but you can only do this once you have the data to work with.

The idea behind using a stigmergic system is to get the information there in the first place - not in dealing with it once it is there.

The theory has nothing at all to do with the storing, ordering and categorizing of the information. It is about creating a system that enables people to collaborate.

The theoretical stuff isn't anything to do with what happens in a computer. It is about what is happening to the system as a whole: how people react to the system; what makes them collaborate in such a system; how the system evolves to become increasingly more efficient.

The trick is to step back from what you see, to get a view of the system in its entirety. Don't look at the toadstools. Look at the hidden fungus.

Stigmergy originated from the study of ants. But, the understanding came not from looking at the behaviour of individual ants, but in seeing an ant colony as an entity in its own right. Not hundreds of thousands of individual insects, but a two kilogram creature, whose structure consists of several hundred thousand components.

What is stigmergy?

The discovery

Stigmergy was originally discovered and named in the 1950's by Pierre-Paul Grasse, a biologist studying ants and termites. He was intrigued to learn how these virtually brainless creatures could create highly sophisticated messaging systems and build extremely complex architectural structures. It was a mystifying puzzle that nobody had ever been able to explain.

What he uncovered defied rational explanation. There were no plans, organization or control built into the brains or genes of the ants. There were no leaders or organisers. The sophisticated frameworks and complex structures were emerging spontaneously.

The understanding eventually came about through knowledge gained in the study of the self-organizing characteristics of complex systems.

Complexity arises in stigmergic systems because individuals interact with the environment - not with each other.

They interact with the environment by making changes to it. These changes affect the way further changes are made.

This is the key to understanding Stigmergy: ants, termites (and people) are genetically programmed to react to features in a common environment causing them to make changes. In humans, this takes the form of inbuilt emotional impulses, based on a complex combination of needs to see improvement, cooperate or to compete.

This gives rise to a positive feedback effect, where information feeds upon information (much the same effect as when conversations can take unpredictable directions according to the way people respond to each other's comments).

It has only recently been recognized that the concept of stigmergy - with its ability to create increasingly complex structures - provides a plausible explanation as to how and why the Internet and the World Wide Web have self-organized to become increasingly more complex. Peope see what is on the web and become inspired to add to it, change it or improve it.

What is an evolutionary strategy?

An evolutionary strategy is a process of continuous reproduction, trials and selections.

This takes the form of continuously producing new generations, and where the best features of generations are selected for and carried forward and the worst features are left behind. This results in systems that become increasingly more efficient and more organized.

An evolutionary strategy has to be applied to stigmergic systems to ensure that they grow and evolve efficiently.

Crowd-sourced Info Towns Use Successive Generations

Using an evolutionary strategy to develop a Web site is quite different from the way Web sites are usually developed because it is not planned - it emerges.

This technique is particularly suited for designing websites to attract visitors interested in a particular subject area.

Understanding how stigmergy, self-organization, selection and evolution work for ants, allows us to develop highly efficient information sharing systems for niche subject areas.

Evolutionary selection achieved by submissions and voting

Crowd-sourced info towns create an effect similar to producing new generations by frequently updating the towns. Each update involves getting suggestions and recommendations from users and a voting system of curators that adds or removes content.

This enables the information in a town to become increasingly efficient and allows adaptation to take place to cope with changing trends and technological advances.

Self-organization is unplanned organization that emerges from an open system of interacting components. The system can be thought of as lifting itself up by its boot straps.

This is the way a self-organizing Web site can be developed.

Seeing the environment as a whole

A website is the visible part of a wider system that includes: visitors to the website; the people who make changes to the website; the people who choose the information that appears on the Web site; search engines; the whole of the World Wide Web and the Internet.

All of these various factors have an influence on the way in which a Web site will self-organize.

It is impossible to even begin to try to understand how all these different influences have an effect on the organization that emerges.

This is why it is necessary to use an evolutionary strategy to develop a stigmergic system website over a continuous series of rebuilds in the form of both updates and architectural changes.

With an evolutionary strategy, you don't plan or predict how the system will perform. You observe what the system is doing then make changes to see if it performs better. Using a controlled trial and error process, the system has to be steered along a path of increasing order and organization.

This is how Nature achieves self-organization and although it might seem to be a crazy way to design a Web site it is surprizingly effective.

Town Visitors Are The Driving Force

Crowd-sourced Info Towns have changes and selection provided by people who visit the town (website). Individually they propose changes but collectively (curators) they decide upon which changes go through to next generations (updates of the town).

The paradox

It may be difficult to see how a Web site can be self-organizing, if it is dependent upon humans making decisions and making changes. But, these people are themselves components of the system.

They are not planning what the system will do, but responding to what the system is doing. Their decisions and changes are influenced as much by the system as their actions are influencing the system.

Chaos and complexity

A system is described as complex when variables are interdependent: when changing one has a knock-on effect that changes others. This can result in a system becoming unstable, small changes bringing about sudden and unpredictable wholesale change. The system can oscillate wildly or settle down into a different steady state.

Weather is complex, so are business or competitive environments. They are unpredictable. Difficult or impossible to control.

With the advent of powerful computers in the 1960's, complex systems could be modeled and studied. Out of these studies, dynamic patterns emerged. These patterns showed that by altering the variables of a complex system it could be made to switch between a number of steady states. These steady states became known as 'strange attractors' and the strategic use of this knowledge has become known as 'Chaos Theory'.

It then became apparent that nature was making use of this phenomenon in its evolutionary strategy.

By changing the variables of a complex system (a biological organism) in a variety of different ways, it could selectively proceed through a sequence of different steady states towards increased organization and efficiency. It is this strategy that causes species and ecosystems to become increasingly more organized and efficient.

This same strategy can be used in complex business environments like the Web. There is no planning or prediction involved. All it takes is to engineer a highly adaptable infrastructure. This can then be used to create many small changes and take advantage of any favorable outcomes that emerge.

This is the way nature does it - with considerable success. And we can copy this strategy.

The philosophy and mind-set for this approach is totally at odds with conventional business strategies that seek to bring about order and control.

Instead of trying to enforce order onto an erratic and unpredictable environment, chaos theory provides the conceptual models that allow us to take advantage of change and to capitalize on favorable emergent developments.

Our current project "Crowd-sourced Info Towns" is based upon these principles.

See details in the side panel.