Copied over from the original site
This page was written for the original site in 2003. It explains a project that existed then.
Although this project is no longer in existence, much of the thinking behind it has relevance to our current projects, which were born out of these earlier projects.
Designing a stigmergic Kempelen Box
A stigmergic Kempelen Box is a twenty-first century version of Wolfgang von Kempelen's "Mechanical Turk", an eighteenth century chess playing machine. However, unlike Kempelen's original machine, this latter day version doesn't hold real people. It holds agents that represent people, and these agents are programmed to carry answers to specific questions in their memories.
At a first impression, this looks a simplistic kind of novelty program - the kind of application that anybody could quickly put together with the aid of some elementary database knowledge - and it would be easy to dismiss this as a triviality. However, there is much more to this than meets the eye.
Forget the presentation and the programming aspects for the moment and consider the two main features: the agents who represent the people in the box and the questions they can answer.
Agents as a group object
At a superficial level, a Kempelen Box is a place on the Web where agents hang out. People can go to this place to communicate with the agents (through their browsers) to ask them questions. Agents will respond according to whatever answers are programmed into their memories.
There seems to be nothing very remarkable about this. But, if you go above the level of the individual agents to consider them as a group, you'll notice something rather interesting. Look at diagram 1. See how a number of individuals acting as a group can become an entity in its own right. It is at this level that the real action happens.
Figure 1 - When individuals form into a group, this group acquires an identity of its own, quite independent of the individuals
Technical note:
The structure of a Kempelen box is similar to the kind of conventional structures you'll find in most database or XML applications. It consists of a number of items that each have properties and a method allowing these properties to be compared easily.
Nothing very special here, but it can become something special if you stop thinking in terms of arrays, fields in a database, or XML tab hierarchies, and see it simply as a black box containing agents that answer questions. Then go one step further and take this black box out of any kind of database and put it on the Web as an isolated independent entity (Note: this can easily be done by creating it out of javascript modules - incorporating any necessary database functions within the javascript programming structure).
Once you do this, you free yourself from any metaphors associated with conventional data processing programming structures and can start to ask yourself what you want to do with this black box and how you can make use of it. If you stop to think about what's in the box and how it can be programmed in a different way, you've lost the plot because conventional database processing concepts are out of place here.
Once you've made the paradigm shift, to see these agents as a group object, it immediately become clear that these are not simply passive groupings, but dynamic objects with the ability to evolve and become increasingly more efficient. This is illustrated in figure 2.
Figure 2 - A group can become more effective and efficient by continually replacing the weakest members
Now look again at the Kempelen box in the link on the left. From diagram 2, you can soon realize how the quality of the answers given by the agents could become better and better as time goes by if the quality of the agents are continuously improved.
This is the level to concentrate upon. The creation of a Kempelen Box is not about presentation, programming techniques or language preferences...
...it is about a strategy to continuously improve the quality of the agents.
The questions
Agent memories are created by getting experts to answer a questionnaire. The questions in this questionnaire are critical to the way in which the Kempelen Box performs. As people cannot ask the box any question they like, these questions will determine the kind of knowledge that can be obtained from the box. If the questions are trivial, the functionality of the box will be trivial. If they are too specific, the box will allow only a limited range of information to be accessed. If they are too vague the information coming from the box will be too varied.
With a little thought, you soon realize that the questions are effectively defining the nature of the box. It can be made to focus in on any subject area. It can provide precise detail or broad guidance. The choice of questions can also extend beyond that of simply supplying information, it can bring people into direct contact with the worlds of the people whom the agents represent. The questions can be chosen to provide new directions, places to make contacts and find rich sources of information that people may never before have known existed.
Note: systems designed to organize information, often make the assumption that people know what they are looking for. The structuring, categorization and hierarchical arrangements are designed to take people straight to the information they need as quickly and efficiently as possible. But, the Web has taught us that going directly to the information we seek isn't necessarily such a good thing. The act of searching for information can be even more rewarding than finding it immediately. A search process can become a voyage of discovery, with the goal that is sought often becoming of lesser importance than the goal actually achieved. This philosophy can be incorporated into the functionality of the box by a suitable choice of questions.
Clearly, the questions that can be asked in a questionnaire are not something that can be determined logically. Their composition will be a creative process, requiring flair and imagination. For example, the questions might be determined by thinking about the kind of questions that might be asked by people talking to a group of experts in a cafe next to a library. This was the theme of the questions used in the example Kempelen Box and is illustrated in figure 3.
Figure 3 - A questionnaire can be devized by imagining the kind of questions that might be asked in a cafe situated next to a library
This is the kind of scenario that you should carry in your mind when thinking about what is inside a Kempelen box. It isn't just a mechanical device to efficiently deliver answers to a list of questions. It is a representation of a dynamic situation.
Figure 4 - Designing a questionnaire is a creative process, where you have to imagine a real life scene
Copying the way ants do it
If the Kempelen Box is to evolve to become increasingly efficient, the agents must continually change. But it is more than just the replacement of agents that has to be considered; the answers they give to questions must also change and even the questions themselves. For this, theKempelen Box has to be organic in nature, able to adapt and respond to the way people want to use the box.
To arrange for this, a database mind set is not appropriate. Neither is the logical structuring of XML. The only way this can be handled satisfactorily is to employ the methods used by biological systems. They have evolved to cope efficiently with this kind of situation by allowing systems to self-organize.
Such a system is used by ant colonies to build complex nests and create sophisticated information networks. They don't use a central database, instead they have genes that enable them to create a virtual database in their environment. They also have genes that allow it to read and write into it. This is the kind of strategy that is suitable for creating and maintaining the organic structure of a Kempelen Box and is the method used in the example.
Note: see the tutorial 'Harnessing the power of stigmergy' in the 'Research' section of this site.
Designing for change
Anyone can set easily up a Kempelen Box. It can be programmed using a variety of different techniques. But, this isn't the challenge. The challenge is to get them working efficiently. After all, if you put a Kempelen Box up on any Web site, how do you get people to come along and create agents? How do you know what are the right set of questions? How do you get the agents, the questions and the answers to continuously improve?
This would represent awesome problems for a marketing or PR team, but Nature has come up with some elegant solutions. She solves for these kinds of problems with a combination of two strategies: Evolution and Stigmergy.
Evolution involves creating a system that continuously changes for the better. This can be applied to a Web site.
Note: see the tutorial 'Organic Web site prototyping' in the 'Research' section of this site.
Stigmegy is the phenomenon whereby people react to a situation by changing it.
Note: see the tutorial 'Self-organization' in the 'Research' section of this site.
Using a different mind set
Everyone is aware that the first time a Web site is put up it is unlikely to be immediately successful Sure, you can do something novel or gimmicky to get visitors to the site, but Web sites have a purpose and to get a Web site to fulfil this primary objective is far from easy. Experience has taught us that a Web site has to be monitored, constantly changed, reconfigured and continuously tweaked before it is performing efficiently. A long process that can take months or even years.
Kempelen Boxes are no different. They have to attract visitors. They have to attract people who'll create agents. They have to evolve so that they are asking the right kind of questions.
Where can you get information to guide a suitable strategy to make this process more efficient?
Once again we can turn to Nature to supply us with some answers. But, these answers you won't get from most biology textbooks because the answers have only come about in recent years and have not yet made it into mainstream biology courses. The answers are to be found in the arcane theories of complexity and chaos.
Complex systems - and any system involving people comes under this category - are inherently unstable. But nature has found a way of controlling this instability and turning it into an advantage for the purpose of evolution. This strategy can provide invaluable insights for the purposes of creating successful Web sites and Kempelen Boxes.
It is this strategy that will form the basis for developing Web sites and Kempelen Boxes in the research project. There is a tutorial for this, but it won't be put on the Web site until the research project has provided several case histories to back it up.
Virtual hierarchies
Once you get used to the idea of a Kempelen Box being able to exist outside of the environment of a database and existing as an independent object on the Web, it isn't too hard to imagine them popping up all over the place. Kempelen Boxes can evolve to specialize in all kinds of different subject areas. They can also link up to each other, to create a navigational hierarchy between them.
As these Kempelen Boxes exist in an independent state and are not confined to the organization of a proprietary backend database, the navigational hierarchy will be dynamic and self-organizing. It will evolve and constantly change, much like the pheromone trails left by ants. This is in sharp contrast to lists of links on Web pages or RSS streams, where branching is random. Kempelen Box branching will result in the evolution of a higher level of organization, which will become increasingly more efficient. This is the end goal of the research project: to initiate such a self-organizing, organic system.
Taking part in the research project
The idea of the research project is to create several of these Kempelen Boxes and see how to improve their efficiency. The agents can be given functions. More sophisticated methods of visitor interaction with the agents can be introduced. The way the Kempelen Boxes interact to create links and virtual navigation hierarchies can be developed.
The more people that become involved in this project, the more avenues that can be explored. If you would like to take part in this project, either as a partner or a sponsor please get in contact.
The aim will be to get at least ten Kempelen Boxes working successfully by the end of this year.
Alternative ways of creating Kempelen Boxes
There are many different ways Kempelen Boxes can be built. They lend themselves well to page generation techniques of backend databases and to XML. However, these methods are best left until a fully developed Kempelen Box has emerged because they are not easy to work with when you have constant evolutionary change.
The strategy we are using is to arrange for the system to emulate biological systems as closely as possible. This necessitates functions being carried out in client browsers rather than in a central database. This may seem an inefficient and cumbersome approach, but it enables quite a different mind set to be applied, which can lead to further initiatives coming from the biological world.
After all, you can "feeze frame" the development cycle at any stage to create a more efficient presentation (a process that should take a competent Web site developer only a few days). The downside is that such a move may result in the design getting locked into a system that is soon outdated as the biologically inspired Kempelen Boxes continue to evolve.
Various explanations relating to the overall development of this technology can be found in the 'Research' and 'References' section of this site. A working example can be seen by looking at the 'SOE project'