Sharing bookmarks - a simple example of a stigmergic system

This is a simple, non-technical example to illustrate how to use techniques borrowed from biological systems.

It describes a way in which individuals, acting independently, can collectively create an information resource that becomes progressively more organized and efficient (using stigmergy and an evolutionary process).

Note: This example may be better understood if you have read through the concepts first.

Imagine several people, surfing the Web, each looking for information in the same subject area. Probably, they would all be bookmarking web pages they find particularly useful and interesting.

It would make sense if they got together as a group and agreed to put all their bookmarks into a common pool - where they would be able to see what each other had found to be interesting and useful.

Consider now, how they might go about arranging for this sharing process to be optimally efficient - eliminating any poor quality bookmarks; keeping the number of bookmarks in the pool down to as small a size as possible consistent with fully covering the subject area; dividing up the search area between them.

1) Submitting bookmarks to a common pool

It would be helpful if the pooling process arranged for the bookmarks to be submitted in such a way as to separate them out into specific categories. This could be done with the aid of a suitable form:

Form for sorting bookmarks into categories

The questions used in this form (pop-up window)

2) Bookmark list submissions

Here is an illustration of how the bookmark collections of about fifty people (searching for information on the search engine optimization of a web site) might look after making their submissions using such a form:

Contributions to a pool of bookmarks

3) Combining the lists

When combined together into a common pool, these bookmark collections provide a wide ranging resource, reflecting the various different search efforts of the whole group:

Grouping all the bookmark lists together

4) Associating the bookmarks with their contributor

If many people are contributing bookmarks to a pool, there may be much variation in emphasis, bias or subjective values. To be able to make comparisons, it would be useful to include with the bookmarks some information about the person who had contributed the bookmarks:

Identifying the source of the bookmarks

Similarly, it may be useful to be able to isolate and examine the bookmark lists of particular contributors.

Separating out the individual contributions

Note: in both of these lists, the left hand column (of the table on the right) allows togggling between subject categories and individual lists.

5) Optimizing the lists

If collections of bookmarks are continuously being added, the pool would probably accumulate a large number of poor quality links. These can be eliminated if members of the group were able to create their own customized versions of the pool, according to their preferences for certain contributors.

Creating customized versions of the pool

Notice that customization is not arranged by selecting individual bookmarks, but by selecting for groups of bookmarks made by particular contributors.

6) Displaying the customized pools

It would be of value to everyone in the group if they could see each others customized selections.

Effectively, this would be like adding a judgmental value to the groups of bookmarks that have been selected - separating them out from the other bookmarks that haven't been perceived as being particularly valuable:

Nine customized pools

7) How the evolutionary process kicks in

The system described above will seem ludicrously naive in terms of conventional business thinking.

There may be a procedure for choosing favored groups of bookmarks, but, it wouldn't stop the gradual accumulation of redundancy, out-of date information and poor quality links. In no time at all, the resource could become so clogged up with poor quality links that the system would become inefficient and impractical to use.

However, this situation can easily be remedied by limiting the number of bookmarks that appear in each category. Then, any new groups of bookmarks coming into the pool could be arranged to replace groups of inferior bookmarks (inferior groups of bookmarks being identified as those not being selected for any of the customized pools).

The result will be a steady turnover of bookmarks in the main pool, with the least valuable bookmarks being continuously removed.

In other words, the number of bookmarks in each category is kept constant, but the quality of the bookmarks is allowed to continuously improve (the system acting like a memory that remembers only a fixed number of the best links and quickly forgets the rest).

Not only will this memory improve, it will also be able to adapt to technological or competitive change - by losing bookmarks that relate to information that becomes redundant or out of date due to technological change or competitive advances.

8) Improving the quality of bookmark contributions

A powerful driving force in biological evolution is the concept of stigmergy. This is the phenomenon whereby people respond to what they see in an environment by adding to it or making changes.

Evolutionary design strategies that use proto-typing employ stigmergy. People add an improvement to a prototype and this might elicit a response from someone else to make another improvement. A continuous sequence of people changing the prototype in different ways results in a design that evolves over time rather than be pre-planned.

Stigmergy is commonly observed in the design of websites, where people see what others are doing and incorporate some of those ideas in with their own. As can readily be appreciated, this stigmergic behavior has resulted in website designs becoming progressively more complex and sophisticated - as ideas and techniques are rapidly disseminated, combined and improved.

It is this same phenomenon that can cause contributions to a shared pool of bookmarks to become increasingly more valuable.

The selection and elimination process will prompt many bookmark contributors to update and improve their contributed lists - especially if they find their selections are being ignored. They will see gaps in the knowledge base and seek to fill them. They will add updates or alternatives to what is already there. They will create new, more efficient groupings.

As a team effort, everyone will feel a responsibility to contribute their fair share. - which will only be recognized by having bookmarks that remain in the pool. This is the psychological pressure that drives the system - the same pressure that drives every cooperative or collaborative undertaking.

To facilitate the continuous updating of contributed bookmark lists, contributors can be provided with a control panel that allows them to easily edit their list of bookmark contributions.

Control panel for modifying a list of bookmarks

9) Self-organization

Members of the group of contributors will quickly learn that, rather than try to cover the whole subject area, their efforts will be most efficiently employed if they concentrate upon specific categories. Thus, the work that goes into searching and updating will become more efficient, as different members of the group begin to take responsiblity for different categories.

In this way, the group activity self-organizes, to become optimally efficient. It doesn't need any central control to make this happen. It is a natural emergence that always seems to happen in collaborative systems.

Self-organization can occur in other ways. As categories fill up with bookmarks, someone might recognize a need to sub-divide a category. Perhaps a category will be associated with other types of knowledge that has no categorization, so a new category might be added. In these ways, the structure of the resource will change organically, responding to needs.

10) It's about people, not information

Most people, looking at this example, would assume it is only to do with creating an information resource. But, it is about something far more important: it is about gathering together a group of collaborators.

It is not just the information that is being selected for, it is also selecting for the people who are contributing the information. This is where the true value lies: the value is in the people.

This aspect is dealt with in more detail in the explanation of the concepts.

11) A conceptual framework

The example provides a conceptual framework with which to learn more about the way biological strategies can be used in information technology. It takes your mind away from the logical, algorithmic world of conventional information processing to focus upon the complexity of human interaction.

Built into most people's psyche is a natural inclination to collaborate and cooperate. The study of biological systems helps us to understand and take full advantage of these inherent tendencies, allowing us to create a variety of different ways to deal more effectively with the overwhelming, chaotic environment of the information that is available to us on the Web.

12) Technicalities of the structure

The example would appear to be the product of a database program. In fact, it is created using a system that emulates the way information is stored and processed by biological cells. Instead of records and fields, it uses interacting agents that have functions and memories.

This method is more flexible and versatile than database technology, as it can more precisely simulate biological systems that evolve and self-organize.

With a database system, structure and organization has to be designed and pre-planned. With an agent based system, structure and organization emerge organically, finding its own way to operate with maximum efficiency.

13) Polymorphing

Because they use agent technology, rather than a central database, stigmergic systems can be presented in a variety of different ways and take a variety of different forms. Different metaphors can be used to create imaginative scenarios for presentation purposes.

Components of a system can be distributed around the web, where each component can presented in a different way or be controlled from an independent source. There is no need to have any common formalized structure or central control - systems will always tend to evolve into a form that is most effective and most efficient.

An insight into this strange world of organic databases and agent technology can be gleaned from the notes of three Bot seminars.

In particular, look at the notes of the second talk, given at BOT2001 in San Francisco. This will add a new perspective to the simple example illustrated here.

A outline plan for a project to use this system to enable cancer patients to find suitable treatment trials was prototyped in the CTW Project (2002). If you click on the "R" category of this demo, and follow through the links marked in red, you'll see how an information space is divided up in such a way that medical practitioners, clinical trial managers and patients interested in a particular type of cancer all arrive at a particular place where they can share and compare information.

More detailed explanations, articles and tutorials on various aspects of using biological technique can be found in the "Technical Notes" and "References".