Madhu's Corner :: Write :: Artificial Dreaming

Interesting concept, perhaps.
Dangerous implications.

Artificial Dreaming - isn't that term a contradiction in itself? The process of dreaming is essentially so natural and spontaneous that any connection with artificiality seems contrived. Dreams and hallucinations are part of the mysteries associated with the brain. The human brain has been the inspiration for much of the research in data processing. It is, quite simply the best processor that we know of. Directly leading from this line of thought are neural networks. Derived in major part from accepted and established methods, by which the brain processes information, neural networks are a well-known field today. Not surprisingly, since neural networks were supposed to mimic the brain, work is going into making neural networks dream.

In order to better understand what this means, we must deviate a little to examine some of the information we have about dreaming. Dreaming is a process by which random unconnected events occur in the brain. This leads to the impression to the person in the dream that the events are happening in real life. There are currently many theories about the Rapid Eye Movement (REM) phase of sleep and dreaming. Other questions include why we can or cannot remember our dreams.

However, on a more practical level, what does dreaming give us? Leaving aside the psychic and psychiatric aspects of that question, from a purely information processing point of view, a dream is a very useful event. The brain, limitless as it may seem, is a finite system. There is a pressing need to optimize and correlate the vast amount of information that we receive each day. A dream, in that context is a practical method by which the brain ensures that information is stored efficiently for usage. The brain sifts through events in dreams, organizing them, revisiting old memories and optimizing storage.

The immediate implications of this line of thought are easy to see. In neural networks too, dreams optimize memory utilization. A famous example used is that of a study by Hopfield and Tank(1987) where a neural network was made to learn telephone numbers. The Artificial Neural Network (ANN) was trained to retrieve a 10-digit number from just 2 digits. As the new numbers were being learnt, the previous numbers would take longer for the ANN to return. In effect, it meant that the ANN was beginning to forget the older numbers. Eventually, the ANN was producing spurious numbers that it had never seen - in effect it was making them up. In order to improve performance, random memories were induced into the network. The input to the network was set at constant values. Internally, neurons were fired randomly, as there was some form of varying input. After a given period, it was found that the ANN now responded to all numbers in the same time and produced no spurious responses. There was a startling improvement in the overall performance.

That is not all. Something even more exciting has been recently put forward. Now, creativity is a very human and an intangible process. However, the focus is now on ANNs that create information. That statement may seem far-fetched, but it is true and it has already been done. The process first detailed by Stephen Thaler (1995) involves primarily teaching an ANN about a certain problem. Once this is accomplished, the network is duplicated so that there are twins that "think" the same way. One of them dreams while the other watches and makes tangible sense of the dream. This leads to new information that was not present in the data initially provided to the network.

For example, an ANN was exposed to various chemical formulae, until it began to understand the fixed ratio's by which elements combine. At this point, the ANN was given information about various compounds of Oxygen and Hydrogen including water (H2O). Asked to dream, the network was able to conceive of Hydrogen Peroxide (H2O2) - something it had not known before. This ANN was also taught about the relative hardness of each compound and asked to recognize extremely hard compounds. The next step was to ask it to dream in this state. It dreamt of hard compounds, some that were well known commercially and some new ones (which were all patented). The details of this procedure and other such processes are available on the Internet. This process extended to music, wherein an ANN listened to popular music and then independently created new music. What next is left to the realm of imagination.

Even more contentious are the ethical questions involved. Who owns the creativity that the machine produced? Does the machine want to dream? Do we have a right to make it dream or create? Are the humans from whom the initial information was derived, part of the process? These and other questions that we all have to grapple with as machines increase in their complexity and effectiveness.

What does this mean to the average human being? Does he see a need to feel threatened? Are machines finally winning the race? Not really. The kind of intuitive leaps that humans make are based on their overall information (often termed "domain" knowledge). It is very difficult to create and supply an ANN with all the information that a human routinely carries around. However, in specialized situations, ANN will begin to take over creative tasks. That is something that we all will have to accept. Nevertheless, there is good news at the end of the tunnel. With the human brain having around 10^18 neurons, there is no possibility that ANN's are going to get the better of us, just yet. Moreover, machines ruling over humans are just an idle daydream.