23 Jan ’08

Memory

Memory of the critters will be a fairly simple thing to start with. The short term memory can be imagined as a simple tube, in which the experiences of the animal are entered on one side, and slowly shoved through the tube as more new experiences are added. Experiences come in three forms: observations, actions (your own), effects (for example, a change in your energy state).



At a fairly early point in life the tube will be full, and experiences will `fall out' of the tube at the other side. These experiences are stored into the long-term memory of the critter, and as they are stored, they associate themselves with a few of the experiences that are currently in the short-term tube.

Later on, when the animal encounters an experience for which it has a memory, this memory will also give the animal acces to some predictions about what is going to happen next. what actions it performed in the past, and what effect these actions had. This (hopefully) is enough for the animal to make an informed decision.

More details on the memories: if an animal experiences something again, the memory already carries with it a number of associations (3 in the example), but in total has room for a predefined number of associations (current default is 50). These associations are randomly overwritten. As a result, those experiences that often follow after a particular memorized experience will occur more frequently in the array of associations that comes with the memory. Therefore, the frequency with which something is associated with a particular memorized experience provides a mechanism by which multiple responses to a particular experience can co-exist with different `weights' each.

One possible future extension is not to pick random associations, but to increase the chance on associations that are temporally closer to the oldest one (a poisson distribution).

Codewise, the implementation is as follows

(deftem creature 

   stm nil

   stmsize 7

   ltm (table)

   ltmsize 50

   s2ltm 3)



(= creature (table))

(= (creature "ltm") (table))



o = observation, i = the memory to be stored, stm / ltm = short / longterm memory.

(def add-observation (creature o)

   (= creature!stm (cons o creature!stm))

   (let i (car (nthcdr creature!stmsize creature!stm))

      (if i

         (do

            (if (no (creature!ltm i)) (= ((creature 'ltm) i) (n-of creature!ltmsize nil)))

            (repeat creature!s2ltm

               (let stm-pp (rem i (rem nil creature!stm))

                  (if (~is 0 (len stm-pp))

                     (= ((creature!ltm i) (rand 50)) (car (nthcdr (rand (len stm-pp)) stm-pp))))))

            )))

   (= creature!stm (firstn creature!stmsize creature!stm))

   nil)

Update 9 may 2008:
I made a small change to the add-observation code to allow different craas to have a different short and/or longterm memory array size, and associate a different number of short-term memory with a longterm memory, as I've no idea yet what are good and functional sizes.

One of the major things that is still lacking from this memory is the issue of saturation / filtering uninteresting information. If there is always grass around you, you probably don't associate it with something, or let it fill your short-term memory. Two solutions come to mind: a module that 'filters' your observations in some way (perhaps by distincting between expected / unexpected, or by flagging things as 'not interesting'). A second possibility is that you can lump similar observations so that they don't take as much space in your short-term memory (1, 2, many trees). Moving from one place in the forest to another would just add a single 'many trees' observation to your shortterm memory, rather than filling your short-term memory with 'tree','tree','tree','tree','tree', and missing the 'bear'.
~