Georgios Pierris, one of my PhD students had a very good idea today. In combining SOMs and RL we are defining two landscapes, the SOM landscape and the reward landscape. It would be very interesting to see if it would be possible to use the SOM landscape to approximate the RL landscape and, thus, calculate utilities for unexplored areas of the RL landscape. It brings to mind Ulrich Nehmzow's work on sub-symbolic planning.
John Pisokas and Ulrich Nehmzow, Performance Comparison of Three Subsymbolic Action Planners for Mobile Robots, Robotics and Autonomous Systems, 51(1):55-67, 2005
At the Cognitive Robotics Research Centre at the University of Wales, Newport, we have been working on a reinforcement learning self-organizing map (RLSOM). Without going into the details of the RLSOM algorithm, I'd liek to list some potential extensions: