Reinforcement Learning Pac-Man

Overview

This project was implemented using the programming language Python using the graphics engine PyGame. The Just-In-Time compiler Psyco was used to speed up training and testing of the project. Python and PyGame are needed if you wish to watch Pac-Man learn or if you want to play the game.

The base of this project is a working version of Pac-Man that we developed from the ground up to be flexible and modular. All of the levels were designed by us. The first level was modeled after the actual level of the original Pac-Man game. Anyone may easily make new levels for the game by simply editing a level file. Details about that can be found in the final document.

The user can change some very modular configuration files to make the game play exactly as they wish. They may elect to play the game themselves, or just watch Pac-Man learn to play the game.

Results

Overall, this project was a huge success. We were able to implement a Pac-Man game that is playable by a person as well as playable by an AI Pac-Man. The human playable mode is quite enjoyable with smooth game play and an infinite number of new and exciting levels. The learning Pac-Man mode was able to successfully play the game very well. A highly trained Pac-Man can achieve an average score of about 12,000 points and usually gets to level 7 or so. This is much better than many human players are able to do. Can you beat Pac-Man?

Download

Links for the source code, design document and final report are below.

Future Work

We intend to extend the learning of Pac-Man to be more robust. We want to add several new features to the function approximation learning mode to see how good he can get. This will come when we have some free time. Also, next semester we will be taking a Neural Networks class and would like to convert the learning mode to take advantage of the great power of neural nets.

We are also going to add sound effects. If anyone knows where we can get some free sound effects or would like to make some for us, please send an email to asmith12@eecs.ku.edu.

To Do

• Sound Effects
• Improved PyGame performance to reduce CPU usage
• Better feature set to make Pac-Man perform better
• Fruits - These were left out for simplicity
• New Levels - Feel free to send any particularly good level designs to asmith12@eecs.ku.edu
• Change the speed of the agents so that the ghosts move closer to the speed of Pac-Man like in the original version of the game.

Screen Shots

Here are some screen shots of me playing some of the levels we made: