Grid Agents: Part 01
The Idea
I've been wanting to build a simulation game using AI agents for a while now. There have been some really great ones based on a research paper from a few years ago and turned into simulations such as AI Town.
These games were created as a pixel-art RPG type of game. Although I would love to replicate this in the future, for now I wanted to make something simpler as far as the graphics for now. I came across a tweet where someone was demonstrating creating a simple grid style agent game through prompting in OpenAI. This seemed like a great base to try to replicate and would also give me an avenue to use generative AI coding tools to implement.
Getting Started
The first thing I did was take a couple screen shots from the video of the previous example app and feed that into ChatGPT using o3-mini.
Analyze the screenshots and try to replicate this multi-agent game in Python. Do not write code yet, just act as a project manager describing the game and requirements to build it
Next I took this output and inputted it back into ChatGPT but this time using o3-mini-high asking it to create the game in python using pygame. It outputted a simple python script for the game, but of course I needed it to do more.
The agents should be LLM powered, independent but interacting with each other. Each agent should have configurable features like bio, llm model, etc. Rather than write the code for that now, write out the software dev plan
From here I iterated the plan until I came up with a good project plan to start with.
Moving to Cursor
I'm a huge fan of Cursor. I've been using it for a while to help me with projects. Although my background is as a software developer, like a lot of people, my list of personal projects far exceeds my time to actually write them. For this Cursor and similar AI coding tools have been great. I've been using others like GitHub Copilot, but I keep coming back to Cursor.
OK, so the next step was to take my project plan from ChatGPT and move it to a document in Cursor. For projects in Cursor I usually create a "projects-doc.md". I use this as a reference for the coding assistant to understand what I want build.
From here I can start prompting the coding assistant/agent in Cursor
Using the @project-doc.md let's start developing this application. Before writing any code tell me your plan so I can make sure it follows my vision.
I'll do a deeper dive into Cursor and how I use it in a future post, but for now the focus is on what I am building with it.
Early Stages
Initially I had planned on doing a replica of the colored grid approach from the app I had seen, but that quickly morphed into something a bit different. Instead I settled on more of what is seen in the screenshot below, where I have 10 x 10 grid with buildings, streets, and vendor locations. For the purposes of this simulation the "streets" are basically sidewalks and we have a car-less little town. While I've been playing around with the idea of having the buildings be homes and places of work for our residents, as of the current iteration they are simply places for them not to be able to go so they stick to moving around the streets, with the exception of vendor locations.
Vendors include a mix of places to eat, coffee shops, and places to shop.
Once I had them moving around ok, and not getting stuck in one spot too much I moved on having them chat. This of course is one of the main points of this simulation...to see how they can interact with each other. Some of the goals I have for this is to have them meet new agents in the simulation, interacting over a longer term with friends, etc. For instance if Alice tells Bob about her plans for today, if Bob runs into her tomorrow he will remember the conversation and ask about it.
One of the issues I was running into was they tended to say "Hey X!" a lot even during the same conversation. My way around that for now is to tell them not to do that in the prompt. I'd like to refine it in the future so they are better at doing it at the start of the conversation when appropriate.
That's it for now. As I continue to work on it, here are some of the main goals I'm planning on tackling:
- Improve the conversation flow
- Implement a memory system for the agents
- Create a web-based UI to watch/replay the scenarios