Predator-Prey-Grass

Evolution in a multi-agent reinforcement learning gridworld

This repo explores the interplay between nature (inherited traits via reproduction and mutation) and nurture (behavior learned via reinforcement learning) in ecological systems. We combine Multi-Agent Reinforcement Learning (MARL) with evolutionary dynamics to explore emergent behaviors in a multi-agent dynamic ecosystem of Predators, Prey, and regenerating Grass. Agents differ by speed, vision, energy metabolism, and decision policies—offering ground for open-ended adaptation. At its core lies a gridworld simulation where agents are not just trained—they are born, age, reproduce, die, and even mutate in a continuously changing environment.

Trained Predator-Prey-Grass walls-occlusion environment

Environments:

• Base environment: The two-policy base environment. (implementation, results)

• Mutating agents: A four-policy extension of the base environment. (implementation, results)

• Centralized training: A single-policy variant of the base environment (implementation)

• Walls occlusion: An extension with walls and occluded vision (implementation)

• Selfish gene: An extension with lineage rewards (implementation)

Experiments:

• Testing the Red Queen Hypothesis in the co-evolutionary setting of (non-mutating) predators and prey (implementation, results)

• Testing the Red Queen Hypothesis in the co-evolutionary setting of mutating predators and prey (implementation, results)

• Testing cooperation by adding lineage rewards instead of direct reproduction rewards (results)

Hyperparameter tuning

• Hyperparameter tuning base environment - Population-Based Training (Implementation)

Installation of the repository

Editor used: Visual Studio Code 1.101.0 on Linux Mint 22.0 Cinnamon

1. Clone the repository:

   git clone https://github.com/doesburg11/PredPreyGrass.git

2. Open Visual Studio Code and execute:
   • Press ctrl+shift+p
   • Type and choose: "Python: Create Environment..."
   • Choose environment: Conda
   • Choose interpreter: Python 3.11.13 or higher
   • Open a new terminal

   • pip install -e .

3. Install the additional system dependency for Pygame visualization:

   • conda install -y -c conda-forge gcc=14.2.0

Quick start

Run the pre-trained policy in a Visual Studio Code terminal:

python ./src/predpreygrass/rllib/base_environment/evaluate_ppo_from_checkpoint_debug.py

Or a random policy:

python ./src/predpreygrass/rllib/base_environment/random_policy.py

References

• RLlib: Industry-Grade, Scalable Reinforcement Learning
• Paper Collection of Multi-Agent Reinforcement Learning (MARL)
• Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer