Simone Reale

MAS-based Sequential Optimization for Fairness in Urban Green Planning

This repository implements a reinforcement learning framework for adaptive urban green planning, building on the TALEA project and its static optimization approach for TALEA Green Cells (TGCs) placement in Bologna (check the TALEA_Green_Cells repository). The work reframes green infrastructure allocation as a sequential decision-making problem: instead of solving a one-shot constraint optimization, a learning agent selects interventions year by year, balancing environmental benefits, implementation costs, and social fairness over a multi-year horizon.

The simulation is grounded in open geospatial data from the Bologna Digital Twin (100×100 m grid cells with NDVI, UHEI, population density, and free-space indicators) and models vegetation growth, heat dynamics, and budget constraints within a custom Gymnasium environment trained with Proximal Policy Optimization (PPO).

Example of green intervention assignments ranked by fairness score, overlaid on Urban Heat Exposure Index (UHEI):

Note

For the full theoretical background, framework architecture, reward formulation, and experimental discussion, see the project report.

Environment Modelling

The core simulation is GreenCityEnv, a custom Gymnasium environment that translates the urban planning problem into a Markov Decision Process (MDP).

State space — Each grid cell is described by seven features: NDVI, UHEI, population density, existing green area, available free space, newly added green coverage, and the age of new interventions.

Action space — At each yearly step, the agent assigns a continuous percentage of free space to convert into new green infrastructure for every cell, subject to budget and optional per-year cell limits.

Dynamics — The environment models baseline vegetation decay, logistic maturation of new plantings, non-linear NDVI–vegetation cover relationships, and corresponding UHEI updates. Each episode spans a configurable horizon (default 30 years).

Reward — Step rewards favour vegetation improvements in high-heat, high-population areas while penalising intervention costs; a terminal bonus encourages equitable long-term green distribution.

Usage

1. Install dependencies

pip install -r requirements.txt

2. Random environment test

Run a quick sanity check with random actions (validates the Gymnasium interface and prints step-by-step metrics):

cd src
python green_city_env.py

3. Train and run the PPO agent

Launch the full training and evaluation pipeline (saves model weights, logs, GeoJSON outputs, and comparison plots under results/):

cd src
python ppo_agent.py

Tip

Optional hyperparameters can be passed via CLI flags (e.g. --episode-length, --total-budget, --total-timesteps, --seed). Run python ppo_agent.py --help for the full list.

Repository Core Files

File	Description
src/green_city_env.py	Defines GreenCityEnv: loads geospatial grid data, implements the MDP step/reset logic, and exposes observation and action spaces.
src/ppo_agent.py	Implements the Actor-Critic CNN, PPO training with GAE, observation wrapper, evaluation against a random baseline, and result export.
report_Simone_Reale.pdf	Complete academic report covering motivation, RL background, framework design, and analysis.