GR4SP

Generic Recursive Simulation of Socio-technical Systems for Service Provision

GR4SP is an agent-based simulation model of the Victorian electricity system in Australia. The core engine is written in Java (using the MASON framework) and models how generators, consumers, networks, and policy interact over time. A Python layer built on EMA Workbench enables sensitivity analysis and large-scale scenario experiments. Jupyter notebooks in experiments/notebookGr4sp/ support result analysis and visualisation. While the data is Victorian electricity-specific, GR4SP's structure can guide similar simulations in other contexts.

---

Prerequisites

Tool	Version	Purpose
Java JDK 17+	17 or later	Run the simulation
PostgreSQL	14 or later	Electricity system database
VS Code + Extension Pack for Java	Any recent	Edit and build
Miniforge	Python 3.10+	Experiments, notebooks, and the data-update scripts only

---

Quick Start

Step 1 — Clone

git clone https://github.com/gr4sp/simulationEngine.git
cd simulationEngine

Windows: enable long paths first. A few data files in experiments/assesmentData/ have descriptive names up to 139 characters. Combined with a deep clone location, these can exceed the legacy 260-character Windows path limit and the clone fails with error: unable to create file ...: Filename too long. Enable long-path support once (Administrator PowerShell):

git config --system core.longpaths true

Without Administrator rights, use git config --global core.longpaths true, or simply clone into a short path such as C:\gr4sp.

Step 2 — Run the setup script

The setup script checks Java, builds the project, creates output directories, and loads the database.

Windows (VS Code terminal):

Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\setup.ps1

Linux / macOS:

chmod +x setup.sh && ./setup.sh

Both scripts will prompt for your PostgreSQL password and handle everything automatically.

One-time PostgreSQL configuration

GR4SP connects to the database without a password (JDBC). You need to configure PostgreSQL to trust local connections. Run the following as Administrator (Windows) or with sudo (Linux/Mac), then restart PostgreSQL.

Windows (run each line separately in an Administrator PowerShell):

(Get-Content "C:\Program Files\PostgreSQL\18\data\pg_hba.conf") -replace '(host\s+all\s+all\s+127\.0\.0\.1/32\s+)scram-sha-256','${1}trust' -replace '(host\s+all\s+all\s+::1/128\s+)scram-sha-256','${1}trust' | Set-Content "C:\Program Files\PostgreSQL\18\data\pg_hba.conf"

Restart-Service postgresql-x64-18

Linux / macOS — edit /etc/postgresql/XX/main/pg_hba.conf (where XX is your PostgreSQL version) and set the host lines for 127.0.0.1/32 and ::1/128 to trust, then run:

sudo service postgresql restart

Step 3 — Run the simulation

Windows:

.\runGr4sp.bat

Linux / macOS:

./runGr4sp.sh

The simulation runs the default scenario (simulationSettings/VIC.yaml). Results appear in csv/ and plots/ when complete.

---

Run with Graphical UI

To watch the simulation progress with live plots:

Windows:

.\runGr4spUI.bat

Linux / macOS:

./runGr4spUI.sh

---

Build from Source (VS Code)

The project uses Gradle. No manual javac commands needed.

• VS Code: open the Gradle panel (elephant icon in the left sidebar) → gr4sp → Tasks → build → double-click build
• Terminal:

  .\gradlew.bat build   # Windows
  ./gradlew build       # Linux / macOS

Compiled classes go to build/classes/java/main/.

---

Changing Simulation Settings

Scenario settings are defined in simulationSettings/*.yaml. The default scenario loaded at startup is VIC.yaml.

Key parameters:

Parameter	Values	Description
reportGeneration	"full" / "light"	full saves all CSV data and plots (~10 MB); light saves summary files only (~75 KB)
logLevel	"OFF" / "WARNING" / "ON"	Simulation logging verbosity
simulationDates.startDate	YYYY-MM-DD	Simulation start date
simulationDates.endDate	YYYY-MM-DD	Simulation end date

Note: folderOutput is auto-detected from the working directory — do not edit it.

---

Running Experiments (Python / EMA Workbench)

Create a conda environment with Miniforge and install the experiment dependencies:

conda create -n gr4sp python=3.12
conda activate gr4sp
pip install JPype1 pandas ipyparallel SALib numpy scipy matplotlib PyYAML

Each new terminal session, re-activate the environment with conda activate gr4sp.

EMA Workbench itself does not need installing — it is vendored in experiments/EMAworkbench/ and imported from there.

Scenarios are defined in experiments/scenarios.yaml, which holds the uncertainties, constants, and run settings for each one. Run a scenario by name from the experiments/ folder:

cd experiments
python run_experiment.py BAU        # BAU, JT, LCT, or ST
python run_experiment.py JT --dry-run    # print the run plan without executing
python run_experiment.py LCT --processes 4

Results are written to experiments/simulationData/gr4sp_<SCENARIO><date>.tar.gz. Note that the sampled scenarios (JT, LCT, ST) run thousands of simulations and take hours; BAU is a single deterministic run and is the quickest way to check your setup works.

settingsExperiments.json is pre-configured — the JVM path and classpath are detected automatically. No manual editing required, but the project must have been built first (.\gradlew.bat build) so that build/runtime-libs/ exists.

---

Updating Victorian Electricity Data

Scripts in scripts/data/ refresh gr4spdb with current AEMO demand/price, Open Electricity generation, ERA5 solar/temperature, and CER rooftop solar data. They use the same gr4spdb database as the simulation, so re-run them periodically to keep scenarios current.

Install their dependencies (in the same gr4sp conda environment used for experiments, or a separate one):

conda activate gr4sp
pip install -r scripts/data/requirements.txt

Copy scripts/data/.env.example to scripts/data/.env and fill in the required credentials (an Open Electricity API key from platform.openelectricity.org.au, and a Copernicus Climate Data Store key from cds.climate.copernicus.eu — accept the ERA5 dataset's license on its download page before first use). .env is gitignored; never commit it.

Then run each script from the project root as needed:

python scripts/data/fetch_aemo_demand.py          # VIC1 demand & price (AEMO NEMWeb, no API key needed)
python scripts/data/fetch_openelectricity.py       # Generation & revenue by fuel type
python scripts/data/fetch_era5.py                  # Solar irradiance & temperature (CDS; can take hours — queues per month)
python scripts/data/fetch_cer_solar.py              # Rooftop solar installations by postcode

Each script is idempotent — safe to re-run, and re-running fetch_openelectricity.py after fetch_era5.py backfills the temperaturec column from ERA5's output. See the docstring at the top of each script for what it fetches and any known data-source limitations.

---

Analysing Results

Jupyter notebooks for scenario analysis, sensitivity analysis, and visualisation are in experiments/notebookGr4sp/. Open them in VS Code or JupyterLab.

Install the notebook dependencies first:

conda activate gr4sp
pip install -r experiments/notebookGr4sp/requirements.txt

The notebooks analyse the results of EMA experiment runs, which are large .tar.gz archives that are not stored in git. The published run results are archived on Zenodo record 8320754 and can be downloaded with the fetch script:

cd experiments/simulationData
python fetch_results.py --list          # show what is available, download nothing
python fetch_results.py                 # fetch the 14 result archives (~1.3 GB)
python fetch_results.py --all           # also fetch the input CSV/XLSX files
python fetch_results.py JT              # fetch only archives matching a substring

The script skips files already present, so it is safe to re-run. Fetching only the archives a given notebook needs is usually enough — see experiments/notebookGr4sp/reproducibility/ for which notebook depends on which archive.

---

Troubleshooting

Error	Cause	Fix
NullPointerException in LoadData	PostgreSQL not configured to trust local connections	Follow the trust configuration in Step 2
Problems reading YAML file	Running simulation from wrong directory	Always run from the project root (gr4sp/)
gradlew: Permission denied	Fresh clone on Linux/macOS	chmod +x gradlew
Filename too long during git clone (Windows)	Legacy 260-character path limit	git config --system core.longpaths true, then re-clone — or clone into a shorter path such as C:\gr4sp
Java not found during setup	JAVA_HOME not yet in terminal PATH	The setup script auto-detects from JAVA_HOME; or restart VS Code
pg_restore warnings about adminpack	Extension not available in newer PostgreSQL	Harmless — data is restored correctly