Local Setup — SLURM

This guide covers a full local setup with Keycloak authentication — required for SLURM job submission. It builds on the Celery worker setup and adds Keycloak for both the backend and frontend.

When to use this setup

Scenario	Recommended setup
Frontend / Geant4 in-browser only	Local Frontend Demo
Full stack with SHIELD-HIT12A or FLUKA	Local Celery
Full stack with SLURM cluster	This page

What you’ll run

You need three terminals for the backend, plus one for the frontend:

Terminal	What runs
1	Celery simulation worker
2	Celery helper worker
3	Flask API
4	Frontend dev server

Prerequisites

• Python 3.9+ with Poetry installed
• Node.js 20+ with npm
• Docker
• Git

Backend setup

1. Clone and install dependencies

Clone the repository:

git clone https://github.com/yaptide/yaptide.git

Navigate to the yaptide/ directory:

cd yaptide

Install Python dependencies with Poetry. This will create a virtual environment in .venv/ and install all required packages for the backend:

poetry install

2. Download simulator binaries

YAPTIDE stores licensed simulator binaries on S3-compatible object storage. The download-shieldhit command has two modes depending on whether S3 credentials are provided:

• Without S3 credentials (default demo mode): downloads the freely available SHIELD-HIT12A demo binary directly from the project website. No configuration needed.
• With S3 credentials + --decrypt flag: downloads the full licensed binary from your S3 bucket and decrypts it using the configured encryption key.

Linux

poetry run yaptide/admin/simulators.py download-shieldhit --dir bin

Windows (PowerShell)

poetry run yaptide\admin\simulators.py download-shieldhit --dir bin

To use S3 (for the full licensed version), create a .env file in the project root with the following variables:

.env

S3_ENDPOINT=https://your-s3-endpoint.com
S3_ACCESS_KEY=your-access-key
S3_SECRET_KEY=your-secret-key
S3_ENCRYPTION_PASSWORD=your-encryption-password
S3_ENCRYPTION_SALT=your-encryption-salt
S3_SHIELDHIT_BUCKET=your-bucket-name
S3_SHIELDHIT_KEY=shieldhit-filename.tar.gz

Then run the download with S3 and decryption enabled:

Linux

poetry run yaptide/admin/simulators.py download-shieldhit --dir bin --decrypt

Windows (PowerShell)

poetry run yaptide\admin\simulators.py download-shieldhit --dir bin --decrypt

Note

The free demo binary is sufficient for development and testing. The full licensed version of SHIELD-HIT12A (and FLUKA) requires S3 access and an encryption key — contact your institution or the SHIELD-HIT12A team for access.

3. Start Redis

Redis is the message broker between Flask and the Celery workers. The simplest approach is a Docker container:

docker run --detach --publish 6379:6379 --name yaptide_redis_local redis:7-alpine

4. Setup .env

Add these variables to the .env file in the project root:

.env

KEYCLOAK_BASE_URL='http://127.0.0.1:8080'
KEYCLOAK_REALM='yaptide'
KEYCLOAK_CLIENT_ID='yaptide-app'
CERT_AUTH_URL="http://127.0.0.1:5001"

# enable slurm docker container to reach the backend
FLASK_RUN_HOST=0.0.0.0
BACKEND_EXTERNAL_URL=http://host.docker.internal:5000
CA_ALLOW_INSECURE_TOKENS=true

# number of workers to run in the slurm container
# running simulations with JOBS > WORKERS works, but will be slower
SLURM_CPU_WORKER_COUNT=2

5. Start the Celery simulation worker

This worker picks up simulation jobs from the queue and runs the simulator. It needs access to the simulator binaries (the bin directory from step 2).

Linux

PATH=$PATH:bin BACKEND_INTERNAL_URL=http://127.0.0.1:5000 BACKEND_EXTERNAL_URL=http://host.docker.internal:5000 CELERY_BROKER_URL=redis://127.0.0.1:6379/0 CELERY_RESULT_BACKEND=redis://127.0.0.1:6379/0 poetry run celery --app yaptide.celery.simulation_worker worker --events -P eventlet --hostname yaptide-simulation-worker --queues simulations --loglevel=warning

Windows (PowerShell)

$Env:PATH += ";" + (Join-Path -Path (Get-Location) -ChildPath "bin"); $env:BACKEND_INTERNAL_URL="http://127.0.0.1:5000"; $env:BACKEND_EXTERNAL_URL="http://host.docker.internal:5000"; $env:CELERY_BROKER_URL="redis://127.0.0.1:6379/0"; $env:CELERY_RESULT_BACKEND="redis://127.0.0.1:6379/0"; poetry run celery --app yaptide.celery.simulation_worker worker --events -P eventlet --hostname yaptide-simulation-worker --queues simulations --loglevel=warning

Why these variables?

• PATH — so the worker can find the SHIELD-HIT12A binary in bin/
• BACKEND_INTERNAL_URL — the worker reports progress back to Flask at this address
• CELERY_BROKER_URL / CELERY_RESULT_BACKEND — connect to Redis for task dispatch and result storage

6. Start the Celery helper worker

The helper worker handles post-processing tasks (collecting results, cleanup). Open a second backend terminal and go to the yaptide/ directory. Run:

Linux

FLASK_SQLALCHEMY_DATABASE_URI=sqlite:///instance/db.sqlite BACKEND_INTERNAL_URL=http://127.0.0.1:5000 BACKEND_EXTERNAL_URL=http://host.docker.internal:5000 CELERY_BROKER_URL=redis://127.0.0.1:6379/0 CELERY_RESULT_BACKEND=redis://127.0.0.1:6379/0 poetry run celery --app yaptide.utils.helper_worker worker --events --hostname yaptide-helper-worker --queues helper --loglevel=warning

Windows (PowerShell)

$env:FLASK_SQLALCHEMY_DATABASE_URI="sqlite:///instance/db.sqlite"; $env:BACKEND_INTERNAL_URL="http://127.0.0.1:5000"; $env:BACKEND_EXTERNAL_URL="http://host.docker.internal:5000"; $env:CELERY_BROKER_URL="redis://127.0.0.1:6379/0"; $env:CELERY_RESULT_BACKEND="redis://127.0.0.1:6379/0"; poetry run celery --app yaptide.utils.helper_worker worker --events --hostname yaptide-helper-worker --queues helper --loglevel=warning

7. Start the Flask API

Open a third backend terminal and go to the yaptide/ directory. Run:

Linux

FLASK_USE_CORS=True FLASK_SQLALCHEMY_DATABASE_URI="sqlite:///db.sqlite" CELERY_BROKER_URL=redis://127.0.0.1:6379/0 CELERY_RESULT_BACKEND=redis://127.0.0.1:6379/0 poetry run flask  --app yaptide.application run --host 0.0.0.0

Windows (PowerShell)

$env:FLASK_USE_CORS="True"; $env:FLASK_SQLALCHEMY_DATABASE_URI="sqlite:///db.sqlite"; $env:CELERY_BROKER_URL="redis://127.0.0.1:6379/0"; $env:CELERY_RESULT_BACKEND="redis://127.0.0.1:6379/0"; poetry run flask --app yaptide.application run --host 0.0.0.0

Why FLASK_USE_CORS=True? During development, the frontend runs on localhost:3000 and the backend on localhost:5000. Without CORS enabled, the browser blocks cross-origin requests.

This creates db.sqlite inside ./instance/ (default Flask instance folder).

--host 0.0.0.0 flag allows the SLURM container to reach the backend at http://host.docker.internal:5000 (the host machine).

8. Start Slurm, Keycloak and CA containers

Local slurm setup emulates PLGrid SLURM cluster settings and authentication. It requires multiple services to run, including a slurm database, a Keycloak instance, and a certificate authority. All of these can be started with a single command:

docker compose -f docker-compose.slurm.yml up --detach

After the containers are up, run the setup script:

./scripts/setup_local_slurm.sh

If you want to connect to the slurm shell, run (this is not required for the setup):

docker exec -it slurmctld /bin/bash

The docker compose and setup script do the following things:

• Set up auth as described here
• Mount shared /tmp/scratch on all nodes
• Add $SCRATCH variable /tmp/scratch/{USER}
• Create devuser in the Slurm cluster
• Add the cluster to yaptide database
• Copy the Shieldhit binary into the cluster
• Create a LMOD module for Shieldhit
• Install pymchelper library on all nodes (required for merging results)

Caution

The slurm container must be able to reach the backend API at http://host.docker.internal:5000 (the host machine), which might require additional configuration on some operating systems.

Frontend setup

Exit the yaptide/ directory in the 4th terminal:

cd ..

Clone the frontend repo:

git clone https://github.com/yaptide/ui.git

Navigate to the ui directory:

cd ui

Pull the converter submodule. The converter is a standalone Python package that translates the editor’s JSON project format into native input files for simulation engines:

git submodule update --init --recursive

Install frontend dependencies:

npm install

Create a .env file in the ui/ directory:

ui/.env

REACT_APP_ALT_AUTH='plg'
REACT_APP_KEYCLOAK_BASE_URL='http://127.0.0.1:8080'
REACT_APP_KEYCLOAK_REALM='yaptide'
REACT_APP_KEYCLOAK_CLIENT_ID='yaptide-app'
REACT_APP_BACKEND_URL='http://127.0.0.1:5000'

Start the frontend

npm run start

Open http://127.0.0.1:3000. Click “Connect with PLGrid” and log in with username: devuser, password: password. The setup is complete now. The page reloads on edits.

Caution

Make sure to access frontend using http://127.0.0.1:3000. http://localhost:3000 will not work due to Keycloak security policies.

Code formatting

The backend uses Ruff (ruff check + ruff format) via pre-commit hooks:

poetry run pre-commit install

After installing, hooks run automatically on every git commit. If a hook fails:

1. The commit is aborted
2. Some hooks auto-fix files (for example, the ruff check --fix and ruff format hooks) — just commit again
3. Other issues are reported in the terminal for manual fixing

To run all hooks manually:

pre-commit run --all-files

Frontend uses prettier for formatting, run it with:

npm run format