System topology
High level overview
The image below sketches out the components of the NOUMENA environment. At the center we have the three components that constitute the NOUMENA runtime: the Engine, the (optional) History application and the (optional) Read Model. These components all connect to a PostgreSQL database.
Applications are deployed in the engine, after which they become available through all three components. Authorization is managed using an Identity and Access Management of choice, as long as it supports OIDC.
We typically suggest running a reverse proxy in front of the runtime.
Example topology
Depending on individual preferences and requirements there are several ways to realize the environment. The diagram below shows an example of a possible configuration.
Example configuration
Docker compose file
Realising such a set-up can be achieved with a docker compose file. What follows is a minimal working example:
services:
# the engine requires a PostgreSQL database
engine-db:
image: postgres:14.12-alpine
mem_limit: 256m
environment:
- POSTGRES_DB=platform
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- ENGINE_DB_USER=engine
- ENGINE_DB_PASSWORD=engine_pwd
- HISTORY_DB_USER=history
- HISTORY_DB_PASSWORD=history_pwd
- READ_MODEL_DB_USER=read_model
- READ_MODEL_DB_PASSWORD=read_model_pwd
volumes:
# refer to database user setup script example (below) for details on how to set up database users
- ./bin/init-db-users.sh:/docker-entrypoint-initdb.d/init-db.sh
healthcheck:
test: pg_isready -U postgres
interval: 1s
timeout: 5s
retries: 50
engine:
# refer to Dockerfile example (below) for what would usually get wrapped around an engine image
build: .
ports:
- "12000:12000"
mem_limit: 512m
environment:
ENGINE_DB_URL: "jdbc:postgresql://engine-db/platform"
ENGINE_DB_USER: engine
ENGINE_DB_PASSWORD: engine_pwd
ENGINE_DB_READ_MODEL_USER: read_model
ENGINE_DB_HISTORY_USER: history
ENGINE_DB_HISTORY_SCHEMA: history
ENGINE_ALLOWED_ISSUERS: "http://keycloak:11000/realms/noumena"
depends_on:
engine-db:
condition: service_healthy
keycloak:
condition: service_healthy
history:
# For a production setup, we strongly recommend to use the *actual* version number!
image: ghcr.io/noumenadigital/packages/history:latest
ports:
- "12010:12010"
mem_limit: 512m
environment:
HISTORY_DB_URL: "jdbc:postgresql://engine-db/platform"
HISTORY_DB_USER: history
HISTORY_DB_PASSWORD: history_pwd
depends_on:
engine-db:
condition: service_healthy
engine:
condition: service_healthy
read-model:
# For a production setup, we strongly recommend to use the *actual* version number!
image: ghcr.io/noumenadigital/packages/read-model:latest
ports:
- "5555:5555"
environment:
READ_MODEL_DB_URL: "postgres://read_model:read_model_pwd@engine-db:5432/platform"
READ_MODEL_DB_USER: read_model
READ_MODEL_ALLOWED_ISSUERS: "http://keycloak:11000/realms/noumena"
READ_MODEL_ENGINE_HEALTH_ENDPOINT: "http://engine:12000/actuator/health"
depends_on:
engine-db:
condition: service_healthy
engine:
condition: service_healthy
# the engine requires an OIDC provider for authentication
# Keycloak provides a lot of options to get this up and running
keycloak:
image: quay.io/keycloak/keycloak:23.0
command: start --hostname-strict=false --hostname-strict-https=false --metrics-enabled=true --db=postgres
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: Keycloak123!
KC_DB_URL: "jdbc:postgresql://keycloak-db/postgres"
KC_DB_USERNAME: postgres
KC_DB_PASSWORD: testing
KC_HEALTH_ENABLED: "true"
KC_HTTP_ENABLED: "true"
KC_HTTP_PORT: 11000
KC_HOSTNAME: keycloak
healthcheck:
test: ["CMD-SHELL", "exec 3<>/dev/tcp/127.0.0.1/11000;echo -e 'GET /health/ready HTTP/1.1\r\nhost: http://localhost\r\nConnection: close\r\n\r\n' >&3;if [ $? -eq 0 ]; then echo 'Healthcheck Successful';exit 0;else echo 'Healthcheck Failed';exit 1;fi;"]
interval: 1s
retries: 60
depends_on:
keycloak-db:
condition: service_healthy
keycloak-db:
image: postgres:14.12-alpine
mem_limit: 256m
environment:
POSTGRES_PASSWORD: testing
healthcheck:
test: pg_isready -U postgres
interval: 1s
timeout: 5s
retries: 50
To run this set-up, execute the following command:
docker compose up --wait --build
Refer to the docker configuration example sections of the Engine, History, and Read Model applications for more details about setting up individual applications.
Dockerfile
To deploy an engine image with NPL source code, you can wrap the engine image and add NPL, which gets deployed during engine startup.
# For a production setup, we strongly recommend to use the *actual* version number!
FROM ghcr.io/noumenadigital/packages/engine:latest
# Files in npl/npl-1.0.0 contain NPL source code
COPY npl/migration.yml migration.yml
COPY npl/npl-1.0.0 npl-1.0.0
# Copy migration files to the /migrations folder
COPY npl/npl-1.0.0/* /migrations/npl-1.0.0/
Database setup
We recommend setting up distinct database users for the different applications. What follows is the example script used
in the docker compose file, init-db-users.sh
:
#!/bin/sh
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE ROLE "$ENGINE_DB_USER" LOGIN PASSWORD '$ENGINE_DB_PASSWORD' NOINHERIT;
GRANT CREATE ON DATABASE "$POSTGRES_DB" TO "$ENGINE_DB_USER";
--
CREATE ROLE "$READ_MODEL_DB_USER" LOGIN PASSWORD '$READ_MODEL_DB_PASSWORD' NOINHERIT;
GRANT CONNECT ON DATABASE "$POSTGRES_DB" TO "$READ_MODEL_DB_USER";
--
CREATE ROLE "$HISTORY_DB_USER" LOGIN PASSWORD '$HISTORY_DB_PASSWORD' NOINHERIT;
GRANT CREATE ON DATABASE "$POSTGRES_DB" TO "$HISTORY_DB_USER";
EOSQL