Skip to content

Engine application

Background

The Operating Engine consists of the engine application, and its companion applications:

This document describes the setup for the engine application.

Configuration

Docker image

The engine application is provided as a docker image which includes a default configuration.

API

The engine application runs on port 12000 by default. The engine admin endpoints are exposed on port 12700 by default. The engine management endpoints are exposed on port 12400 by default.

Database user

A dedicated database user for the engine application should be created and configured for the application to start (refer to the example setup). Since the engine application manages its own database schema, make sure to provide it with GRANT CREATE ON DATABASE privileges.

Environment variables

Deprecated environment variables and changes in default values of feature flags are documented here.

Feature flags

The following environment variables can be configured to enable or disable feature flags:

Environment variable Description Default value
FEATURE_NON_STANDARD_NPL_TYPES Feature flag - enables recognition of non-standard types as NPL types to prevent redefinition false

Engine

The following environment variables can be configured for the engine application:

Environment variable Description Default value
ENGINE_DEV_MODE Engine running mode - indicates whether application runs in development mode false
ENGINE_SPRING_ACTUATOR_EXPOSE Engine monitoring - exposed Spring actuator endpoints info,health,service-health,metrics,prometheusmetrics
ENGINE_PORT Engine Core API port 12000
ENGINE_ADMIN_HOST Engine Admin API host 127.0.0.1
ENGINE_ADMIN_PORT Engine Admin API port 12700
ENGINE_MANAGEMENT_HOST Engine Management API host 127.0.0.1
ENGINE_MANAGEMENT_PORT Engine Management API port 12400
ENGINE_MANAGEMENT_ARCHIVE_SIZE_LIMIT Engine Management API archive size limit 10000000
ENGINE_NPL_MIGRATION_DIRECTORY_PATH NPL migrations - path to the folder where the NPL migration artifacts reside /migrations
ENGINE_NPL_MIGRATION_RUN_ONLY NPL migrations - environment identifier tag to be matched against change descriptor attribute 'run-only' values
ENGINE_SESSION_STATE_READ_CACHE_MAX_ENTRIES Runtime session - state read cache size 500
ENGINE_STREAMS_BUFFER_MAX_SIZE Streams - buffer streams until maximum size has been reached 50
ENGINE_STREAMS_BUFFER_MAX_TIMEOUT Streams - buffer streams until timeout has elapsed PT0.5S
ENGINE_PROTOBUF_DESER_RECURSION_LIMIT Protobuf - deserialization recursion limit 100

Database

Environment variable Description Default value
ENGINE_DB_URL Engine DB - URL jdbc:postgresql://localhost:5432/platform
ENGINE_DB_USER Engine DB - user platform_owner
ENGINE_DB_PASSWORD Engine DB - password n/a (mandatory value)
ENGINE_DB_SCHEMA Engine DB - schema noumena
ENGINE_DB_HISTORY_USER Engine DB - History user n/a (optional value)
ENGINE_DB_HISTORY_SCHEMA Engine DB - History schema history
ENGINE_DB_READ_MODEL_USER Engine DB - Read Model user n/a (optional value)
ENGINE_DB_SSL Engine DB - use SSL false
ENGINE_DB_POOL_MAXIMUM_POOL_SIZE Engine DB - maximum size of the connection pool 10
ENGINE_DB_POOL_MAXIMUM_CONNECTION_LIFETIME Engine DB - maximum lifetime of a connection in the pool PT30M

AMQP publication

The following environment variables can be configured to set up the engine's publication of notifications to an AMQP broker:

Environment variable Description Default value
AMQP_BROKER_URL The URL of the AMQP broker. AMQP publication will be completely disabled if this is left unset.
AMQP_QUEUE_NAME The name of the queue to publish notifications to. messages
AMQP_USERNAME The username to use when connecting to the broker. guest
AMQP_PASSWORD The password to use when connecting to the broker.
AMQP_POLLING_PERIOD_SECONDS The period between polling the internal publication queue for new notifications to publish. 5
AMQP_CONNECTION_TIMEOUT_SECONDS The timeout for establishing a connection to the broker. 5
AMQP_BATCH_SIZE The maximum number of notifications to retrieve from the internal postgres queue. Does not affect the client publication behaviour. 10

NPL Contributor Libraries

Environment variable Description Default value
NPL_CONTRIB_PATH The default location of NPL contributor libraries. /npl-contrib

Authentication

The following environment variables can be configured to set up the engine's integration with the authentication server(s):

Environment variable Description Default value
ENGINE_ALLOWED_ISSUERS Comma-delimited list of trusted JWT issuers. JWTs are only accepted if the value in the iss field matches one of these values exactly. http://localhost:11000/realms/noumena
ENGINE_ISSUER_OVERRIDE Instead of extracting the issuer from the iss field, always use this URL as the issuer; for development purposes only

Logging

The following environment variables can be configured for finer control of the engine application's logging:

Environment variable Description Default value Example value
ENGINE_LOG_CONFIG Logging configuration (file path) n/a (optional value) classpath:/logback-json.xml
ENGINE_ROOT_LOG_LEVEL Logging level (root) INFO
ENGINE_LOG_LEVEL Logging level - Engine application INFO
ENGINE_AUTH_LOG_LEVEL Logging level - Authentication INFO
ENGINE_SPRING_LOG_LEVEL Logging level - Spring INFO
ENGINE_REQUEST_LOG_LEVEL Logging level - Http requests WARN

Swagger UI

The following environment variables can be configured to set up the engine application's Swagger UI:

Environment variable Description Default value
SWAGGER_ENGINE_URL Swagger UI - URL to connect to the Engine Core API http://localhost:${ENGINE_PORT}
SWAGGER_ENGINE_ADMIN_URL Swagger UI - URL to connect to the Engine Admin API http://${ENGINE_ADMIN_HOST}:${ENGINE_ADMIN_PORT}
SWAGGER_ENGINE_MANAGEMENT_URL Swagger UI - URL to connect to the Engine Management API http://${ENGINE_MANAGEMENT_HOST}:${ENGINE_MANAGEMENT_PORT}
SWAGGER_SECURITY_AUTH_URL Swagger UI - URL to connect to the Authentication API http://localhost:11000/realms/noumena
SWAGGER_SECURITY_CLIENT_ID Swagger UI - IAM client ID nm-platform-service-client
SWAGGER_SECURITY_CLIENT_SECRET Swagger UI - IAM client secret 87ff12ca-cf29-4719-bda8-c92faa78e3c4

Example setup

Database user setup

The following SQL script creates a dedicated database user for the engine application.

-- the use of 'noinherit' is regarded good practice, engine user privileges are obtained by explicit 'set role'
create role engine login password <pwd> noinherit;
grant create on database mydatabase to engine;

Please refer to the database setup example for the Read Model and history application for further details.

Docker configuration

# engine application (here is a sub-selection to get you started)
engine:
  image: ghcr.io/noumenadigital/packages/engine:latest
  depends_on:
    - postgres_db
  ports:
    - "${ENGINE_PORT:-12000}:12000"
    - "${ENGINE_ADMIN_PORT:-12700}:12700"
    - "${ENGINE_MANAGEMENT_PORT:-12400}:12400"
  environment:
    ENGINE_ADMIN_HOST: ${ENGINE_ADMIN_HOST:-0.0.0.0}
    ENGINE_MANAGEMENT_HOST: ${ENGINE_MANAGEMENT_HOST:-0.0.0.0}
    ENGINE_DB_URL: "jdbc:postgresql://postgres_db/platform"
    ENGINE_DB_SCHEMA: engine-schema
    ENGINE_DB_USER: engine
    ENGINE_DB_PASSWORD: <pwd>
    ENGINE_DB_READ_MODEL_USER: read_model
    ENGINE_DB_HISTORY_USER: history
    ENGINE_DB_HISTORY_SCHEMA: history-schema
    SWAGGER_ENGINE_ADMIN_URL: "http://localhost:12700/"
    SWAGGER_ENGINE_MANAGEMENT_URL: "http://localhost:12400/"
    ENGINE_ALLOWED_ISSUERS: "http://localhost:11000/realms/noumena"
    # ...

For further configuration options, see environment variables.

A complete docker compose file example can be found here.

Health check

Once the engine application is deployed, you can test its healthcheck endpoint with:

curl localhost:12000/actuator/health

Swagger UI

You can explore the APIs provided by the engine application via Swagger UI, available at http://localhost:12000/swagger-ui/.