Skip to content

Deploying migrations

You can deploy your migrations either at engine startup or via the management API.

Deploying during engine start up

The time it takes to apply migrations can and does vary, and applications will experience downtime while they are being applied. The duration of the downtime will depend on the number of protocols that exist in the system. It is therefore better to have the engine be offline while a migration occurs. The files which constitute a migration (NPL sources, Kotlinscript files, and migration descriptor) can be placed in any local directory or system mount. It will need to be configured in the docker setup as a volume mount.

Listed below are the step-by-step instructions to apply migrations at startup.

  1. Create any needed migration files. Ascertain that the migration does not break existing systems, or ensure that existing systems have been adapted accordingly. Pay attention to changing protocol interfaces within or outside of NPL, and existing protocol references that may no longer be present.
  2. Place all migration files within the desired location. The files can be organized within sub-directories as long as they are contained under the configured directory, which acts as a root directory.
  3. All migrations are transactional, so the changes will not persist unless the migration was a success. However, to ensure that there are no other issues, it is strongly encouraged to try the migration on a test system first.
  4. Configure a volume mount linking the local directory containing the migration artifacts to its container counterpart.
  5. Set the named ENGINE_NPL_MIGRATION_DIRECTORY_PATH environment variable with the absolute path to the container location of the migration components. This is an optional step, /migrations is the default value.
  6. Start the platform, wait for it to complete the startup process, and inspect the output. The engine logs will provide information about any migrations that have been applied, and any potential issues.

Upon success, the engine will run as normal using the newly migrated system.

Upon failure, the engine startup will halt with an error status. The engine logs will contain any migration errors that have occurred. If a failure occurred, fix any errors in the migration and then restart the engine.

Example migration using the Seed Project

If you don't already have it, clone the Seed Project and follow the README instructions.

Example Docker config

FROM registry.noumenadigital.com/noumena/platform/engine:latest

# Set migration artifact directory, and set feature flag to "true"
ENV ENGINE_NPL_MIGRATION_DIRECTORY_PATH="/npl"

# Configure docker mount point
COPY src/main/yaml /npl/yaml
COPY src/main/npl  /npl/npl-1.0.0

Start the engine (and initiate the migration)

Run the following command:

make run

Once the process is finished, run the command below to see the logs. The Dockerfile is already configured.

docker-compose logs engine

Engine log excerpt of a successful migration

---------------------------------------------------------------------
NPL Migration Log (id: 94c23cc4-9b34-49cf-8376-7f0d1c047155, state: OPENED):

┌─────┬────────────────┬────┬──────────┬──────────────────┬─────┬─────────────────────────────
│     │Platform version│Type│Result    │Id                │Entry│When
├─────┼────────────────┼────┼──────────┼──────────────────┼─────┼─────────────────────────────
│00001│2021.1.175      │run │SUCCESSFUL│1.0.0::0::npl-load│npl  │2022-09-01T13:47:33.921240Z -
│     │                │    │          │                  │     │2022-09-01T13:47:38.297954Z
└─────┴────────────────┴────┴──────────┴──────────────────┴─────┴─────────────────────────────

2022-09-01 13:47:38 INFO  Completing migration 94c23cc4-9b34-49cf-8376-7f0d1c047155}
2022-09-01 13:47:38 INFO  NPL migrations were successfully applied.
2022-09-01 13:47:49 INFO  Exposing 5 endpoint(s) beneath base path '/actuator'
2022-09-01 13:47:50 INFO  Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-09-01 13:47:50 INFO  Initializing Servlet 'dispatcherServlet'
2022-09-01 13:47:50 INFO  Completed initialization in 4 ms
2022-09-01 13:47:51 INFO  Started task scheduler.
2022-09-01 13:47:51 INFO  Starting notifications handler.
2022-09-01 13:47:51 INFO  Listening to PostgreSQL notifications.

Deploying via the Application Management API

Given all the files necessary for a migration (NPL sources, Kotlinscript files, and migration descriptor), they can be deployed as a zip file directly onto a running engine using the Application Management API's deployment endpoint

For example, the following directory

> tree deployment
deployment
├── kotlin-script
│   ├── S0001_migration.kts
│   ├── S0002_test.kts
│   └── S0003_migration.kts
├── npl-1.0.0
│   ├── Contract.npl
│   └── Entry.npl
├── npl-1.0.1
│   ├── Contract.npl
│   └── lib
│       └── Calc.npl
├── npl-1.0.2
│   ├── experimental
│   │   └── ExperimentalFeature.npl
│   └── prod
│       ├── Contract.npl
│       └── lib
│           └── Calc.npl
└── migration.yml

can be zipped up and deployed as follows:

> cd deployment
> zip -r deployment.zip *
> curl -X PUT \
  'http://localhost:12400/management/application' \
  -H "Authorization: Bearer ${IAM_TOKEN}" \
  -H 'Content-Type: multipart/form-data' \
  -F 'archive=@deployment.zip;type=application/zip'