Migration DSL

NPL Migrations are used to modify existing prototypes or introduce new ones. The Kotlin migration DSL (Domain Specific Language) is used to write such is the tool to write migrations.

Migration integration

In a big project migrations can be very big and in many cases contain a lot of hard-coded strings, repetitive blocks, and boilerplate. This invites a lot of copy/paste practices that can lead to errors.

To help with this both the migration runner and DSL provide some convenience functions to make migrations more concise and less error-prone.

In a typical migration only a small portion of the protocol instances need a custom transformation that alters their states/data. The others merely need to have the instance ID (prototypeId) which they point to, changed. This should not be thought of as a no-op. The new NPL code can contain substantial logic changes for example. The platform dsl provides the retag()(see: retag in noumena docs) operation that will simplify the re-tagging of many prototypes.

The migration runner provides a function called mapPrototypesInMigration to discover all the prototype IDs in a migration. The function reduces the amount of hard-coded values in migration scripts and allows connecting any script with the migration context. mapPrototypesInMigration is built on top of a simpler convenience function in the DSL: com.noumenadigital.platform.migration.dsl.MigrationHelperKt.mapPrototypes().

The simplest way to use mapPrototypesInMigration function in a migration script is as the following:

val protos = mapPrototypesInMigration()
val contract = protos.match("Contract")
val tac = protos.match { endsWith("TermsAndConditions") }

migration("${protos.current} to ${protos.target}")
    .transformProtocol(contract.current, contract.target) {
        // do some transformation
    }
    .transformProtocol(tac.current, tac.target) {
        // do some transformation
    }
    .retag(protos) // re-tag all other protocols, structs, identifiers, and enums

mapPrototypesInMigration() will read the migration.yml file loaded at runtime and create a PrototypeIdPairs instance that maps the last two changesets (second to last as 'current' and last as 'target'). This should be all that is needed in many situations.

Using match

Finding an IdPair-matching 'current' prototype can be done in two ways:

With the default matcher:

val contract = protos.match("Contract")

The above will try to perform a simple match on prototypes containing "Contract". As a consequence it will not work for example if there are prototypes having "Contract" in the name with suffixes (e.g. "Contract", "ContractA").

With custom matcher:

val tac = protos.match { endsWith("TermsAndConditions") }

Using custom matcher gives a possibility to apply any matcher. In the provided example endsWith is used to match, therefore if there are prototypes with suffixes (like "TermsAndConditionsB"), the matcher will still work correctly and match only the prototype ending with "TermsAndConditions".

Special cases

mapPrototypesInMigration() supports some simple special cases. However, if a migration is too complicated it might make sense to build the com.noumenadigital.platform.migration.dsl.PrototypeIdPairs by hand, or use the lower-level com.noumenadigital.platform.migration.dsl.MigrationHelperKt.mapPrototypes() function.

mapPrototypesInMigration() arguments allow to:

• override protocols
• override structs
• override identifiers
• override enums
• override the current and/or target tag for all prototypes

Adding a prototype in target

When a new protocol appears in the target npl there will be no re-mapping to do. So an override should be added to indicate that it should be ignored.

val protos = mapPrototypesInMigration(overrideProtocols = listOf(IdPair("", "/sys-2.0?/SomeNewThing")))

note: similar overrides can also be specified for structs, identifiers, and enums

Discarding a prototype

When a prototype in the current system should not be carried forward then mark it as such with an override. Then retag() will not try to re-tag it when no transforms have been defined.

val protos = mapPrototypesInMigration(overrideProtocols = listOf(IdPair("/sys-1.0?/SomeOldThing", "")))

note: similar overrides can also be specified for structs, identifiers, and enums

Renaming a prototype

When a prototype has been renamed then the prototype mapper cannot derive these automagically. In this case an override should be added. The target value in the IdPair must match the name in the npl for this override to be applied.

val protos = mapPrototypesInMigration(
    overrideProtocols = listOf(
        IdPair(
            "/sys-1.0?/SomeBadlyNamedThing",
            "/sys-1.0?/SomeBetterNamedThing"
        )
    )
)

note: similar overrides can also be specified for structs, identifiers, and enums

Custom current and/or target tags

If a system already containing npl and protocol instances is being bootstrapped during a migration for the first time then we need a migration that will take whatever the state of the current system is to a known migrated state.

The migration descriptor file could look as follows:

systemUnderAudit: sys

  - name: 1.0
      changes:
          - migrate:
            dir-list: npl/app
            script: migration.kts

In migration.kts we should initialize the prototype pairs with the appropriate current tag which can not be derived from the migration.yml file.

// if no tag was used
val protos = mapPrototypesInMigration(current = "")

// if some arbitrary tag was used
val protos = mapPrototypesInMigration(current = "arbitrary-name-alpha")