Skip to content

Migration DSL

NPL Migrations are used to modify existing prototypes or introduce new ones. The Kotlin migration DSL (Domain Specific Language) is used to write such migrations.

Glossary

Before diving in to the specifics of the migration DSL, let's define terminology used.

The term prototype is a colloquial name for a user-defined NPL type, such as a protocol or a struct type. A unique ID for a prototype is called prototype ID. Note that, in the migration DSL, we also use type ID to refer to a prototype ID.

A prototype ID consists of the tag and the fully-qualified type name. Do not confuse the prototype ID with the simple name of an NPL type, which is the unqualified type name used at the declaration site within NPL sources.

The tag is the NPL version, typically consisting of the system under audit plus a version number. For more details, refer to the migration descriptor docs. Upgrading to a target NPL version typically means lifting the version number, say from "app-1.0.0" to "app-2.0.0". Migrating prototypes between two NPL versions is also called re-tagging.

In an NPL migration, you will want to define data mappings between old (or, current) and new (or, target) prototypes in order to accommodate changes in the NPL code between the two NPL versions.

For an example, consider the following protocol definition:

package my.pkg

protocol[p] MyTermsAndConditions() {};

The simple name of this protocol type is "MyTermsAndConditions".

Given the current tag (NPL version) "app-1.0.0" and the qualified path "/my/pkg", it's prototype ID will be "/app-1.0.0?/my/pkg/MyTermsAndConditions".

Given the target tag "app-2.0.0", the target prototype ID after re-tagging will become "/app-2.0.0?/my/pkg/MyTermsAndConditions".

Note that, in this example, neither the qualified path nor the simple name of this prototype have changed between the two NPL versions.

Migration integration

In a big project migrations can be very big and in many cases contain a lot of hard-coded strings, repetitive blocks, and boilerplate. This invites a lot of copy/paste practices that can lead to errors.

To help with this both the migration runner and DSL provide some convenience functions to make migrations more concise and less error-prone.

In a typical migration only a small portion of the protocol instances need a custom transformation that alters their states/data. The others merely need to have the instance ID (prototypeId) which they point to, changed. This should not be thought of as a no-op. The new NPL code can contain substantial logic changes for example. The migration DSL provides the retag(see retag) operation that will simplify the re-tagging of many prototypes.

The migration runner provides a function called mapPrototypesInMigration to discover all the prototype IDs in a migration. The function reduces the amount of hard-coded values in migration scripts and allows connecting any script with the migration context.

The simplest way to use mapPrototypesInMigration function in a migration script is as the following:

val prototypes = mapPrototypesInMigration()
val contract = prototypes.match("Contract")
val termsAndConditions = prototypes.match { endsWith("TermsAndConditions") }

migration("${prototypes.current} to ${prototypes.target}")
    .transformProtocol(contract.current, contract.target) {
        // do some custom transformation
    }
    .transformProtocol(termsAndConditions.current, termsAndConditions.target) {
        // do some custom transformation
    }
    // re-tag all other protocols, structs, identifiers, and enums
    .retag(prototypes)

The function mapPrototypesInMigration will read the migration.yml file loaded at runtime and create a PrototypeIdPairs instance that maps the last two changesets (second to last as current and last as target). This should be all that is needed in many situations.

Using match

Finding an IdPair-matching current prototype can be done in two ways:

With the default matcher:

val contract = prototypes.match("Contract")

The above will try to perform a simple match on prototypes containing "Contract". As a consequence it will not work for example if there are multiple prototypes having "Contract" in the name with suffixes (e.g. "Contract", "ContractA").

With a custom matcher:

// matches a prototype ID "/app-1.0.0?/my/pkg/MyTermsAndConditions"
val termsAndConditions = prototypes.match { endsWith("TermsAndConditions") }

Custom matchers allow you to apply custom logic while matching. In the provided example, endsWith is used to match, therefore if there are prototypes with suffixes (like "TermsAndConditionsB"), the matcher will still work correctly and match only the prototype ending with "TermsAndConditions".

Another example would be to match a prototype ID by its simple name (prototype ID without the tag and path):

fun matchSimpleName(name: String): IdPair = prototypes.match { ".*/$name\$".toRegex() matches this }

// matches a prototype ID "/app-1.0.0?/my/pkg/MyTermsAndConditions"
val termsAndConditions = prototypes.matchSimpleName("MyTermsAndConditions")

Special cases

The function mapPrototypesInMigration supports some simple special cases. However, if a migration is too complicated it might make sense to build the PrototypeIdPairs by hand, or use the lower-level mapPrototypes function.

mapPrototypesInMigration arguments allow to:

  • override protocols
  • override structs
  • override identifiers
  • override enums
  • override the current and/or target tag for all prototypes

Adding a prototype in target

When a new protocol appears in the target NPL there will be no re-mapping to do. So an override should be added to indicate that it should be ignored.

val prototypes = mapPrototypesInMigration(
    overrideProtocols = listOf(
        IdPair("", "/app-2.0.0?/example/SomeNewThing")
    )
)

note: similar overrides can also be specified for structs, identifiers, and enums

Discarding a prototype

When a prototype in the current system should not be carried forward then mark it as such with an override. Then retag will not try to re-tag it when no transforms have been defined.

val prototypes = mapPrototypesInMigration(
    overrideProtocols = listOf(
        IdPair("/app-1.0.0?/example/SomeOldThing", "")
    )
)

note: similar overrides can also be specified for structs, identifiers, and enums

Renaming a prototype

When a prototype has been renamed then the prototype mapper cannot derive these automagically. In this case an override should be added. The target value in the IdPair must match the name in the NPL for this override to be applied.

val prototypes = mapPrototypesInMigration(
    overrideProtocols = listOf(
        IdPair(
            "/app-1.0.0?/example/SomeBadlyNamedThing",
            "/app-2.0.0?/example/SomeBetterNamedThing"
        )
    )
)

note: similar overrides can also be specified for structs, identifiers, and enums

Custom current and/or target tags

If a system already containing NPL and protocol instances is being bootstrapped during a migration for the first time then we need a migration that will take whatever the state of the current system is to a known migrated state.

The migration descriptor file could look as follows:

systemUnderAudit: sys

  - name: 1.0
      changes:
          - migrate:
            dir-list: npl/app
            script: migration.kts

In migration.kts we should initialize the prototype pairs with the appropriate current tag which can not be derived from the migration.yml file.

// if no tag was used
val prototypes = mapPrototypesInMigration(current = "")

// if some arbitrary tag was used
val prototypes = mapPrototypesInMigration(current = "arbitrary-name-alpha")