Skip to content

Testing migrations

Set-up

An NPL migration involves applying one or more changes specified in the migration.yml file. The testing tool, EngineMigrationTester, facilitates the execution of changes either all at once or individually, enabling comprehensive testing of new source code, migration configurations, and procedures. Migration tests can be written in Kotlin, using the EngineMigrationTester API to interact with the engine, or directly in NPL.

To get started,

  1. Specify the directory where the migration artifacts reside. To see what artifacts are necessary, see Migrations - getting started.
  2. Use a classpath resolver to determine the actual system location of the migration artifact directory.
  3. Provide various configuration details using the MigrationInMemoryTestConfiguration object.
  4. Write the test.
  5. Instantiate the tester.

Running a migration

In a single step

To run all changes and then verify, simply use the run method and then execute any necessary assertions.

// run all changes in migration
tester.run()

//  ... assertions verifying that the migration was successful ...

Step by step

To run multiple changes independently, use the runTo function. The runTo function will apply the changes in order, determined by the descriptor file, up to and including the change with the specified changeset name. All previously migrated steps will be skipped. This allows you to verify that the migration script and process have been successful.

// run to specific change
tester.runTo("1.0.2")

//  ... assertions verifying specified change was migrated successfully ...

tester.runTo("1.0.3")

//  ... assertions verifying specified change was migrated successfully ...

Verifying a change

To validate a changes, assertions can be made in both Kotlin and NPL. E.g.:

NPL

@test
function test101(test: Test) -> {
    var foo = loadProtocol<Foo>(test, "foo").getOrFail();

    test.assertEquals(100, foo.getValue[p]());
};

Kotlin

val allStates = tester
    .getAllProtocolStates()
    .map { it.value }
    .toList()
assertEquals(listOf(calculator101.value), allStates)

val allCalculatorStates = tester
    .getCurrentProtocolStates(prototypeId("1.0.1"))
    .map { it.value }
    .toList()
assertEquals(listOf(calculator101.value), allCalculatorStates)

Writing tests

Using Kotlin

This method uses the Engine API to handle all protocol interaction with the kernel. This method requires the developer to be more aware of system level details such as protocol IDs, qualified type names and the protocol variable frame. This can sometimes require casting and other workarounds to make the tests more concise and easier to understand.

The Engine API enables you to create protocols, execute protocol actions, and retrieve the current protocol state. Examples can be found below.

JavaDocs

For detailed specifications of the EngineMigrationTester class' public methods, refer to the published Javadocs.

Example

In the following example, we'll use a Calculator protocol to illustrate how to test each change separately by asserting success after each changeset. To see more information about available operations for the NPL Number type, click here.

Calculator.npl

Version 1.0.0

protocol[party] Calculator() {
    var value = 0;
    permission[party] add(addend: Number) returns Number {
        value = (value + addend);
        return value;
    }
    permission[party] subtract(subtrahend: Number) returns Number {
        value = (value - subtrahend);
        return value;
    }
    permission[party] setValue(newValue: Number) { value = newValue; }
    permission[party] clear() { value = 0; }
}

Added multiply and divide permissions

Version 1.0.1

protocol[party] Calculator(private var value: Number) {
    permission[party] add(addend: Number) returns Number {
        value = (value + addend);
        return value;
    }
    permission[party] subtract(subtrahend: Number) returns Number {
        value = (value - subtrahend);
        return value;
    }
    permission[party] multiply(factor: Number) returns Number {
        value = (value * factor);
        return value;
    }
    permission[party] divide(divisor: Number) returns Number {
        require(divisor > 0, "Divisor cannot be zero");
        value = (value / divisor);
        return value;
    }
    permission[party] setValue(newValue: Number) { value = newValue; }
    permission[party] getValue() returns Number { return value; };
    permission[party] clear() { value = 0; }
}

To begin, instantiate the tester by providing the migration artifacts root directory and the Configuration object.

fun getTester(directory: File) = EngineMigrationTester(
    migrationHome = directory,
    configuration = MigrationInMemoryTestConfiguration(),
)

Next, run the migration using the run or runTo methods as mentioned earlier. Follow up with the necessary assertions to ensure the migration was successful

test("A sample of using EngineMigrationTester using Kotlin") {
    fun Value.toNumber() = (this as? NumberValue)?.value?.intValueExact() ?: 0
    fun prototypeId(version: String) = "/TEST_SYSTEM-$version?/npl/Calculator"

    val party = PartyValue("accountant")
    val migrationRoot = getCalculatorMigrationBuilder(tempdir()).build()

    getTester(migrationRoot).use { tester ->
        tester.runTo("1.0.0")

        val protocolId = tester.create(
            prototypeId = prototypeId("1.0.0"),
            arguments = listOf(),
            parties = listOf(party),
            observers = mapOf(),
        ).id

        tester.selectAction(
            protocolId = protocolId,
            action = "add",
            arguments = listOf(NumberValue(30)),
            party = party,
        ).toNumber().also { got ->
            assertEquals(30, got)
        }

        tester.selectAction(
            protocolId = protocolId,
            action = "subtract",
            arguments = listOf(NumberValue(15)),
            party = party,
        ).toNumber().also { got ->
            assertEquals(15, got)
        }

        tester.selectAction(protocolId, "clear", listOf(), party)

        val calculator100 = tester.getProtocolStateById(protocolId)
        assertNotNull(calculator100)
        assertEquals(0, calculator100.value.frame.slots["value"]?.toNumber())

        tester.selectAction(
            protocolId = protocolId,
            action = "setValue",
            arguments = listOf(NumberValue(24)),
            party = party,
        )

        tester.runTo("1.0.1")

        val calculator101 = tester.getProtocolStateById(protocolId)
        assertNotNull(calculator101)
        assertEquals(
            expected = 24,
            actual = calculator101.value.frame.slots["value"]?.toNumber(),
        )

        tester.selectAction(
            protocolId = protocolId,
            action = "divide",
            arguments = listOf(NumberValue(12)),
            party = party,
        ).toNumber().let { got ->
            assertEquals(2, got)
        }

        tester.selectAction(
            protocolId = protocolId,
            action = "multiply",
            arguments = listOf(NumberValue(8)),
            party = party,
        ).toNumber().let { got ->
            assertEquals(16, got)
        }

        val allStates = tester
            .getAllProtocolStates()
            .map { it.value }
            .toList()
        assertEquals(listOf(calculator101.value), allStates)

        val allCalculatorStates = tester
            .getCurrentProtocolStates(prototypeId("1.0.1"))
            .map { it.value }
            .toList()
        assertEquals(listOf(calculator101.value), allCalculatorStates)
    }
}

Using NPL

New feature (added in 2024.1.8)

Tests written in NPL are easier to write and understand. Protocols are instantiated without the need to save the protocol ID or specify the fully qualified type. All types are handled natively, as they would be in an NPL application. Access to functions, permissions and data is determined by protocol access modifiers. The developer no longer needs to be concerned with anything but the NPL constructs needed for the test itself.

The runNPL method can execute tests using either an inline snippet or a Path object pointing to a file that contains the test source. Whether using a test source file or a code snippet, one or more tests can be specified. This collection of tests is referred as a test set. Tests in a test set are not guaranteed to be executed in any particular order.

It is possible to save a protocol for reference in subsequent runNPL invocations. To do this, you must first store the protocol in the test where it was instantiated using the storeProtocol method. Along with the protocol instance, a unique key must be provided. This key will be used to load the protocol in later tests. Protocols must be stored if you want to preserve state changes. Conversely, if you do not wish to preserve state changes that occur during the test, simply avoid using storeProtocol.

storeProtocol can be invoked multiple times with any protocol. However, a storage key can be used only once per test set. If using an existing key, the given reference will replace the previous one, making the previous reference inaccessible.

To retrieve a stored protocol, use the loadProtocol method. Supply the expected protocol type as a type argument along with the key used to store it. An Optional of the fully typed protocol will be returned. If no protocol is associated with the given key, it will return an empty Optional. If the type of the associated protocol does not match the specified type, an error will be thrown.

Example

Below is the Kotlin test example provided earlier, now converted to use NPL.

getTester(migrationRoot).use { tester ->
    tester.runTo("1.0.0")
    tester.runNPL(
        name = "test-1.0.0",
        snippet = """
            package test;

            use npl.Calculator;

            const party = 'accountant';

            @test
            function calculator100Test(test: Test) -> {
                var calculator = Calculator[party]();

                test.assertEquals(30, calculator.add[party](30));
                test.assertEquals(15, calculator.subtract[party](15));

                calculator.clear[party]();
                test.assertEquals(0, calculator.value);
                calculator.setValue[party](24);

                storeProtocol(test, "calc", calculator);
            };
        """.trimIndent(),
    )

    tester.runTo("1.0.1")
    tester.runNPL(
        name = "test-1.0.1",
        snippet = """
            package test;

            use npl.Calculator;

            const party = 'accountant';

            @test
            function calculator101Test(test: Test) -> {
                var calculator = loadProtocol<Calculator>(test, "calc")
                                    .getOrFail();

                test.assertEquals(24, calculator.getValue[party]());

                calculator.divide[party](12);
                test.assertEquals(2, calculator.getValue[party]());

                calculator.multiply[party](8);
                test.assertEquals(16, calculator.getValue[party]());
            };
        """.trimIndent(),
    )
}

Using Kotlin and NPL

Currently, within NPL, certain inspections are not allowed due to access restrictions. Naturally, these would not be possible in native NPL tests. When testing a migration scenario, it may be desirable to verify these restricted elements. For this reason, it is still possible, using Kotlin, to perform any extra assertions that are not possible natively in NPL.

Version 1.0.0

protocol[p] Foo(var n: Number) {}

Version 1.0.1

protocol[p] Foo(private var n: Number) {
    permission[p] getValue() returns Number { return n; }
}


In the example below, the current tag is TEST_SYSTEM-1.0.1, given that the tester has run up to change 1.0.1

tester.runTo("1.0.1")

// Run tests in NPL and assert
tester.runNPL(
    name = "test-1.0.1",
    snippet = """
        package test;

        use npl.Foo;

        const p = 'aParty';

        @test
        function test101(test: Test) -> {
            var foo = loadProtocol<Foo>(test, "foo").getOrFail();

            test.assertEquals(100, foo.getValue[p]());
        };
    """.trimIndent(),
)

// Run further assertions in Kotlin
val prototypeId = "/TEST_SYSTEM-1.0.1?/npl/Foo"
val foo = tester.getCurrentProtocolStates(prototypeId).toList().single()

fun ProtocolState.partyEntityValues(name: String) =
    value.parties()[name]?.entity?.value?.get("party")

assertEquals(setOf("aParty"), foo.partyEntityValues(name = "p"))