Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate build and tests to the latest version of the DAML SDK #75

Open
da-tanabe opened this issue Apr 1, 2020 · 1 comment
Open

Migrate build and tests to the latest version of the DAML SDK #75

da-tanabe opened this issue Apr 1, 2020 · 1 comment

Comments

@da-tanabe
Copy link
Contributor

da-tanabe commented Apr 1, 2020

dazl is having its third birthday soon, and has consequently evolved over the years and kept up with successive versions of the DAML SDK. The tooling and deployed artifacts had changed dramatically with DAML's open sourcing back in April 2019, but dazl continues to use an outdated mechanism for running Sandbox that is more inline with what the SDK looked like before that point.

Use the official DAML SDK Docker container for running Sandbox.

Many tests require that the Sandbox be running. dazl defines a function, dazl.sandbox(), that makes it easy to start a sandbox from DAML files. It was built under the assumption that Sandbox expects raw DAML files (was the case up until SDK 0.10), and hacked in to support building a DAR transparently from a root DAML file for subsequent versions. This workflow has greatly improved in the DAML SDK itself (daml start is always sufficient now), but dazl has not been updated to take advantage of this (and particularly the tests). To fully fix, this means:

  • Launch the Sandbox through the DAML SDK docker image. Unfortunately, because of the way CircleCI works, this has to be started alongside the container that runs the tests, which means either taking advantage of ResetService or rewriting the tests to run independently of each other by running as different Party's (the latter being preferred).
  • The tests generally require different sets of models. As of 0.13.5, the Sandbox supports uploading DARs after it has already been launched, but many of the tests expect to supply a DAR as a startup parameter (which is more difficult in a world where CircleCI insists on the DAML SDK being launched as a separate container "next to" the test container). The tests will need to be changed to supply DARs via upload instead.
  • The local workflow should still continue to use the DAML SDK and not require Docker, as that greatly slows down the iteration loop for running individual tests.

This also enables the python build to be moved off of a combo Java/Python Docker image and instead move to Circle's official Python image, as the tests no longer require the JDK to successfully run (provided the DAML SDK container is running).

This also helps facilitate a full migration off of Bintray/Maven Central:

  • Protobuf files that describe the gRPC Ledger API and the DAML-LF format are expected to be hosted on Bintray
  • Sandbox JARs are expected to be hosted in a specific way on Bintray/Maven Central.
  • dazl downloads individual files that are part of the SDK individually and runs them directly; this mechanism no longer works for newer SDK versions (0.13.41+) so upgrading the tests to newer versions of the DAML SDK are blocked until the mechanism for running Sandbox is changed accordingly.
@richardkapolnai-da
Copy link
Collaborator

Imho the DAZL developers would appreciate something like dazl.sandbox(), in order to avoid boilerplate code in their integration tests.

Just an idea: it could even ensure different parties for each test with some extra layer:

sandbox = sandbox_builder.new()
client = sandbox.aio_party()  # a brand new party like 'TestParty1'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants