Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add type descriptors for data constructors in Java, when necessary #301

Merged
merged 53 commits into from
Nov 27, 2023
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
461dd3d
Fix UTF8
robin-aws Oct 20, 2023
c31cee8
Temporarily try CI on 4.3
robin-aws Oct 20, 2023
45ee695
Merge branch 'main-1.x' into robin-aws/add-type-descriptors-for-java
robin-aws Oct 20, 2023
478c043
Get Resource test model working
robin-aws Oct 23, 2023
8f40bb7
Merge branch 'robin-aws/add-type-descriptors-for-java' of github.com:…
robin-aws Oct 23, 2023
fe5f975
Handle aggregate type descriptors in general
robin-aws Oct 23, 2023
ea48656
Fix KMS
robin-aws Oct 23, 2023
6ba962d
Fix a few more model tests
robin-aws Oct 23, 2023
e841a8a
Missing file
robin-aws Oct 23, 2023
213da2b
Missed one in DDB
robin-aws Oct 23, 2023
0a24bc8
Add dafnyVersion parameter, make type descriptors conditional
robin-aws Oct 28, 2023
6d8b292
Add —dafny-version to CLI
robin-aws Oct 29, 2023
180d29c
Test multiple Dafny versions in CI
robin-aws Oct 29, 2023
5c65aa7
Specify —dafny-version
robin-aws Oct 29, 2023
05327d8
Avoid conditional type descriptors in UTF8
robin-aws Oct 29, 2023
4d67a3d
Working around create_Success in test services
robin-aws Oct 29, 2023
ad4ac89
Fix KMS
robin-aws Oct 30, 2023
49c7de3
Fix test shim type descriptor
robin-aws Oct 30, 2023
ae8d542
public
robin-aws Oct 30, 2023
9e2cb47
Fix unit tests and local service
robin-aws Oct 30, 2023
33f74af
Fix DDB and Errors
robin-aws Oct 30, 2023
f0493fb
License headers, typos
robin-aws Oct 30, 2023
efed9b2
Shim.createFailureOfException -> Shim.createFailureOfError
robin-aws Oct 30, 2023
b6f4f09
Typo
robin-aws Oct 30, 2023
fb538e0
Typos, typos everywhere…
robin-aws Oct 30, 2023
c8f245e
Fix test
robin-aws Oct 30, 2023
8a1786b
Cleanup
robin-aws Oct 30, 2023
6e64b30
Refactor CI to handle nightly builds correctly
robin-aws Oct 31, 2023
538d872
Need full commit sha
robin-aws Oct 31, 2023
fb2d1e4
Use latest edition in sample
robin-aws Oct 31, 2023
eb785e0
Introduce proper DafnyVersion class
robin-aws Oct 31, 2023
82349d6
License headers
robin-aws Oct 31, 2023
60557cb
Move interop methods into Dafny (StandardLibrary and abstract server …
robin-aws Nov 2, 2023
7c693c9
Use the same helpers for test wrappers too
robin-aws Nov 2, 2023
f7c8069
Fix remaining TestModels
robin-aws Nov 2, 2023
e27cd3e
Fix unit tests
robin-aws Nov 2, 2023
ff32cc7
Document new StandardLibrary module
robin-aws Nov 2, 2023
93f39db
Miscellaneous fixes
robin-aws Nov 2, 2023
6ca9e79
Make sure CLi actually fails without —dafny-version, add it in CI eve…
robin-aws Nov 2, 2023
6c3ac1a
Upgrade setup-dafny-action to 1.7.0
robin-aws Nov 2, 2023
31595e5
Fix import statements for WrappersInterop
robin-aws Nov 2, 2023
9305206
Getting tired of only getting partial CI results
robin-aws Nov 2, 2023
f20738b
Using service client interface instead of concrete class in abstract …
robin-aws Nov 2, 2023
3d3e52f
Should be traitForServiceClient after all
robin-aws Nov 2, 2023
28adebb
Fix remaining models
robin-aws Nov 2, 2023
183b0fa
Fix Dependencies
robin-aws Nov 2, 2023
937bda6
I said, FIX DEPENDENCIES
robin-aws Nov 2, 2023
b0527ea
Skip Extendable model on 4.3
robin-aws Nov 21, 2023
3752aa6
Merge branch 'main-1.x' into robin-aws/add-type-descriptors-for-java
robin-aws Nov 22, 2023
6eed666
Remove dead code, more comments on the helper methods
robin-aws Nov 22, 2023
6f66074
Merge branch 'robin-aws/add-type-descriptors-for-java' of github.com:…
robin-aws Nov 22, 2023
0db5dec
Apply suggestions from code review
robin-aws Nov 27, 2023
855d329
PR feedback
robin-aws Nov 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/workflows/manual.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# This workflow invokes other workflows with the requested Dafny build.
# It is primarily meant for manual compatibility testing,
# such as trying out what the next pending nightly build will do ahead of time.
name: Manual CI

on:
workflow_dispatch:
inputs:
dafny:
description: 'The Dafny version to use'
required: true
type: string

jobs:
manual-ci-verification:
uses: ./.github/workflows/test_models_dafny_verification.yml
with:
dafny: ${{ inputs.dafny }}
manual-ci-java:
uses: ./.github/workflows/test_models_java_tests.yml
with:
dafny: ${{ inputs.dafny }}
manual-ci-net:
uses: ./.github/workflows/test_models_net_tests.yml
with:
dafny: ${{ inputs.dafny }}
29 changes: 29 additions & 0 deletions .github/workflows/nightly_dafny.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# This workflow invokes other workflows with the nightly Dafny build
name: Dafny Nightly

on:
schedule:
# Nightly build against Dafny's nightly prereleases,
# for early warning of verification issues or regressions.
# Timing chosen to be adequately after Dafny's own nightly build,
# but this might need to be tweaked:
# https://github.com/dafny-lang/dafny/blob/master/.github/workflows/deep-tests.yml#L16
- cron: "30 16 * * *"

jobs:
dafny-nightly-verification:
# Don't run the cron builds on forks
if: github.event_name != 'schedule' || github.repository_owner == 'smithy-lang'
uses: ./.github/workflows/test_models_dafny_verification.yml
with:
dafny: 'nightly-latest'
dafny-nightly-java:
if: github.event_name != 'schedule' || github.repository_owner == 'smithy-lang'
uses: ./.github/workflows/test_models_java_tests.yml
with:
dafny: 'nightly-latest'
dafny-nightly-net:
if: github.event_name != 'schedule' || github.repository_owner == 'smithy-lang'
uses: ./.github/workflows/test_models_net_tests.yml
with:
dafny: 'nightly-latest'
40 changes: 40 additions & 0 deletions .github/workflows/pull.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# This workflow runs for every pull request
name: PR CI

on:
pull_request:

jobs:
pr-ci-verification:
strategy:
fail-fast: false
matrix:
dafny-version: [
4.1.0,
4.3.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You talk about 4.2 in the PR but don't test it here?

Also, given the repetition of version you may want to pull that out as well.
Just saying

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair. Although getting to 4.3 across the board is more important than 4.2 in the broader context.

Copy link
Contributor Author

@robin-aws robin-aws Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was actually a little worried about creating so many individual GHA jobs by testing on more than 2. As part of a different PR I'd like to first get the parameterization of test models out of workflow files: https://github.com/smithy-lang/smithy-dafny/pull/257/files#diff-a933719600e0ce4e5ada8a6a7a64dfb38e005f6145c00147d9a02a16c69409d7R20-R24 After that I'd be happier to test on more dafnies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I am still learning how to best utilize GHA/Ws,
and it seems that GitHub is evolving them as well.

And our test models need to be refactored to an integration test project...

]
uses: ./.github/workflows/test_models_dafny_verification.yml
with:
dafny: ${{ matrix.dafny-version }}
pr-ci-java:
strategy:
fail-fast: false
matrix:
dafny-version: [
4.1.0,
4.3.0
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Is there no way to declare this once and reference it elsewhere in this file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely possible but not nearly as simple as you'd think :) I made the change as I already had to do something similar for Dafny (where we needed a lot more dynamic calculation)

uses: ./.github/workflows/test_models_java_tests.yml
with:
dafny: ${{ matrix.dafny-version }}
pr-ci-net:
strategy:
fail-fast: false
matrix:
dafny-version: [
4.1.0,
4.3.0
]
uses: ./.github/workflows/test_models_net_tests.yml
with:
dafny: ${{ matrix.dafny-version }}
42 changes: 42 additions & 0 deletions .github/workflows/push.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# This workflow runs for every push to main-1.x
name: Push CI

on:
push:
branches:
- main-1.x

jobs:
push-ci-verification:
strategy:
fail-fast: false
matrix:
dafny-version: [
4.1.0,
4.3.0
]
uses: ./.github/workflows/test_models_dafny_verification.yml
with:
dafny: ${{ matrix.dafny-version }}
push-ci-java:
strategy:
fail-fast: false
matrix:
dafny-version: [
4.1.0,
4.3.0
]
uses: ./.github/workflows/test_models_java_tests.yml
with:
dafny: ${{ matrix.dafny-version }}
push-ci-net:
strategy:
fail-fast: false
matrix:
dafny-version: [
4.1.0,
4.3.0
]
uses: ./.github/workflows/test_models_net_tests.yml
with:
dafny: ${{ matrix.dafny-version }}
41 changes: 16 additions & 25 deletions .github/workflows/test_models_dafny_verification.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,33 +2,20 @@
name: Library Dafny verification

on:
pull_request:
push:
branches:
- main-1.x
workflow_dispatch:
# Manual trigger for this workflow, either the normal version
# or the nightly build that uses the latest Dafny prerelease
# (accordingly to the "nightly" parameter).
workflow_call:
inputs:
nightly:
description: 'Run the nightly build'
required: false
type: boolean
schedule:
# Nightly build against Dafny's nightly prereleases,
# for early warning of verification issues or regressions.
# Timing chosen to be adequately after Dafny's own nightly build,
# but this might need to be tweaked:
# https://github.com/dafny-lang/dafny/blob/master/.github/workflows/deep-tests.yml#L16
- cron: "30 16 * * *"
dafny:
description: 'The Dafny version to run'
required: true
type: string

jobs:
verification:
# Don't run the nightly build on forks
if: github.event_name != 'schedule' || github.repository_owner == 'smithy-lang'
strategy:
fail-fast: false
matrix:
dafny:
- ${{ inputs.dafny }}
library: [
TestModels/dafny-dependencies/StandardLibrary, # This stores current Polymorph dependencies that all TestModels depend on
TestModels/Aggregate,
Expand Down Expand Up @@ -62,6 +49,11 @@ jobs:
TestModels/aws-sdks/sqs-via-cli,
]
os: [ ubuntu-latest ]
exclude:
# This model tickles a tricky Dafny verification regression to be fixed in 4.4
# https://github.com/dafny-lang/dafny/pull/4800
- dafny: 4.3.0
library: TestModels/Extendable
runs-on: ${{ matrix.os }}
env:
DOTNET_CLI_TELEMETRY_OPTOUT: 1
Expand All @@ -74,16 +66,15 @@ jobs:
- uses: actions/checkout@v2

- name: Setup Dafny
uses: dafny-lang/setup-dafny-action@v1.6.1
uses: dafny-lang/setup-dafny-action@v1.7.0
with:
# A && B || C is the closest thing to an if .. then ... else ... or ?: expression the GitHub Actions syntax supports.
dafny-version: ${{ (github.event_name == 'schedule' || inputs.nightly) && 'nightly-latest' || '4.1.0' }}
dafny-version: ${{ inputs.dafny }}

- name: Generate Polymorph Wrapper Dafny code
shell: bash
working-directory: ./${{ matrix.library }}
run: |
make polymorph_dafny
make polymorph_dafny DAFNY_VERSION_OPTION="--dafny-version $DAFNY_VERSION"

- name: Verify ${{ matrix.library }} Dafny code
shell: bash
Expand Down
39 changes: 15 additions & 24 deletions .github/workflows/test_models_java_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,33 +2,24 @@
name: Library Java tests

on:
pull_request:
push:
branches:
- main-1.x
workflow_dispatch:
# Manual trigger for this workflow, either the normal version
# or the nightly build that uses the latest Dafny prerelease
# (accordingly to the "nightly" parameter).
workflow_call:
inputs:
nightly:
description: 'Run the nightly build'
required: false
type: boolean
schedule:
# Nightly build against Dafny's nightly prereleases,
# for early warning of verification issues or regressions.
# Timing chosen to be adequately after Dafny's own nightly build,
# but this might need to be tweaked:
# https://github.com/dafny-lang/dafny/blob/master/.github/workflows/deep-tests.yml#L16
- cron: "30 16 * * *"
dafny:
description: 'The Dafny version to run'
required: true
type: string

jobs:
testJava:
# Don't run the nightly build on forks
if: github.event_name != 'schedule' || github.repository_owner == 'smithy-lang'
strategy:
fail-fast: false
matrix:
dafny-version: [
4.1.0,
4.3.0
]
library: [
TestModels/dafny-dependencies/StandardLibrary, # This stores current Polymorph dependencies that all TestModels depend on
# TestModels/Aggregate,
Expand Down Expand Up @@ -82,10 +73,9 @@ jobs:
- uses: actions/checkout@v3

- name: Setup Dafny
uses: dafny-lang/setup-dafny-action@v1.6.1
uses: dafny-lang/setup-dafny-action@v1.7.0
with:
# A && B || C is the closest thing to an if .. then ... else ... or ?: expression the GitHub Actions syntax supports.
dafny-version: ${{ (github.event_name == 'schedule' || inputs.nightly) && 'nightly-latest' || '4.1.0' }}
dafny-version: ${{ inputs.dafny }}

- name: Setup Java
uses: actions/setup-java@v3
Expand All @@ -108,9 +98,10 @@ jobs:
- name: Generate Polymorph Dafny and Java code
shell: bash
working-directory: ./${{ matrix.library }}
# Use $DAFNY_VERSION from setup-dafny-action to handle nightlies correctly
run: |
make polymorph_dafny
make polymorph_java
make polymorph_dafny DAFNY_VERSION_OPTION="--dafny-version $DAFNY_VERSION"
make polymorph_java DAFNY_VERSION_OPTION="--dafny-version $DAFNY_VERSION"

- name: Setup Java 8 for running tests
uses: actions/setup-java@v3
Expand Down
36 changes: 10 additions & 26 deletions .github/workflows/test_models_net_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,32 +2,17 @@
name: Library net tests

on:
pull_request:
push:
branches:
- main-1.x
workflow_dispatch:
# Manual trigger for this workflow, either the normal version
# or the nightly build that uses the latest Dafny prerelease
# (accordingly to the "nightly" parameter).
workflow_call:
inputs:
nightly:
description: 'Run the nightly build'
required: false
type: boolean
schedule:
# Nightly build against Dafny's nightly prereleases,
# for early warning of verification issues or regressions.
# Timing chosen to be adequately after Dafny's own nightly build,
# but this might need to be tweaked:
# https://github.com/dafny-lang/dafny/blob/master/.github/workflows/deep-tests.yml#L16
- cron: "30 16 * * *"
dafny:
description: 'The Dafny version to run'
required: true
type: string

jobs:
testDotNet:
# Don't run the nightly build on forks
if: github.event_name != 'schedule' || github.repository_owner == 'smithy-lang'
strategy:
fail-fast: false
matrix:
library: [
TestModels/dafny-dependencies/StandardLibrary, # This stores current Polymorph dependencies that all TestModels depend on
Expand Down Expand Up @@ -85,10 +70,9 @@ jobs:
- uses: actions/checkout@v3

- name: Setup Dafny
uses: dafny-lang/setup-dafny-action@v1.6.1
uses: dafny-lang/setup-dafny-action@v1.7.0
with:
# A && B || C is the closest thing to an if .. then ... else ... or ?: expression the GitHub Actions syntax supports.
dafny-version: ${{ (github.event_name == 'schedule' || inputs.nightly) && 'nightly-latest' || '4.1.0' }}
dafny-version: ${{ inputs.dafny }}

- name: Setup .NET Core SDK ${{ matrix.dotnet-version }}
uses: actions/setup-dotnet@v3
Expand Down Expand Up @@ -116,8 +100,8 @@ jobs:
shell: bash
working-directory: ./${{ matrix.library }}
run: |
make polymorph_dafny
make polymorph_net
make polymorph_dafny DAFNY_VERSION_OPTION="--dafny-version $DAFNY_VERSION"
make polymorph_net DAFNY_VERSION_OPTION="--dafny-version $DAFNY_VERSION"

- name: Compile ${{ matrix.library }} implementation
shell: bash
Expand Down
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,7 @@
/.history
/.smithy.lsp.log

*/**/.idea
*/**/.idea
/codegen/smithy-dafny-codegen/bin
/codegen/smithy-dafny-codegen-cli/bin
/smithy-dafny-conversion/bin
2 changes: 1 addition & 1 deletion TestModels/Aggregate/src/Index.dfy
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ module {:extern "simple.aggregate.internaldafny"} SimpleAggregate refines Abstra
}

method SimpleAggregate(config: SimpleAggregateConfig)
returns (res: Result<SimpleAggregateClient, Error>) {
returns (res: Result<ISimpleAggregateClient, Error>) {
var client := new SimpleAggregateClient(Operations.Config);
return Success(client);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ public static Result<ISimpleConstraintsClient, Error> WrappedSimpleConstraints(S
simple.constraints.model.SimpleConstraintsConfig wrappedConfig = ToNative.SimpleConstraintsConfig(config);
simple.constraints.SimpleConstraints impl = SimpleConstraints.builder().SimpleConstraintsConfig(wrappedConfig).build();
TestSimpleConstraints wrappedClient = TestSimpleConstraints.builder().impl(impl).build();
return Result.create_Success(wrappedClient);
return simple.constraints.internaldafny.__default.CreateSuccessOfClient(wrappedClient);
}
}
2 changes: 1 addition & 1 deletion TestModels/Constraints/src/Index.dfy
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ module {:extern "simple.constraints.internaldafny" } SimpleConstraints refines A
}

method SimpleConstraints(config: SimpleConstraintsConfig)
returns (res: Result<SimpleConstraintsClient, Error>)
returns (res: Result<ISimpleConstraintsClient, Error>)
{
var client := new SimpleConstraintsClient(Operations.Config);
return Success(client);
Expand Down
2 changes: 1 addition & 1 deletion TestModels/Constructor/src/Index.dfy
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ module {:extern "simple.constructor.internaldafny" } SimpleConstructor refines A
}

method SimpleConstructor(config: SimpleConstructorConfig)
returns (res: Result<SimpleConstructorClient, Error>)
returns (res: Result<ISimpleConstructorClient, Error>)
{
var configToAssign := Operations.Config(
blobValue := config.blobValue.UnwrapOr([0]),
Expand Down
Loading
Loading