Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[4/?] Add ability to run code examples in the playground: Get pony snippets tested with ponyc at CI time #549

Open
shaedrich opened this issue May 27, 2024 · 2 comments · May be fixed by #550

Comments

@shaedrich
Copy link
Collaborator

shaedrich commented May 27, 2024

Part of #340

  • Where do we put the expectations for stdOut, stdErr, and exitCode? Because if we don't do that, only snippets that print to stdOut could be tested. docstring as frontmatter perhaps?
Chapter Snippet stdOut stdErr error line exitCode
Getting started ▸ Hello world hello-world-main.pony Hello, world! n/a n/a 0
Types ▸ Classes classes-swap-values-sugar.pony a = 1, b = 2
a = 2, b = 1
n/a n/a 0
Types ▸ Classes classes-swap-values.pony a = 1, b = 2
a = 2, b = 1
n/a n/a 0
Types ▸ Classes classes-wombat.pony Your default wombat's name is "Fantastibat"
Your hungry wombat's name is "Nomsbat"
n/a n/a 0
Types ▸ Primitives primitives-doors.pony Is door open? false
2 + 3 = 5
n/a n/a 0
Types ▸ Actors actors-sequential.pony This is printed first
This is printed last
n/a n/a 0
Types ▸ Traits and interfaces traits-and-interfaces-multiple-traits.pony Their name is "Bob" and they are not bald n/a n/a 0
Types ▸ Traits and interfaces traits-and-interfaces-nested-traits.pony Their name is "Bob" and they are not bald n/a n/a 0
Types ▸ Traits and interfaces traits-and-interfaces-nominal-and-structural-subtyping.pony Their name is "Bob"
Their name is "Larry"
n/a n/a 0
Types ▸ Traits and interfaces traits-and-interfaces-nominal-subtyping-in-pony.pony Their name is "Larry" n/a n/a 0
Types ▸ Traits traits-and-interfaces-open-world-typing.pony size: 2, space: 8
size: 2, space: 8
size: 3, space: 8
size: 3, space: 4
n/a n/a 0
Types ▸ Traits and interfaces traits-and-interfaces-private-methods.pony n/a interfaces can't have private methods, only traits can 9:11 1?
Types ▸ Traits and interfaces traits-and-interfaces-trait.pony Their name is "Bob" n/a n/a 0
Types ▸ Type aliases type-aliases-enumerations-apply.pony red: #FF0000FF
green: #FF00FF
blue: #FFFF
n/a n/a 0
Types ▸ Type aliases type-aliases-complex-types-trait.pony Bob, aged 42, feels "Great!" n/a n/a 0
@ponylang-main ponylang-main added the discuss during sync Should be discussed during an upcoming sync label May 27, 2024
@shaedrich
Copy link
Collaborator Author

shaedrich commented May 31, 2024

That's the current output:

Errors:
grafik

Successful:
grafik

@jemc
Copy link
Member

jemc commented Jun 25, 2024

As discussed before and in other PRs, we're mainly concerned with the checking whether something compiles or not (and a regular expression match for the error message in the expected-not-to-compile case).

So that approach drastically reduces how much of this expectation information we need to track - we essentially don't need to track anything at all for examples which are expected to compile, and for those which are expected to have a compilation error, we only need a string / regular expression to match against the error output.

As for the question of where these expectations should be tracked, I think if they were in Pony docstring of the source files themselves, that would risk confusion for the reader of these snippets in the playground (they might likely ask themselves if these expectations were actually of a part of Pony the language). So that leaves basically two options:

  1. A central file somewhere with a list of expected-not-to-compile sample names, each mapped to a string with a pattern to check for in the compilation error.
  2. Decentralized files - each file having the same name as the sample it maps to, but with some extra suffix (such as .error.txt) and the content of the file is the expected error pattern to match.

We're not strongly opinionated about these two approaches, but I have a slight preference for the latter.

@jemc jemc removed the discuss during sync Should be discussed during an upcoming sync label Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants