Errors not thrown in Test environment (program silently continues)

[zygmuntszpak]

Describe the bug
When running ] test and there are any errors (e.g. a MethodError) in the program, they are not printed to the REPL and there is no stacktrace. Instead, the program silently exits the function and continues. In my case, I had a subtle MethodError inside of a onbutton(button) do block, and only part of the lines of code in the onbutton were executed, and when the problematic line was hit the program would just exist the do block and continue.

Error stacktrace
There is no error stack trace, which is precisely the problem.

To reproduce
Create a simple reactive model with a button and add the following to the handler function.

  onbutton(model.button_pressed) do
      println("Step 1")
      rubbish() # This method doesn't exist, or has some other error.
      println("Step 2")
   end

The program will never execute Step 2, but there will be no warning, error or stack trace. In my instance, the button was supposed to remove a record from a list, and I could not understand why the record was not being removed. After a lot of time debugging and poking around, I noticed that the function was not executing all lines of code but exiting early. I was completely caught off-guard by the fact that no error message or stack trace was being generated and was starting to wonder if there is some race condition in Genie that made the event handler exit sooner than it should.

Expected behavior
When running the test environment one would expect that errors and their stack traces are displayed so that one can understand where and why certain @test asserts are failing.

Additional context

Julia Version 1.8.2
Commit 36034abf26 (2022-09-29 15:21 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 8 × Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 4 on 8 virtual cores
Environment:
  JULIA_REVISE = auto
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 4

  [c43c736e] Genie v5.11.0
  [c6228e60] GenieAutoReload v2.0.1
  [4e5d9629] GenieDevTools v2.0.0
  [6d011eab] Inflector v1.0.1
  [2913bbd2] StatsBase v0.33.21
  [4acbeb90] Stipple v0.25.11
  [ec984513] StipplePlotly v0.13.2
  [a3c5d34a] StippleUI v0.21.0

I believe the problem can be traced back to this line in runtests.jl

Logging.global_logger(NullLogger())
#NullLogger()
#Logger which disables all messages and produces no output - the logger equivalent of /dev/null.

[AbhimanyuAryan]

hey this looks like a Stipple issue not Genie. Can you please close this here and move it to https://github.com/GenieFramework/Stipple.jl

[zygmuntszpak]

Hi,
I don't see how this is a Stipple issue. The problem can be traced back to this file in this repository: https://github.com/GenieFramework/Genie.jl/blob/master/test/runtests.jl

[essenciary]

@zygmuntszpak I need to check but:

a) @AbhimanyuAryan might be right - there is a known issue that errors inside event handlers in Stipple are not shown (which looks like your example)

b) The line in Genie is most likely harmless: Genie's tests use additional packages to collect and reformat the tests results. This means that they are stopped from being logged directly, but the logs are collected and outputted at the end. Most likely, due to the Stipple issue, Genie had nothing to log.