Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(debugger): Add support for debugging tests (#5208) #7

Open
wants to merge 31 commits into
base: feat/5208-debug-tests-repl-dap
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
37283ac
wip
anaPerezGhiglia Jul 3, 2024
21f0739
Compile for debug & run debugger on `nargo test --debug`
anaPerezGhiglia Jul 3, 2024
1589176
wip: extract common behavior
anaPerezGhiglia Jul 12, 2024
1580110
wip: evaluate test status from debugger result
anaPerezGhiglia Jul 12, 2024
e3f10d6
wip: register last seen error in debugger context
anaPerezGhiglia Jul 12, 2024
d9aaa1d
[skip-ci] failing compilation
anaPerezGhiglia Jul 16, 2024
6d16905
Expose errors when the debugger execution finishes
anaPerezGhiglia Jul 16, 2024
aebba26
remove unnecessary mut
anaPerezGhiglia Jul 16, 2024
8b96a17
Ignore --debug mode for tests with arguments
anaPerezGhiglia Jul 23, 2024
7b3961a
return brillig solver ownership
anaPerezGhiglia Jul 24, 2024
536a89f
Map opcode resulution erros in debugger as in executor
anaPerezGhiglia Jul 24, 2024
54fd555
format code
anaPerezGhiglia Jul 24, 2024
6b1cc78
Remove debug println!
anaPerezGhiglia Jul 24, 2024
16a76a1
Add 'Debug test' codelens request
anaPerezGhiglia Jul 26, 2024
873d516
add test_name arg to dap interface
anaPerezGhiglia Jul 26, 2024
92e551d
Compile test function for debugging if test_name is present
anaPerezGhiglia Jul 29, 2024
8da3eef
extract common behavior for building FileManager and ParsedFiles
anaPerezGhiglia Jul 29, 2024
5a1625e
Remove unused imports
anaPerezGhiglia Jul 29, 2024
9eacfe6
Notify if test passed or failed
anaPerezGhiglia Aug 7, 2024
c36f131
fix compile error after rebase
anaPerezGhiglia Aug 7, 2024
c8f8b40
Fix clippy warnings
anaPerezGhiglia Aug 7, 2024
ac3aa22
ignore clippy warnings since functions will be modified
anaPerezGhiglia Aug 7, 2024
7914726
run cargo fmt
anaPerezGhiglia Aug 7, 2024
19bf1e9
Move 'debug test' REPL responsibility from test_cmd to debug_cmd
anaPerezGhiglia Aug 8, 2024
2dc687c
simplify debu_test function
anaPerezGhiglia Aug 8, 2024
b70cd79
remove execution_helpers auxiliary file
anaPerezGhiglia Aug 8, 2024
8d76cac
Expand todo comment
anaPerezGhiglia Aug 8, 2024
2dbda88
Improve error message on unexpected compile error
anaPerezGhiglia Aug 9, 2024
4364c48
Re-shape file_manager_and_file_from to build_workspace_file_manager
anaPerezGhiglia Aug 9, 2024
5ef8e84
Document new "debug test" feature
anaPerezGhiglia Aug 9, 2024
38b58e7
Remove stale TODO
anaPerezGhiglia Aug 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

43 changes: 42 additions & 1 deletion docs/docs/how_to/debugger/debugging_with_the_repl.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ sidebar_position: 1

#### Pre-requisites

In order to use the REPL debugger, first you need to install recent enough versions of Nargo and vscode-noir.
In order to use the REPL debugger, first you need to install recent enough versions of Nargo and vscode-noir.

## Debugging a simple circuit

Expand Down Expand Up @@ -162,3 +162,44 @@ Finished execution
Upon quitting the debugger after a solved circuit, the resulting circuit witness gets saved, equivalent to what would happen if we had run the same circuit with `nargo execute`.

We just went through the basics of debugging using Noir REPL debugger. For a comprehensive reference, check out [the reference page](../../reference/debugger/debugger_repl.md).

## Debugging a test function

Let's debug a simple circuit:

```rust
#[noir]
fn test_simple_equal() {
let x = 2;
let y = 1 + 1;
assert(x == y, "should be equal");
}
```

To start the REPL debugger for a test function, using a terminal, go to a Noir circuit's home directory. Then invoke the `debug` command setting the `--test-name` argument.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To start the REPL debugger for a test function, using a terminal, go to a Noir circuit's home directory. Then invoke the `debug` command setting the `--test-name` argument.
To debug a test function using the REPL debugger, navigate to a Noir project directory inside a terminal, and run the `nargo debug` command passing the `--test-name your_test_name_here` argument.


```bash
nargo debug --test-name test_simple_equal
```

After that, the debugger has started and works the same as debugging a main function, you can use any of the above explained commands to control the execution of the test function.

### Test result

The debugger does not end the session automatically. Once you finish debugging the execution of the test function you will notice that the debugger remain in the `Execution finished` state. If you are done debugging the test function you should exit the debugger by using the `quit` command. Once you finish the debugging session you should see the test result.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The debugger does not end the session automatically. Once you finish debugging the execution of the test function you will notice that the debugger remain in the `Execution finished` state. If you are done debugging the test function you should exit the debugger by using the `quit` command. Once you finish the debugging session you should see the test result.
The debugger does not end the session automatically. Once you finish debugging the execution of the test function you will notice that the debugger remains in the `Execution finished` state. When you are done debugging the test function you can exit the debugger by using the `quit` command. Once you finish the debugging session you should see the test result.


```text
$ nargo debug --test-name test_simple_equal

[simple_noir_project] Starting debugger
At opcode 0:0 :: BRILLIG CALL func 0: inputs: [], outputs: []

> continue
(Continuing execution...)
Finished execution

> quit
[simple_noir_project] Circuit witness successfully solved
[simple_noir_project] Testing test_simple_equal... ok
[simple_noir_project] 1 test passed
```
12 changes: 9 additions & 3 deletions docs/docs/how_to/debugger/debugging_with_vs_code.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,15 @@ This guide will show you how to use VS Code with the vscode-noir extension to de

## Running the debugger

The easiest way to start debugging is to open the file you want to debug, and press `F5`. This will cause the debugger to launch, using your `Prover.toml` file as input.
The easiest way to start debugging is to open the file you want to debug, and click on `Debug` codelens over main functions or `Debug test` over `#[test]` functions

You should see something like this:
If you don't see the codelens options `Compile|Info|..|Debug` over the `main` function or `Run test| Debug test` over a test function then you probably have the codelens feature disabled. For enabling it head to the extension configuration and turn on the `Enable Code Lens` setting.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you don't see the codelens options `Compile|Info|..|Debug` over the `main` function or `Run test| Debug test` over a test function then you probably have the codelens feature disabled. For enabling it head to the extension configuration and turn on the `Enable Code Lens` setting.
If you don't see the codelens options `Compile|Info|..|Debug` over the `main` function or `Run test| Debug test` over a test function then you probably have the codelens feature disabled. To enable it open the extension configuration page and check the `Enable Code Lens` setting.


![Debugger codelens](@site/static/img/debugger/debugger-codelens.png)

Another way of starting the debugger is to press `F5` on the file you want to debug. This will cause the debugger to launch, using your `Prover.toml` file as input.

Once the debugger has started you should see something like this:

![Debugger launched](@site/static/img/debugger/1-started.png)

Expand Down Expand Up @@ -65,4 +71,4 @@ We just need to click the to the right of the line number 18. Once the breakpoin

Now we are debugging the `keccak256` function, notice the _Call Stack pane_ at the lower right. This lets us inspect the current call stack of our process.

That covers most of the current debugger functionalities. Check out [the reference](../../reference/debugger/debugger_vscode.md) for more details on how to configure the debugger.
That covers most of the current debugger functionalities. Check out [the reference](../../reference/debugger/debugger_vscode.md) for more details on how to configure the debugger.
22 changes: 14 additions & 8 deletions docs/docs/reference/debugger/debugger_repl.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,23 @@ Runs the Noir REPL debugger. If a `WITNESS_NAME` is provided the debugger writes

### Options

| Option | Description |
| --------------------- | ------------------------------------------------------------ |
| Option | Description |
| --------------------------------- | ----------------------------------------------------------------------------------- |
| `-p, --prover-name <PROVER_NAME>` | The name of the toml file which contains the inputs for the prover [default: Prover]|
| `--package <PACKAGE>` | The name of the package to debug |
| `--print-acir` | Display the ACIR for compiled circuit |
| `--deny-warnings` | Treat all warnings as errors |
| `--silence-warnings` | Suppress warnings |
| `-h, --help` | Print help |
| `--package <PACKAGE>` | The name of the package to debug |
| `--print-acir` | Display the ACIR for compiled circuit |
| `--deny-warnings` | Treat all warnings as errors |
| `--silence-warnings` | Suppress warnings |
| `--test-name` <TEST_NAME> | The name of the test function to debug - which name contains this string |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `--test-name` <TEST_NAME> | The name of the test function to debug - which name contains this string |
| `--test-name` <TEST_NAME> | The name (or substring) of the test function to debug |

| `-h, --help` | Print help |

None of these options are required.

:::note
If the `--test-name` option is provided the debugger will debug the matching function instead of the package `main` function.
This argument must only match one function. If the given name matches with more than one test function the debugger will not start.
:::

:::note
Since the debugger starts by compiling the target package, all Noir compiler options are also available. Check out the [compiler reference](../nargo_commands.md#nargo-compile) to learn more about the compiler options.
:::
Expand Down Expand Up @@ -357,4 +363,4 @@ Update a memory cell with the given value. For example:

:::note
This command is only functional while the debugger is executing unconstrained code.
:::
:::
26 changes: 13 additions & 13 deletions docs/docs/reference/debugger/debugger_vscode.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ sidebar_position: 0

The Noir debugger enabled by the vscode-noir extension ships with default settings such that the most common scenario should run without any additional configuration steps.

These defaults can nevertheless be overridden by defining a launch configuration file. This page provides a reference for the properties you can override via a launch configuration file, as well as documenting the Nargo `dap` command, which is a dependency of the VS Code Noir debugger.

These defaults can nevertheless be overridden by defining a launch configuration file. This page provides a reference for the properties you can override via a launch configuration file, as well as documenting the Nargo `dap` command, which is a dependency of the VS Code Noir debugger.

## Creating and editing launch configuration files

Expand Down Expand Up @@ -47,7 +46,7 @@ Name of the prover input to use. Defaults to `Prover`, which looks for a file na
_Boolean, optional._

If true, generate ACIR opcodes instead of unconstrained opcodes which will be closer to release binaries but less convenient for debugging. Defaults to `false`.

#### skipInstrumentation

_Boolean, optional._
Expand All @@ -60,9 +59,9 @@ Skipping instrumentation causes the debugger to be unable to inspect local varia

## `nargo dap [OPTIONS]`

When run without any option flags, it starts the Nargo Debug Adapter Protocol server, which acts as the debugging backend for the VS Code Noir Debugger.
When run without any option flags, it starts the Nargo Debug Adapter Protocol server, which acts as the debugging backend for the VS Code Noir Debugger.

All option flags are related to preflight checks. The Debug Adapter Protocol specifies how errors are to be informed from a running DAP server, but it doesn't specify mechanisms to communicate server initialization errors between the DAP server and its client IDE.
All option flags are related to preflight checks. The Debug Adapter Protocol specifies how errors are to be informed from a running DAP server, but it doesn't specify mechanisms to communicate server initialization errors between the DAP server and its client IDE.

Thus `nargo dap` ships with a _preflight check_ mode. If flag `--preflight-check` and the rest of the `--preflight-*` flags are provided, Nargo will run the same initialization routine except it will not start the DAP server.

Expand All @@ -72,11 +71,12 @@ If the preflight check succeeds, `vscode-noir` proceeds to start the DAP server

### Options

| Option | Description |
| --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| `--preflight-check` | If present, dap runs in preflight check mode. |
| `--preflight-project-folder <PREFLIGHT_PROJECT_FOLDER>` | Absolute path to the project to debug for preflight check. |
| `--preflight-prover-name <PREFLIGHT_PROVER_NAME>` | Name of prover file to use for preflight check |
| `--preflight-generate-acir` | Optional. If present, compile in ACIR mode while running preflight check. |
| `--preflight-skip-instrumentation` | Optional. If present, compile without introducing debug instrumentation while running preflight check. |
| `-h, --help` | Print help. |
| Option | Description |
| --------------------------------------------- | --------------------------------------------------------------------------- |
| `--preflight-check` | If present, dap runs in preflight check mode. |
| `--preflight-project-folder <PREFLIGHT_PROJECT_FOLDER>` | Absolute path to the project to debug for preflight check. |
| `--preflight-prover-name <PREFLIGHT_PROVER_NAME>` | Name of prover file to use for preflight check |
| `--preflight-generate-acir` | Optional. If present, compile in ACIR mode while running preflight check. |
| `--preflight-skip-instrumentation` | Optional. If present, compile without introducing debug instrumentation while running preflight check. |
| `--preflight-test-name <PREFLIGHT_TEST_NAME>` | Optional. If present, debug matching test function instead of main function |
| `-h, --help` | Print help. |
Binary file added docs/static/img/debugger/debugger-codelens.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions tooling/debugger/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ noirc_printable_type.workspace = true
noirc_errors.workspace = true
noirc_driver.workspace = true
noirc_artifacts.workspace = true
noirc_abi.workspace = true
thiserror.workspace = true
codespan-reporting.workspace = true
dap.workspace = true
Expand Down
35 changes: 29 additions & 6 deletions tooling/debugger/src/context.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use crate::foreign_calls::DebugForeignCallExecutor;
use acvm::acir::brillig::BitSize;
use acvm::acir::circuit::brillig::BrilligBytecode;
use acvm::acir::circuit::{Circuit, Opcode, OpcodeLocation};
use acvm::acir::circuit::{Circuit, Opcode, OpcodeLocation, ResolvedOpcodeLocation};
use acvm::acir::native_types::{Witness, WitnessMap, WitnessStack};
use acvm::brillig_vm::MemoryValue;
use acvm::pwg::{
Expand All @@ -12,7 +12,7 @@

use codespan_reporting::files::{Files, SimpleFile};
use fm::FileId;
use nargo::errors::{ExecutionError, Location};
use nargo::errors::{map_execution_error, ExecutionError, Location};
use nargo::NargoError;
use noirc_artifacts::debug::{DebugArtifact, StackFrame};
use noirc_driver::DebugFile;
Expand Down Expand Up @@ -183,9 +183,14 @@

#[derive(Debug)]
pub(super) enum DebugCommandResult {
// TODO: validate comments
// The debugging session is over successfully
Done,
// The session is active and we should continue with the execution
Ok,
// Execution should be paused since we reached a Breakpoint
BreakpointReached(DebugLocation),
// Session is over with an error
Comment on lines +186 to +193
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In all of these, it's probably desirable to use /// to use the comments as documentation.

Error(NargoError<FieldElement>),
}

Expand Down Expand Up @@ -316,6 +321,20 @@
frames
}

fn get_resolved_call_stack(&self) -> Vec<ResolvedOpcodeLocation> {
self.get_call_stack()
.iter()
.map(|debug_loc| {
// usize should be at least u32 for supported platforms
let acir_function_index = usize::try_from(debug_loc.circuit_id).unwrap();
acvm::acir::circuit::ResolvedOpcodeLocation {
acir_function_index,
opcode_location: debug_loc.opcode_location,
}
})
Comment on lines +327 to +334
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be better to implement the From trait (or Into) for ResolvedOpcodeLocation to convert from DebugLocation since both structs are very much equivalent. Then this would become something like:

self.get_call_stack().iter().map(|debug_loc| debug_loc.into()).collect()

.collect()
}

pub(super) fn is_source_location_in_debug_module(&self, location: &Location) -> bool {
self.debug_artifact
.file_map
Expand Down Expand Up @@ -472,9 +491,13 @@
self.brillig_solver = Some(solver);
self.handle_foreign_call(foreign_call)
}
Err(err) => DebugCommandResult::Error(NargoError::ExecutionError(
ExecutionError::SolvingError(err, None),
)),
Err(err) => {
self.brillig_solver = Some(solver);
DebugCommandResult::Error(NargoError::ExecutionError(map_execution_error(
err,
&self.get_resolved_call_stack(),
)))
}
}
}

Expand Down Expand Up @@ -574,7 +597,7 @@
}
}
ACVMStatus::Failure(error) => DebugCommandResult::Error(NargoError::ExecutionError(
ExecutionError::SolvingError(error, None),
map_execution_error(error, &self.get_resolved_call_stack()),
)),
ACVMStatus::RequiresForeignCall(foreign_call) => self.handle_foreign_call(foreign_call),
ACVMStatus::RequiresAcirCall(call_info) => self.handle_acir_call(call_info),
Expand Down Expand Up @@ -904,7 +927,7 @@
outputs: vec![],
predicate: None,
}];
let brillig_funcs = &vec![brillig_bytecode];

Check warning on line 930 in tooling/debugger/src/context.rs

View workflow job for this annotation

GitHub Actions / Code

Unknown word (funcs)
let current_witness_index = 2;
let circuit = Circuit { current_witness_index, opcodes, ..Circuit::default() };
let circuits = &vec![circuit];
Expand All @@ -923,7 +946,7 @@
debug_artifact,
initial_witness,
foreign_call_executor,
brillig_funcs,

Check warning on line 949 in tooling/debugger/src/context.rs

View workflow job for this annotation

GitHub Actions / Code

Unknown word (funcs)
);

assert_eq!(
Expand Down Expand Up @@ -1042,14 +1065,14 @@

let foreign_call_executor =
Box::new(DefaultDebugForeignCallExecutor::from_artifact(true, debug_artifact));
let brillig_funcs = &vec![brillig_bytecode];

Check warning on line 1068 in tooling/debugger/src/context.rs

View workflow job for this annotation

GitHub Actions / Code

Unknown word (funcs)
let mut context = DebugContext::new(
&StubbedBlackBoxSolver,
circuits,
debug_artifact,
initial_witness,
foreign_call_executor,
brillig_funcs,

Check warning on line 1075 in tooling/debugger/src/context.rs

View workflow job for this annotation

GitHub Actions / Code

Unknown word (funcs)
);

// set breakpoint
Expand Down Expand Up @@ -1116,7 +1139,7 @@
};
let circuits = vec![circuit_one, circuit_two];
let debug_artifact = DebugArtifact { debug_symbols: vec![], file_map: BTreeMap::new() };
let brillig_funcs = &vec![brillig_one, brillig_two];

Check warning on line 1142 in tooling/debugger/src/context.rs

View workflow job for this annotation

GitHub Actions / Code

Unknown word (funcs)

let context = DebugContext::new(
&StubbedBlackBoxSolver,
Expand Down
Loading
Loading