Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception processing function chaining arrow operator => in Metapath evaluation #289

Closed
aj-stein-nist opened this issue Jan 12, 2024 · 0 comments · Fixed by #290
Closed
Assignees
Labels
bug Something isn't working java Pull requests that update Java code
Milestone

Comments

@aj-stein-nist
Copy link
Collaborator

aj-stein-nist commented Jan 12, 2024

Describe the bug

While writing more advanced tests for #267, @david-waltermire and I discovered that evaluating Metapath expressions such as string(/document-root/document-element/@id) but the equivalent /document-root/document-element/@id => string() throws a null pointer exception, while the latter is modern syntax for the former. Code in the branch for further explanation and repro in new tests directly related to this issue to come later.

https://github.com/aj-stein-nist/metaschema-java/blob/1bec6492c116b7ae3da4c03d16e09cd879d45985/metaschema-cli/src/test/java/gov/nist/secauto/metaschema/cli/CLITest.java#L100-L112

Initial analysis indicates that FunctioncallContext and ArrowfunctionspecifierContext do not have the same attributes, thus looking the name of the child function fails on this line in the relevant line below. A bug fix is needed.

https://github.com/usnistgov/metaschema-java/blob/develop/core/src/main/java/gov/nist/secauto/metaschema/core/metapath/cst/BuildCSTVisitor.java#L847

Who is the bug affecting?

Developers using the metaschema-java library for processing Metaschema modules and content.

What is affected by this bug?

Compiling and evaluating any Metapath where there will be a arrow operator to chain functions.

When does this occur?

Consistently.

How do we replicate the issue?

{What are the steps to reproduce the behavior?

  1. Download the forementioned repro case branch.
  2. Run mvn test or use Eclipse or another IDE to cause an exception.
  3. Review the exception or stack frame received, which should be similar or identical to below when stepping through relevant code where NPE occurs.
Expand for full stack frame
BuildCSTVisitor.lambda$10(Metapath10$ArrowexprContext, Integer, IExpression) line: 847	
663824117.apply(Object, Object, Object) line: not available	
BuildCSTVisitor(AbstractCSTVisitorBase).handleGroupedNAiry(CONTEXT, int, int, ITriFunction<CONTEXT,Integer,IExpression,IExpression>) line: 278	
BuildCSTVisitor.handleArrowexpr(Metapath10$ArrowexprContext) line: 841	
BuildCSTVisitor.handleArrowexpr(Metapath10$ArrowexprContext) line: 1	
BuildCSTVisitor(AbstractAstVisitor<R>).lambda$24(Metapath10$ArrowexprContext, Metapath10$ArrowexprContext) line: 891	
1605299030.apply(Object) line: not available	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 117	
BuildCSTVisitor(AbstractAstVisitor<R>).visitArrowexpr(ArrowexprContext) line: 891	
Metapath10$ArrowexprContext.accept(ParseTreeVisitor<? extends T>) line: 1477	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitIntersectexceptexpr(IntersectexceptexprContext) line: 604	
Metapath10$IntersectexceptexprContext.accept(ParseTreeVisitor<? extends T>) line: 1399	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitUnionexpr(UnionexprContext) line: 589	
Metapath10$UnionexprContext.accept(ParseTreeVisitor<? extends T>) line: 1326	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitMultiplicativeexpr(MultiplicativeexprContext) line: 639	
Metapath10$MultiplicativeexprContext.accept(ParseTreeVisitor<? extends T>) line: 1253	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitAdditiveexpr(AdditiveexprContext) line: 624	
Metapath10$AdditiveexprContext.accept(ParseTreeVisitor<? extends T>) line: 1172	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitRangeexpr(RangeexprContext) line: 569	
Metapath10$RangeexprContext.accept(ParseTreeVisitor<? extends T>) line: 1111	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitStringconcatexpr(StringconcatexprContext) line: 680	
Metapath10$StringconcatexprContext.accept(ParseTreeVisitor<? extends T>) line: 1053	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitComparisonexpr(ComparisonexprContext) line: 700	
Metapath10$ComparisonexprContext.accept(ParseTreeVisitor<? extends T>) line: 970	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitAndexpr(AndexprContext) line: 747	
Metapath10$AndexprContext.accept(ParseTreeVisitor<? extends T>) line: 907	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitOrexpr(OrexprContext) line: 732	
Metapath10$OrexprContext.accept(ParseTreeVisitor<? extends T>) line: 846	
BuildCSTVisitor(AbstractAstVisitor<R>).delegateToChild(T) line: 136	
BuildCSTVisitor(AbstractAstVisitor<R>).visitExprsingle(ExprsingleContext) line: 170	
Metapath10$ExprsingleContext.accept(ParseTreeVisitor<? extends T>) line: 288	
BuildCSTVisitor(AbstractAstVisitor<R>).handle(T, Function<T,R>) line: 115	
BuildCSTVisitor(AbstractAstVisitor<R>).visitExpr(ExprContext) line: 164	
Metapath10$ExprContext.accept(ParseTreeVisitor<? extends T>) line: 222	
BuildCSTVisitor(AbstractParseTreeVisitor<T>).visit(ParseTree) line: 18	
BuildCSTVisitor(AbstractCSTVisitorBase).visit(ParseTree) line: 130	
MetapathExpression.compile(String) line: 140	
QueryCommand.executeCommand(CLIProcessor$CallingContext, CommandLine) line: 178	
807328355.execute(CLIProcessor$CallingContext, CommandLine) line: not available	
ICommandExecutor$1.execute() line: 48	
CLIProcessor$CallingContext.invokeCommand(CommandLine) line: 406	
CLIProcessor$CallingContext.processCommand() line: 377	
CLIProcessor.parseCommand(String...) line: 193	
CLIProcessor.process(String...) line: 176	
CLI.runCli(String...) line: 69	
CLITest.testQueryCommand() line: 106	
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62	
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
Method.invoke(Object, Object...) line: 566	
ReflectionUtils.invokeMethod(Method, Object, Object...) line: 727	
MethodInvocation<T>.proceed() line: 60	
InvocationInterceptorChain$ValidatingInvocation<T>.proceed() line: 131	
TimeoutExtension.intercept(Invocation<T>, ReflectiveInvocationContext<Method>, ExtensionContext, TimeoutDuration, TimeoutProvider) line: 156	
TimeoutExtension.interceptTestableMethod(Invocation<T>, ReflectiveInvocationContext<Method>, ExtensionContext, TimeoutProvider) line: 147	
TimeoutExtension.interceptTestMethod(Invocation<Void>, ReflectiveInvocationContext<Method>, ExtensionContext) line: 86	
1971764991.apply(InvocationInterceptor, InvocationInterceptor$Invocation, ReflectiveInvocationContext, ExtensionContext) line: not available	
InterceptingExecutableInvoker$ReflectiveInterceptorCall<E,T>.lambda$ofVoidMethod$0(InterceptingExecutableInvoker$ReflectiveInterceptorCall$VoidMethodInterceptorCall, InvocationInterceptor, InvocationInterceptor$Invocation, ReflectiveInvocationContext, ExtensionContext) line: 103	
359922172.apply(InvocationInterceptor, InvocationInterceptor$Invocation, ReflectiveInvocationContext, ExtensionContext) line: not available	
InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker$ReflectiveInterceptorCall, ReflectiveInvocationContext, ExtensionContext, InvocationInterceptor, InvocationInterceptor$Invocation) line: 93	
785570251.apply(InvocationInterceptor, InvocationInterceptor$Invocation) line: not available	
InvocationInterceptorChain$InterceptedInvocation<T>.proceed() line: 106	
InvocationInterceptorChain.proceed(Invocation<T>) line: 64	
InvocationInterceptorChain.chainAndInvoke(Invocation<T>, InterceptorCall<T>, List<InvocationInterceptor>) line: 45	
InvocationInterceptorChain.invoke(Invocation<T>, ExtensionRegistry, InterceptorCall<T>) line: 37	
InterceptingExecutableInvoker.invoke(Invocation<T>, ReflectiveInvocationContext<E>, ExtensionContext, ExtensionRegistry, ReflectiveInterceptorCall<E,T>) line: 92	
InterceptingExecutableInvoker.invoke(Method, Object, ExtensionContext, ExtensionRegistry, ReflectiveInterceptorCall<Method,T>) line: 86	
TestMethodTestDescriptor.lambda$invokeTestMethod$7(ExtensionContext, JupiterEngineExecutionContext) line: 217	
501609049.execute() line: not available	
OpenTest4JAndJUnit4AwareThrowableCollector(ThrowableCollector).execute(ThrowableCollector$Executable) line: 73	
TestMethodTestDescriptor.invokeTestMethod(JupiterEngineExecutionContext, Node$DynamicTestExecutor) line: 213	
TestMethodTestDescriptor.execute(JupiterEngineExecutionContext, Node$DynamicTestExecutor) line: 138	
TestMethodTestDescriptor.execute(EngineExecutionContext, Node$DynamicTestExecutor) line: 68	
NodeTestTask<C>.lambda$executeRecursively$6() line: 151	
1234250905.execute() line: not available	
OpenTest4JAndJUnit4AwareThrowableCollector(ThrowableCollector).execute(ThrowableCollector$Executable) line: 73	
NodeTestTask<C>.lambda$executeRecursively$8(EngineExecutionContext) line: 141	
585324508.invoke(EngineExecutionContext) line: not available	
TestMethodTestDescriptor(Node<C>).around(C, Invocation<C>) line: 137	
NodeTestTask<C>.lambda$executeRecursively$9() line: 139	
1720339.execute() line: not available	
OpenTest4JAndJUnit4AwareThrowableCollector(ThrowableCollector).execute(ThrowableCollector$Executable) line: 73	
NodeTestTask<C>.executeRecursively() line: 138	
NodeTestTask<C>.execute() line: 95	
1787079037.accept(Object) line: not available	
ArrayList<E>.forEach(Consumer<? super E>) line: 1541	
SameThreadHierarchicalTestExecutorService.invokeAll(List<TestTask>) line: 41	
NodeTestTask<C>.lambda$executeRecursively$6() line: 155	
1234250905.execute() line: not available	
OpenTest4JAndJUnit4AwareThrowableCollector(ThrowableCollector).execute(ThrowableCollector$Executable) line: 73	
NodeTestTask<C>.lambda$executeRecursively$8(EngineExecutionContext) line: 141	
585324508.invoke(EngineExecutionContext) line: not available	
ClassTestDescriptor(Node<C>).around(C, Invocation<C>) line: 137	
NodeTestTask<C>.lambda$executeRecursively$9() line: 139	
1720339.execute() line: not available	
OpenTest4JAndJUnit4AwareThrowableCollector(ThrowableCollector).execute(ThrowableCollector$Executable) line: 73	
NodeTestTask<C>.executeRecursively() line: 138	
NodeTestTask<C>.execute() line: 95	
1787079037.accept(Object) line: not available	
ArrayList<E>.forEach(Consumer<? super E>) line: 1541	
SameThreadHierarchicalTestExecutorService.invokeAll(List<TestTask>) line: 41	
NodeTestTask<C>.lambda$executeRecursively$6() line: 155	
1234250905.execute() line: not available	
OpenTest4JAndJUnit4AwareThrowableCollector(ThrowableCollector).execute(ThrowableCollector$Executable) line: 73	
NodeTestTask<C>.lambda$executeRecursively$8(EngineExecutionContext) line: 141	
585324508.invoke(EngineExecutionContext) line: not available	
JupiterEngineDescriptor(Node<C>).around(C, Invocation<C>) line: 137	
NodeTestTask<C>.lambda$executeRecursively$9() line: 139	
1720339.execute() line: not available	
OpenTest4JAndJUnit4AwareThrowableCollector(ThrowableCollector).execute(ThrowableCollector$Executable) line: 73	
NodeTestTask<C>.executeRecursively() line: 138	
NodeTestTask<C>.execute() line: 95	
SameThreadHierarchicalTestExecutorService.submit(TestTask) line: 35	
HierarchicalTestExecutor<C>.execute() line: 57	
JupiterTestEngine(HierarchicalTestEngine<C>).execute(ExecutionRequest) line: 54	
EngineExecutionOrchestrator.execute(TestDescriptor, EngineExecutionListener, ConfigurationParameters, TestEngine) line: 147	
EngineExecutionOrchestrator.execute(LauncherDiscoveryResult, EngineExecutionListener) line: 127	
EngineExecutionOrchestrator.execute(InternalTestPlan, EngineExecutionListener, TestExecutionListener) line: 90	
EngineExecutionOrchestrator.lambda$execute$0(InternalTestPlan, TestExecutionListener) line: 55	
1753127384.accept(Object) line: not available	
EngineExecutionOrchestrator.withInterceptedStreams(ConfigurationParameters, ListenerRegistry<TestExecutionListener>, Consumer<TestExecutionListener>) line: 102	
EngineExecutionOrchestrator.execute(InternalTestPlan, TestExecutionListener...) line: 54	
DefaultLauncher.execute(InternalTestPlan, TestExecutionListener[]) line: 114	
DefaultLauncher.execute(TestPlan, TestExecutionListener...) line: 95	
DefaultLauncherSession$DelegatingLauncher.execute(TestPlan, TestExecutionListener...) line: 91	
SessionPerRequestLauncher.execute(TestPlan, TestExecutionListener...) line: 60	
JUnit5TestReference.run(TestExecution) line: 100	
TestExecution.run(ITestReference[]) line: 40	
RemoteTestRunner.runTests(String[], String, TestExecution) line: 529	
RemoteTestRunner.runTests(TestExecution) line: 756	
RemoteTestRunner.run() line: 452	
RemoteTestRunner.main(String[]) line: 210	

Expected behavior (i.e. solution)

The functions chained successfully compile and evaluate with a valid result.

Other Comments

In the interim, developers can work around this by writing to the traditional syntax without the arrow operator, but maintainers agreed this is important to fix.

@aj-stein-nist aj-stein-nist added bug Something isn't working java Pull requests that update Java code labels Jan 12, 2024
aj-stein-nist added a commit to aj-stein-nist/metaschema-java that referenced this issue Jan 12, 2024
@aj-stein-nist aj-stein-nist self-assigned this Jan 12, 2024
aj-stein-nist added a commit to aj-stein-nist/metaschema-java that referenced this issue Jan 23, 2024
The index and offset for BuildCSTVisitor for arrow chain operator
context processing was off and therefore exceptions were occuring
when handling => function calls during evaluation and therefore
function calls were not instantiated the correct way. We also
added tests to validate it works now, uncovering another issue
in the use of fn:sum casting integers to decimals, to be fixed in
a subsequent commmit.
aj-stein-nist added a commit to aj-stein-nist/metaschema-java that referenced this issue Jan 23, 2024
The index and offset for BuildCSTVisitor for arrow chain operator
context processing was off and therefore exceptions were occuring
when handling => function calls during evaluation and therefore
function calls were not instantiated the correct way. We also
added tests to validate it works now, uncovering another issue
in the use of fn:sum casting integers to decimals, to be fixed in
a subsequent commmit.
aj-stein-nist added a commit to aj-stein-nist/metaschema-java that referenced this issue Jan 23, 2024
The index and offset for BuildCSTVisitor for arrow chain operator
context processing was off and therefore exceptions were occuring
when handling => function calls during evaluation and therefore
function calls were not instantiated the correct way. We also
added tests to validate it works now, uncovering another issue
in the use of fn:sum casting integers to decimals, to be fixed in
a subsequent commmit.
github-merge-queue bot pushed a commit that referenced this issue Jan 23, 2024
* Make => operator eqname only in ANTLR grammar

Per discussion with maintainers, adjust arrowfunctionspecifier to not
parse the arrow function chain operator as a eqname or varref or a
parenthesizedexpr, but only an eqname to fix parsing complexity. The
relevant code to support this in BuildCSTVisitor has still not been
implemented per the TODO comment.

* Fix BuildCSTVisitor context processing for #289

The index and offset for BuildCSTVisitor for arrow chain operator
context processing was off and therefore exceptions were occuring
when handling => function calls during evaluation and therefore
function calls were not instantiated the correct way. We also
added tests to validate it works now, uncovering another issue
in the use of fn:sum casting integers to decimals, to be fixed in
a subsequent commmit.

* Fix fn:sum to detect integers for return type.
@david-waltermire david-waltermire added this to the v1.0.0-M2 milestone Feb 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working java Pull requests that update Java code
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants