Expr transformation

[afsalthaj]

Fixes #314
Fixes #212

On a high level

• Input request details are now as a tree represented using TypeAnnotatedValue
• Worker-Response now returns TypeAnnotatedValue. A new invoke function in worker-service that returns TypeAnnotatedValue (corresponding changes in wasm-rpc), and invoke-function-json reuses this and convert the type-anotated-value to Json.
• Users can write expressions on top of a type-annotated-value and can simultaneously refer to both request and worker response
• For this to happen, Evaluator is now a function that goes from a TypeAnnotatedValue (a merge of input and worker-out, both represented as type-annotated-value) and returns a TypeAnnotatedValue, given an Expr
• Removal of ValueTyped in favour of TypeAnnotatedValue from wasm-rpc
• Removal of GatewayVariables , ResolvedVariables etc which is essentially a hash-map holding all variables across the worker-bridge. The above points contributes to removing them and make things more intuitive.
• A type-inference function that can take a serde_json::Value and get AnalysedType. This is to convert details of a request (which for example is a Json Request Body) to a type-annotated-value. To a great extent, this was done in various places before as well in the code (to evaluate expressions such as ${request.body.user-id} > 1, where we need to fail if user-id is a String. With the above changes, this logic is now more structured in the code. i.e, we make everything typed as early as possible before we do anything!
• In tokeniser of the token was worker.response which is now changed to just worker. worker.response works as before but using Expr::SelectField similar to any field selection - for consistency. This clean up wasn't optional either as type-annotated-value related changes resulted in things like TypeAnnotatedValue::Record { worker -> TypeAnnotatedValue{...}
• And a few other clean up. There were lots of test failures and they are fixed as you can see from the commits.

Note that the changes are corresponding to a local build of wasm-rpc

[afsalthaj]

Progress:

failures:
    expr::tests::expr_to_string_round_trip_match_expr_append
    expr::tests::expr_to_string_round_trip_match_expr_append_suffix
    expr::tests::expr_to_string_round_trip_match_expr_err
    expr::tests::expr_to_string_round_trip_match_expr_ok
    http_request::tests::test_worker_request_cond_expr_resolution
    http_request::tests::test_worker_request_map_list_request_body_resolution
    http_request::tests::test_worker_request_map_request_body_resolution
    http_request::tests::test_worker_request_request_body_direct
    http_request::tests::test_worker_request_request_body_resolution
    http_request::tests::test_worker_request_resolution_with_path_and_query_body_params
    http_request::tests::test_worker_request_resolution_with_path_and_query_params
    http_request::tests::test_worker_request_resolution_with_path_params
    http_request::tests::test_worker_request_resolution_with_record_params
    http_request::tests::test_worker_request_with_request_header_resolution
    http_request::tests::test_worker_resolution_for_cond_expr_req_body_direct_fn_params
    http_request::tests::test_worker_resolution_for_cond_expr_req_body_fn_params
    parser::expr_parser::tests::expr_parser_with_vars

test result: FAILED. 116 passed; 17 failed; 1 ignored; 0 measured; 0 filtered out; finished in 37.83s

[afsalthaj]

Progress:

Mostly tests are failing due to overly flexible type inference from requests, which are being fixed.

failures:
    http_request::tests::test_worker_resolution_for_cond_expr_req_body_direct_fn_params
    http_request::tests::test_worker_resolution_for_cond_expr_req_body_fn_params

test result: FAILED. 16 passed; 2 failed; 0 ignored; 0 measured; 116 filtered out; finished in 3.94s

[afsalthaj]

Finally

running 134 tests
test api_definition::tests::expr_parser_without_vars ... ok
test api_definition::tests::expression_with_predicate0 ... ok
test api_definition::tests::expr_request ... ok
test api_definition::tests::get_path_variables_as_map ... ok
test api_definition::tests::get_query_variables ... ok
test api_definition::tests::split_path_works_with_multiple_values ... ok
test api_definition::tests::split_path_works_with_single_value ... ok
test api_definition::tests::split_path_works_with_variables ... ok
test api_definition::tests::split_path_works_with_variables_and_queries ... ok
test api_definition::tests::expr_worker_response ... ok
test api_definition::tests::expression_with_predicate3 ... ok
test api_definition::tests::test_full_iteration_of_path_pattern ... ok
test api_definition::tests::test_iterator_of_path_pattern ... ok
test api_definition::tests::test_iterator_of_path_pattern_simple ... ok
test api_definition::tests::test_path_patterns_to_string ... ok
test api_definition::tests::expression_with_predicate2 ... ok
test api_definition_repo::tests::test_redis_register ... ignored
test api_request_route_resolver::tests::test_match_literals ... ok
test api_request_route_resolver::tests::test_match_literals_empty_request_path ... ok
test api_request_route_resolver::tests::test_match_literals_empty_spec_path ... ok
test api_definition::tests::expression_with_predicate4 ... ok
test api_definition::tests::expr_parser_with_vars ... ok
test api_definition_repo::tests::test_in_memory_register ... ok
test api_definition::tests::expression_with_predicate1 ... ok
test api_definition::tests::expr_with_if_condition ... ok
test evaluator::tests::test_evaluation_with_pattern_match_none ... ok
test evaluator::tests::test_evaluation_with_pattern_match_use_success_variable ... ok
test evaluator::tests::test_evaluation_with_pattern_match_optional ... ok
test evaluator::tests::test_evaluation_with_request_body_id ... ok
test evaluator::tests::test_evaluation_with_pattern_match ... ok
test evaluator::tests::test_evaluation_with_request_body_index_of_object ... ok
test evaluator::tests::test_evaluation_with_request_body_invalid_object_reference ... ok
test api_definition::tests::expr_with_if_condition_with_expr_right ... ok
test evaluator::tests::test_evaluation_with_request_body_select_index ... ok
test api_definition::tests::expr_with_if_condition_with_expr_left ... ok
test api_definition::tests::expr_with_if_condition_with_with_literals ... ok
test evaluator::tests::test_evaluation_with_request_path ... ok
test evaluator::tests::test_evaluation_with_request_body_select_from_object ... ok
test evaluator::tests::test_evaluation_with_request_body_select_invalid_index ... ok
test evaluator::tests::test_evaluation_with_request_body_select_unknown_field ... ok
test evaluator::tests::test_evaluation_with_zero_worker_response ... ok
test api_definition::tests::expr_with_if_condition_with_expr_left_right ... ok
test evaluator::tests::test_evaluation_with_request_body_invalid_type_comparison ... ok
test expr::tests::test_expr_to_json_value ... ok
test expr::tests::test_round_trip_simple_string ... ok
test expr::tests::test_expr_from_json_value ... ok
test evaluator::tests::test_evaluation_with_request_body_if_condition ... ok
test evaluator::tests::test_evaluation_with_pattern_match_with_select_field ... ok
test evaluator::tests::test_evaluation_with_pattern_match_with_select_from_array ... ok
test http_request::tests::test_worker_request_cond_expr_resolution ... ok
test http_request::tests::test_worker_request_resolution ... ok
test http_request::tests::test_worker_request_request_body_direct ... ok
test http_request::tests::test_worker_request_map_list_request_body_resolution ... ok
test http_request::tests::test_worker_request_map_request_body_resolution ... ok
test http_request::tests::test_worker_request_request_body_resolution ... ok
test http_request::tests::test_worker_request_resolution_with_concrete_params ... ok
test http_request::tests::test_worker_request_resolution_with_path_params ... ok
test http_request::tests::test_worker_request_resolution_with_path_and_query_params ... ok
test http_request::tests::test_worker_request_resolution_with_path_and_query_body_params ... ok
test http_request::tests::test_worker_request_resolution_with_record_params ... ok
test expr::tests::expr_to_string_round_trip_match_expr_append ... ok
test expr::tests::expr_to_string_round_trip_match_expr_err ... ok
test http_request::tests::test_worker_resolution_for_cond_expr_fn_params ... ok
test http_request::tests::test_worker_resolution_for_predicate_gives_bool ... ok
test http_request::tests::test_worker_resolution_for_predicate_gives_bool_greater ... ok
test http_request::tests::test_worker_request_resolution_paths ... ok
test parser::expr_parser::tests::expr_parser_without_vars ... ok
test parser::expr_parser::tests::expr_parser_with_vars ... ok
test parser::expr_parser::tests::expression_with_predicate0 ... ok
test parser::expr_parser::tests::expr_parser_with_path ... ok
test expr::tests::expr_to_string_round_trip_match_expr_append_suffix ... ok
test parser::expr_parser::tests::expr_parser_with_worker_result_path ... ok
test parser::expr_parser::tests::expression_with_predicate_without_outer_interpolation ... ok
test expr::tests::expr_to_string_round_trip_match_expr_ok ... ok
test parser::expr_parser::tests::expression_with_predicate2 ... ok
test parser::expr_parser::tests::expression_with_predicate1 ... ok
test parser::expr_parser::tests::expression_with_predicate000 ... ok
test http_request::tests::test_worker_request_with_request_header_resolution ... ok
test http_request::tests::test_worker_resolution_for_cond_expr_req_body_fn_params ... ok
test parser::expr_parser::tests::test_if_expr_but_as_literal ... ok
test parser::expr_parser::tests::test_if_else_then_expr ... ok
test parser::expr_parser::tests::test_if_else_with_path_variable_in_predicate ... ok
test parser::expr_parser::tests::test_if_expr_with_paranthesis ... ok
test parser::expr_parser::tests::test_if_else_with_path_variable_in_predicate_and_left ... ok
test parser::expr_parser::tests::test_if_else_then_expr_nested ... ok
test parser::expr_parser::tests::test_if_else_with_path_variable_in_predicate_left_and_right ... ok
test http_request::tests::test_worker_resolution_for_cond_expr_req_body_direct_fn_params ... ok
test parser::expr_parser::tests::test_if_expr_with_pattern_match_constants_only ... ok
test parser::expr_parser::tests::test_if_expr_with_pattern_match_variables ... ok
2024-03-30T19:44:03.477780Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: Initiating WebSocket Ping
2024-03-30T19:44:03.477897Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: WebSocket Ping sent
2024-03-30T19:44:03.477991Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: Received WebSocket confirmation Pong
2024-03-30T19:44:03.489234Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: Initiating WebSocket Ping
2024-03-30T19:44:03.489289Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: WebSocket Ping sent
2024-03-30T19:44:03.489365Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: Received WebSocket confirmation Pong
2024-03-30T19:44:03.501044Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: Initiating WebSocket Ping
2024-03-30T19:44:03.501109Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: WebSocket Ping sent
2024-03-30T19:44:03.552129Z DEBUG golem_worker_service_base::service::worker::connect_proxy::keep_alive: Ping confirmation timed out
test service::worker::connect_proxy::keep_alive::test::test_websocket_keep_alive ... ok
test tokeniser::cursor::tests::capture_character_test ... ok
test tokeniser::cursor::tests::capture_empty_test ... ok
test tokeniser::cursor::tests::capture_string_test ... ok
test tokeniser::cursor::tests::capture_string_test_nested ... ok
test tokeniser::cursor::tests::test_capture_string_from ... ok
Fix all tests
test tokeniser::cursor::tests::test_next_non_empty_char ... ok
test parser::expr_parser::tests::test_if_expr_with_grouping_predicate ... ok
test parser::expr_parser::tests::test_if_expr_without_paranthesis ... ok
test tokeniser::tokenizer::tests::test_dot ... ok
test tokeniser::tokenizer::tests::test_false_ifs ... ok
test tokeniser::tokenizer::tests::test_false_ifs2 ... ok
test tokeniser::tokenizer::tests::test_greater_than_partial ... ok
test tokeniser::tokenizer::tests::test_equal_to_with_exprs ... ok
test tokeniser::tokenizer::tests::test_greater_than_no_spaces ... ok
test parser::expr_parser::tests::test_if_expr_with_pattern_match_variable_and_constants ... ok
test tokeniser::tokenizer::tests::test_if_start ... ok
test tokeniser::tokenizer::tests::test_if_then_else_false_expr ... ok
test tokeniser::tokenizer::tests::test_greater_than_with_space ... ok
test tokeniser::tokenizer::tests::test_greater_than_with_exprs ... ok
test tokeniser::tokenizer::tests::test_less_than_with_no_space ... ok
test tokeniser::tokenizer::tests::test_less_than_with_space ... ok
test tokeniser::tokenizer::tests::test_lessthan_partial ... ok
test tokeniser::tokenizer::tests::test_less_than_with_exprs ... ok
test tokeniser::tokenizer::tests::test_open_close_braces ... ok
test tokeniser::tokenizer::tests::test_open_close_square_bracket ... ok
test parser::expr_parser::tests::test_if_expr_with_nested_code ... ok
test tokeniser::tokenizer::tests::test_raw ... ok
test tokeniser::tokenizer::tests::test_request ... ok
test tokeniser::tokenizer::tests::test_if_then_else_predicate ... ok
test parser::expr_parser::tests::test_if_expr_with_pattern_match_nested_some ... ok
test tokeniser::tokenizer::tests::test_if_then_else_multi_line ... ok
test tokeniser::tokenizer::tests::test_with_place_holder_in_beginning ... ok
test tokeniser::tokenizer::tests::test_token_processing_with_dollar ... ok
test tokeniser::tokenizer::tests::test_worker_response ... ok
test tokeniser::tokenizer::tests::test_with_place_holder_in_end ... ok
test tokeniser::tokenizer::tests::test_with_place_holder_in_beginning_and_end ... ok
test tokeniser::tokenizer::tests::test_with_place_holder_anywhere ... ok
test parser::expr_parser::tests::test_if_expr_with_complex_nested_code ... ok
test parser::expr_parser::tests::test_if_expr_with_pattern_match_variables_ok ... ok
test tokeniser::tokenizer::tests::test_token_processing_with_match_expr ... ok
test parser::expr_parser::tests::test_if_else_then_nested_with_equality_predicates ... ok
test evaluator::tests::test_evaluation_with_pattern_match_with_other_exprs ... ok
test api_definition::tests::test_api_spec_encode_decode ... ok

test api_definition::tests::test_api_spec_serde ... ok

test result: ok. 133 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 45.23s

   Doc-tests golem-worker-service-base

[afsalthaj]

Merged #342 to this branch after approval