Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ApiDefinition concept generic #342

Merged
merged 40 commits into from
Apr 2, 2024
Merged

Make ApiDefinition concept generic #342

merged 40 commits into from
Apr 2, 2024

Conversation

afsalthaj
Copy link
Contributor

@afsalthaj afsalthaj commented Mar 31, 2024

Fixes #318
Fixes #280

  • Make GolemWorkerBinding common to all protocols

  • Pull out common data types for an ApiDefinition into api_definition module, applicable to all protocols

  • Make the current resolver generic, where given a requesst (of any protocl) , it can resolve to a GolemWorkerBinding.

  • Make a new module http, which has specific data model for HttpApiDefinition, input http-request, resolver (for golem worker binding) specific to http request

  • Make Validation generic

  • Introduce HasGolemWorkerBinding

  • Better module organisations and privacy

      * internal module within every module for adhoc functions
      * Minimise the number of public modules and functions
      * Use only pub(crate) for modules and functions that's used across the crate, and not expose it further.
      * Expose only those modules and functions in golem-worker-service that's used by golem-worker-service
    
  • Avoid leak of details. ResponseMapping and how to convert etc was called from worker-service. While this is code-reuse this was never required. Given a WorkerResponse (type-annotated-value) rest of the details should be contained within golem-worker-service-base

New Modules in golem-worker-service-base (I am revisiting some of this one last time, after trying to use it from the cloud)

.
├── Cargo.toml
└── src
    ├── api
    │   ├── common.rs
    │   ├── custom_http_request_api.rs
    │   ├── error.rs
    │   ├── healthcheck.rs
    │   ├── mod.rs
    │   └── register_api_definition_api.rs
    ├── api_definition
    │   ├── api_common.rs
    │   ├── http
    │   │   ├── http_api_definition.rs
    │   │   ├── http_oas_api_definition.rs
    │   │   └── mod.rs
    │   └── mod.rs
    ├── app_config.rs
    ├── auth.rs
    ├── evaluator
    │   ├── getter.rs
    │   ├── mod.rs
    │   └── path.rs
    ├── expression
    │   ├── expr.rs
    │   └── mod.rs
    ├── http
    │   ├── http_request.rs
    │   └── mod.rs
    ├── lib.rs
    ├── merge.rs
    ├── metrics.rs
    ├── openapi_yaml.rs
    ├── parser
    │   ├── expr_parser.rs
    │   ├── literal_parser.rs
    │   ├── mod.rs
    │   ├── path_pattern_parser.rs
    │   └── place_holder_parser.rs
    ├── primitive.rs
    ├── repo
    │   ├── api_definition_repo.rs
    │   └── mod.rs
    ├── service
    │   ├── api_definition.rs
    │   ├── api_definition_lookup.rs
    │   ├── api_definition_validator.rs
    │   ├── http
    │   │   ├── http_api_definition_validator.rs
    │   │   └── mod.rs
    │   ├── mod.rs
    │   ├── template
    │   │   ├── default.rs
    │   │   ├── error.rs
    │   │   └── mod.rs
    │   └── worker
    │       ├── connect_proxy.rs
    │       ├── connect_stream.rs
    │       ├── default.rs
    │       ├── error.rs
    │       └── mod.rs
    ├── tokeniser
    │   ├── cursor.rs
    │   ├── mod.rs
    │   └── tokenizer.rs
    ├── worker_binding
    │   ├── golem_worker_binding.rs
    │   ├── mod.rs
    │   └── worker_binding_resolver.rs
    └── worker_bridge_execution
        ├── mod.rs
        ├── worker_request_executor.rs
        └── worker_response.rs

@afsalthaj afsalthaj marked this pull request as ready for review April 2, 2024 08:01
pub template: TemplateId,
pub detail: String,
}
// #[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Object)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete if not needed anymore

@afsalthaj afsalthaj merged commit aefed6f into expr_transformation Apr 2, 2024
@afsalthaj afsalthaj deleted the gol_318 branch April 2, 2024 09:48
@afsalthaj afsalthaj mentioned this pull request Apr 2, 2024
afsalthaj added a commit that referenced this pull request Apr 2, 2024
* Update examples

* Update more info

* Update examples

* Update wasm-rpc

* Initial version of expr transformation

* Remove resolved variables

* Constructor

* Remove ResolvedVariables

* Rename resolved variables from path

* Fix spec path variables

* Form type annotated value from request

* Fix test compilation errors

* Cache template met

* Add typed value function in worker service

* Remove compile time error

* Start fixing errors

* Fix more errors

* Fix all main errors

* Fix more errors

* Temporarily depend on wasm-rpc

* Add tests for type infernce

* Add comments

* Upgrade to latest wasm rpc

* Remove json dependency from type annotated value module

* Fix more errors

* Fix more test cases with 30 remaining

* Handle optional field

* Fix more errors

* Fix evaluator tests with three remaining errors

* Fix all tests in evaluator

* Fix type inference and add more tests

* Fix all tests in http_request

* Fix all parser tests

* Fix all tests

* Fix clippy

* Fix clippy

* Remove comments

* Add more test for type inference

* Fix formatting

* Make ApiDefinition concept generic (#342)

* Make the concept of ApiDefinition generic

* Move validator to generic

* make golem worker binding a generic idea

* Start moving everything to an expression module

* Start introducing more modules

* Start compiling

* Initial draft of code organisation

* Fix errors

* Fix errors

* Fix errors

* Add constraints

* Use Has patterns to support generic

* Response binding

* First compiled version of draft

* Optimise imports

* Make modules private and use pub for crate

* Tighten privacy in every module

* Avoid leaking Response mapping outside base module

* Move internal logic to internal module

* make sure to use mod internal pattern

* Refactor oas_worker_bridge

* Further maximise private modules

* Try to compile golem worker service

* Fix errors

* Fix all compile time errors

* Compile everything

* Make sure everything compile

* Fix clippy

* Remove unused trait

* Fix more errors

* Fix more errors

* Rename

* Rearrange further

* Reorganise code

* Reformat and rename service

* Fix clippy

* Reformat and clippy for worker-service

* Fix formatting

* Fix worker service test

* Delete commented code

* Update yaml file
afsalthaj added a commit that referenced this pull request Apr 3, 2024
* Update examples

* Update more info

* Update examples

* Update wasm-rpc

* Initial version of expr transformation

* Remove resolved variables

* Constructor

* Remove ResolvedVariables

* Rename resolved variables from path

* Fix spec path variables

* Form type annotated value from request

* Fix test compilation errors

* Cache template met

* Add typed value function in worker service

* Remove compile time error

* Start fixing errors

* Fix more errors

* Fix all main errors

* Fix more errors

* Temporarily depend on wasm-rpc

* Add tests for type infernce

* Add comments

* Upgrade to latest wasm rpc

* Remove json dependency from type annotated value module

* Fix more errors

* Fix more test cases with 30 remaining

* Handle optional field

* Fix more errors

* Fix evaluator tests with three remaining errors

* Fix all tests in evaluator

* Fix type inference and add more tests

* Fix all tests in http_request

* Fix all parser tests

* Fix all tests

* Fix clippy

* Fix clippy

* Remove comments

* Add more test for type inference

* Make the concept of ApiDefinition generic

* Move validator to generic

* make golem worker binding a generic idea

* Start moving everything to an expression module

* Start introducing more modules

* Start compiling

* Initial draft of code organisation

* Fix errors

* Fix errors

* Fix errors

* Add constraints

* Use Has patterns to support generic

* Response binding

* First compiled version of draft

* Optimise imports

* Make modules private and use pub for crate

* Tighten privacy in every module

* Avoid leaking Response mapping outside base module

* Move internal logic to internal module

* make sure to use mod internal pattern

* Refactor oas_worker_bridge

* Further maximise private modules

* Try to compile golem worker service

* Fix errors

* Fix all compile time errors

* Compile everything

* Make sure everything compile

* Fix clippy

* Remove unused trait

* Fix more errors

* Fix more errors

* Rename

* Rearrange further

* Reorganise code

* Reformat and rename service

* Fix clippy

* Reformat and clippy for worker-service

* Fix format

* Fix formatting

* Make ApiDefinition concept generic (#342)

* Make the concept of ApiDefinition generic

* Move validator to generic

* make golem worker binding a generic idea

* Start moving everything to an expression module

* Start introducing more modules

* Start compiling

* Initial draft of code organisation

* Fix errors

* Fix errors

* Fix errors

* Add constraints

* Use Has patterns to support generic

* Response binding

* First compiled version of draft

* Optimise imports

* Make modules private and use pub for crate

* Tighten privacy in every module

* Avoid leaking Response mapping outside base module

* Move internal logic to internal module

* make sure to use mod internal pattern

* Refactor oas_worker_bridge

* Further maximise private modules

* Try to compile golem worker service

* Fix errors

* Fix all compile time errors

* Compile everything

* Make sure everything compile

* Fix clippy

* Remove unused trait

* Fix more errors

* Fix more errors

* Rename

* Rearrange further

* Reorganise code

* Reformat and rename service

* Fix clippy

* Reformat and clippy for worker-service

* Fix formatting

* Fix worker service test

* Delete commented code

* Update yaml file

* Fix construction

* Fix construction

* Fix none

* Add test for nest construction

* Add test for Result

* Add test

* Reformat code

* Use proper pattern match

* Use err

* Avoid using unsafe index

* Rename function name

* Rename function name
afsalthaj added a commit that referenced this pull request Apr 3, 2024
* Update examples

* Update more info

* Update examples

* Update wasm-rpc

* Initial version of expr transformation

* Remove resolved variables

* Constructor

* Remove ResolvedVariables

* Rename resolved variables from path

* Fix spec path variables

* Form type annotated value from request

* Fix test compilation errors

* Cache template met

* Add typed value function in worker service

* Remove compile time error

* Start fixing errors

* Fix more errors

* Fix all main errors

* Fix more errors

* Temporarily depend on wasm-rpc

* Add tests for type infernce

* Add comments

* Upgrade to latest wasm rpc

* Remove json dependency from type annotated value module

* Fix more errors

* Fix more test cases with 30 remaining

* Handle optional field

* Fix more errors

* Fix evaluator tests with three remaining errors

* Fix all tests in evaluator

* Fix type inference and add more tests

* Fix all tests in http_request

* Fix all parser tests

* Fix all tests

* Fix clippy

* Fix clippy

* Remove comments

* Add more test for type inference

* Make the concept of ApiDefinition generic

* Move validator to generic

* make golem worker binding a generic idea

* Start moving everything to an expression module

* Start introducing more modules

* Start compiling

* Initial draft of code organisation

* Fix errors

* Fix errors

* Fix errors

* Add constraints

* Use Has patterns to support generic

* Response binding

* First compiled version of draft

* Optimise imports

* Make modules private and use pub for crate

* Tighten privacy in every module

* Avoid leaking Response mapping outside base module

* Move internal logic to internal module

* make sure to use mod internal pattern

* Refactor oas_worker_bridge

* Further maximise private modules

* Try to compile golem worker service

* Fix errors

* Fix all compile time errors

* Compile everything

* Make sure everything compile

* Fix clippy

* Remove unused trait

* Fix more errors

* Fix more errors

* Rename

* Rearrange further

* Reorganise code

* Reformat and rename service

* Fix clippy

* Reformat and clippy for worker-service

* Fix format

* Fix formatting

* Make ApiDefinition concept generic (#342)

* Make the concept of ApiDefinition generic

* Move validator to generic

* make golem worker binding a generic idea

* Start moving everything to an expression module

* Start introducing more modules

* Start compiling

* Initial draft of code organisation

* Fix errors

* Fix errors

* Fix errors

* Add constraints

* Use Has patterns to support generic

* Response binding

* First compiled version of draft

* Optimise imports

* Make modules private and use pub for crate

* Tighten privacy in every module

* Avoid leaking Response mapping outside base module

* Move internal logic to internal module

* make sure to use mod internal pattern

* Refactor oas_worker_bridge

* Further maximise private modules

* Try to compile golem worker service

* Fix errors

* Fix all compile time errors

* Compile everything

* Make sure everything compile

* Fix clippy

* Remove unused trait

* Fix more errors

* Fix more errors

* Rename

* Rearrange further

* Reorganise code

* Reformat and rename service

* Fix clippy

* Reformat and clippy for worker-service

* Fix formatting

* Fix worker service test

* Delete commented code

* Update yaml file

* Fix construction

* Fix construction

* Fix none

* Add test for nest construction

* Add test for Result

* Add test

* Reformat code

* Use proper pattern match

* Use err

* Generic response mapping

* Fix warnings

* Remove todo comment

* Fix tests

* Fix tests

* Reformat code

* Upgrade golem-examples

* Update open api yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants