Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic Response mapping using Expr #349

Merged
merged 117 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
16ed0b7
Update examples
afsalthaj Mar 2, 2024
7dbd8dd
Update more info
afsalthaj Mar 2, 2024
2db9659
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 5, 2024
1e3ad83
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 6, 2024
a2e8bbc
Update examples
afsalthaj Mar 6, 2024
2d431b2
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 6, 2024
768e1de
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 7, 2024
16b425e
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 18, 2024
e8f63a7
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 18, 2024
6526d30
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 21, 2024
2c595df
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 22, 2024
7811ec3
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 25, 2024
d2ea735
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 27, 2024
1493ad2
Update wasm-rpc
afsalthaj Mar 27, 2024
6631b66
Initial version of expr transformation
afsalthaj Mar 28, 2024
886a5e6
Remove resolved variables
afsalthaj Mar 28, 2024
a233ee6
Constructor
afsalthaj Mar 28, 2024
086b99f
Remove ResolvedVariables
afsalthaj Mar 28, 2024
3d4c73c
Rename resolved variables from path
afsalthaj Mar 28, 2024
a4de132
Fix spec path variables
afsalthaj Mar 28, 2024
b11e33c
Form type annotated value from request
afsalthaj Mar 28, 2024
8d8f558
Merge remote-tracking branch 'origin/main'
afsalthaj Mar 28, 2024
2e3bbf6
Merge branch 'main' into expr_transformation
afsalthaj Mar 28, 2024
b34b182
Fix test compilation errors
afsalthaj Mar 28, 2024
68d8e8f
Cache template met
afsalthaj Mar 28, 2024
d3d3d2f
Add typed value function in worker service
afsalthaj Mar 29, 2024
499568b
Remove compile time error
afsalthaj Mar 29, 2024
a8f8bb6
Start fixing errors
afsalthaj Mar 29, 2024
d22a079
Fix more errors
afsalthaj Mar 29, 2024
b6b0de7
Fix all main errors
afsalthaj Mar 29, 2024
d8099b7
Fix more errors
afsalthaj Mar 29, 2024
a9d7321
Temporarily depend on wasm-rpc
afsalthaj Mar 30, 2024
84fec20
Add tests for type infernce
afsalthaj Mar 30, 2024
477e67f
Add comments
afsalthaj Mar 30, 2024
2a8c316
Upgrade to latest wasm rpc
afsalthaj Mar 30, 2024
d79a2d2
Remove json dependency from type annotated value module
afsalthaj Mar 30, 2024
807bc15
Fix more errors
afsalthaj Mar 30, 2024
7dc6ddf
Fix more test cases with 30 remaining
afsalthaj Mar 30, 2024
cd856c3
Handle optional field
afsalthaj Mar 30, 2024
ea97ed7
Fix more errors
afsalthaj Mar 30, 2024
1f34f22
Fix evaluator tests with three remaining errors
afsalthaj Mar 30, 2024
bd1af90
Fix all tests in evaluator
afsalthaj Mar 30, 2024
d0e8fdc
Fix type inference and add more tests
afsalthaj Mar 30, 2024
cbb8dff
Fix all tests in http_request
afsalthaj Mar 30, 2024
5f7b3b8
Fix all parser tests
afsalthaj Mar 30, 2024
0088924
Fix all tests
afsalthaj Mar 30, 2024
0500e23
Fix clippy
afsalthaj Mar 30, 2024
081c13b
Fix clippy
afsalthaj Mar 30, 2024
c8ebe68
Remove comments
afsalthaj Mar 30, 2024
33e4c81
Add more test for type inference
afsalthaj Mar 31, 2024
8adfa48
Make the concept of ApiDefinition generic
afsalthaj Mar 31, 2024
109f8e2
Move validator to generic
afsalthaj Mar 31, 2024
083a9ac
make golem worker binding a generic idea
afsalthaj Mar 31, 2024
8c3f274
Start moving everything to an expression module
afsalthaj Mar 31, 2024
c9d6876
Start introducing more modules
afsalthaj Mar 31, 2024
5257a7b
Start compiling
afsalthaj Mar 31, 2024
8c9a8dd
Initial draft of code organisation
afsalthaj Mar 31, 2024
bd22746
Fix errors
afsalthaj Mar 31, 2024
62c96fa
Fix errors
afsalthaj Mar 31, 2024
403a6be
Fix errors
afsalthaj Mar 31, 2024
a88f93e
Add constraints
afsalthaj Mar 31, 2024
9ca9cce
Use Has patterns to support generic
afsalthaj Mar 31, 2024
086fa89
Response binding
afsalthaj Mar 31, 2024
0ec8ba2
First compiled version of draft
afsalthaj Apr 1, 2024
fe9356a
Optimise imports
afsalthaj Apr 1, 2024
9c3e6bb
Make modules private and use pub for crate
afsalthaj Apr 1, 2024
9e61961
Tighten privacy in every module
afsalthaj Apr 1, 2024
efae2ec
Avoid leaking Response mapping outside base module
afsalthaj Apr 1, 2024
683121d
Move internal logic to internal module
afsalthaj Apr 1, 2024
6d159dc
make sure to use mod internal pattern
afsalthaj Apr 1, 2024
062ab5f
Refactor oas_worker_bridge
afsalthaj Apr 1, 2024
afc96c0
Further maximise private modules
afsalthaj Apr 1, 2024
a902f8d
Try to compile golem worker service
afsalthaj Apr 1, 2024
cffc0d3
Fix errors
afsalthaj Apr 1, 2024
292b90c
Fix all compile time errors
afsalthaj Apr 1, 2024
4196b04
Compile everything
afsalthaj Apr 1, 2024
3805fae
Make sure everything compile
afsalthaj Apr 1, 2024
b76a65e
Fix clippy
afsalthaj Apr 1, 2024
4aa9218
Remove unused trait
afsalthaj Apr 1, 2024
4ab73fe
Fix more errors
afsalthaj Apr 1, 2024
76f4893
Fix more errors
afsalthaj Apr 2, 2024
bb3b0f8
Rename
afsalthaj Apr 2, 2024
18bdad3
Rearrange further
afsalthaj Apr 2, 2024
d262d2d
Reorganise code
afsalthaj Apr 2, 2024
2a43c9c
Reformat and rename service
afsalthaj Apr 2, 2024
a9de472
Fix clippy
afsalthaj Apr 2, 2024
0abcc30
Reformat and clippy for worker-service
afsalthaj Apr 2, 2024
cab00f6
Fix format
afsalthaj Apr 2, 2024
e6bfac1
Merge remote-tracking branch 'origin/main'
afsalthaj Apr 2, 2024
3a82193
Merge branch 'main' into expr_transformation
afsalthaj Apr 2, 2024
b4f5824
Fix formatting
afsalthaj Apr 2, 2024
aefed6f
Make ApiDefinition concept generic (#342)
afsalthaj Apr 2, 2024
0ea0a04
Update yaml file
afsalthaj Apr 2, 2024
a37a012
Merge branch 'expr_transformation' into gol_243
afsalthaj Apr 2, 2024
ba742c0
Fix construction
afsalthaj Apr 2, 2024
48e26e4
Merge remote-tracking branch 'origin/main'
afsalthaj Apr 2, 2024
ee4c188
Merge branch 'main' into gol_243
afsalthaj Apr 2, 2024
383aa17
Fix construction
afsalthaj Apr 2, 2024
0d0bd18
Fix none
afsalthaj Apr 2, 2024
3c5d0cc
Add test for nest construction
afsalthaj Apr 2, 2024
80f7038
Add test for Result
afsalthaj Apr 2, 2024
1a76620
Add test
afsalthaj Apr 2, 2024
481aa94
Reformat code
afsalthaj Apr 2, 2024
149eb88
Use proper pattern match
afsalthaj Apr 2, 2024
5154f1f
Use err
afsalthaj Apr 2, 2024
900e5fc
Generic response mapping
afsalthaj Apr 2, 2024
d066a34
Fix warnings
afsalthaj Apr 2, 2024
32ce4b7
Remove todo comment
afsalthaj Apr 2, 2024
b8532e6
Fix tests
afsalthaj Apr 2, 2024
bcdd01b
Fix tests
afsalthaj Apr 2, 2024
dd6f9c4
Reformat code
afsalthaj Apr 3, 2024
e1e94fc
Merge remote-tracking branch 'origin/main'
afsalthaj Apr 3, 2024
f603147
Merge branch 'main' into gol_316
afsalthaj Apr 3, 2024
0b0b019
Upgrade golem-examples
afsalthaj Apr 3, 2024
34afca0
Merge branch 'main' into gol_316
afsalthaj Apr 3, 2024
d8fec3c
Update open api yaml
afsalthaj Apr 3, 2024
7cdadb0
Merge branch 'main' into gol_316
afsalthaj Apr 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 5 additions & 54 deletions golem-worker-service-base/src/api/register_api_definition_api.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
use std::collections::HashMap;
use std::result::Result;

use poem_openapi::*;
Expand Down Expand Up @@ -37,16 +36,7 @@ pub struct GolemWorkerBinding {
pub worker_id: serde_json::value::Value,
pub function_name: String,
pub function_params: Vec<serde_json::value::Value>,
pub response: Option<ResponseMapping>,
}

#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Object)]
pub struct ResponseMapping {
pub body: serde_json::value::Value,
// ${function.return}
pub status: serde_json::value::Value,
// "200" or if ${response.body.id == 1} "200" else "400"
pub headers: HashMap<String, serde_json::value::Value>,
pub response: Option<serde_json::value::Value>,
}

impl TryFrom<crate::api_definition::http::HttpApiDefinition> for HttpApiDefinition {
Expand Down Expand Up @@ -119,52 +109,13 @@ impl TryInto<crate::api_definition::http::Route> for Route {
}
}

impl TryFrom<crate::worker_binding::ResponseMapping> for ResponseMapping {
type Error = String;

fn try_from(value: crate::worker_binding::ResponseMapping) -> Result<Self, Self::Error> {
let body = serde_json::to_value(value.body).map_err(|e| e.to_string())?;
let status = serde_json::to_value(value.status).map_err(|e| e.to_string())?;
let mut headers = HashMap::new();
for (key, value) in value.headers {
let v = serde_json::to_value(value).map_err(|e| e.to_string())?;
headers.insert(key.to_string(), v);
}
Ok(Self {
body,
status,
headers,
})
}
}

impl TryInto<crate::worker_binding::ResponseMapping> for ResponseMapping {
type Error = String;

fn try_into(self) -> Result<crate::worker_binding::ResponseMapping, Self::Error> {
let body: Expr = serde_json::from_value(self.body).map_err(|e| e.to_string())?;
let status: Expr = serde_json::from_value(self.status).map_err(|e| e.to_string())?;
let mut headers = HashMap::new();
for (key, value) in self.headers {
let v: Expr = serde_json::from_value(value).map_err(|e| e.to_string())?;
headers.insert(key.to_string(), v);
}

Ok(crate::worker_binding::ResponseMapping {
body,
status,
headers,
})
}
}

impl TryFrom<crate::worker_binding::GolemWorkerBinding> for GolemWorkerBinding {
type Error = String;

fn try_from(value: crate::worker_binding::GolemWorkerBinding) -> Result<Self, Self::Error> {
let response: Option<ResponseMapping> = match value.response {
let response: Option<serde_json::value::Value> = match value.response {
Some(v) => {
let r = ResponseMapping::try_from(v)?;
let r = Expr::to_json_value(&v.0).map_err(|e| e.to_string())?;
Some(r)
}
None => None,
Expand Down Expand Up @@ -192,8 +143,8 @@ impl TryInto<crate::worker_binding::GolemWorkerBinding> for GolemWorkerBinding {
fn try_into(self) -> Result<crate::worker_binding::GolemWorkerBinding, Self::Error> {
let response: Option<crate::worker_binding::ResponseMapping> = match self.response {
Some(v) => {
let r: crate::worker_binding::ResponseMapping = v.try_into()?;
Some(r)
let r = Expr::from_json_value(&v).map_err(|e| e.to_string())?;
Some(crate::worker_binding::ResponseMapping(r))
}
None => None,
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ pub fn get_api_definition_from_oas(open_api: &str) -> Result<HttpApiDefinition,
}

mod internal {
use crate::api_definition::http::{MethodPattern, PathPattern, Route};
use crate::api_definition::http::{HttpResponseMapping, MethodPattern, PathPattern, Route};
use crate::expression::Expr;
use crate::worker_binding::{GolemWorkerBinding, ResponseMapping};
use golem_common::model::TemplateId;
use openapiv3::{OpenAPI, PathItem, Paths, ReferenceOr};
use serde_json::Value;
use std::collections::HashMap;

use uuid::Uuid;

pub(crate) const GOLEM_API_DEFINITION_ID_EXTENSION: &str = "x-golem-api-definition-id";
Expand Down Expand Up @@ -134,33 +134,16 @@ mod internal {
) -> Result<Option<ResponseMapping>, String> {
let response = worker_bridge_info.get("response");
match response {
Some(response) => Ok(Some(ResponseMapping {
status: Expr::from_json_value(response.get("status").ok_or("No status found")?)
.map_err(|err| err.to_string())?,
headers: {
let mut header_map = HashMap::new();

let header_iter = response
.get("headers")
.ok_or("No headers found")?
.as_object()
.ok_or("headers is not an object")?
.iter();

for (header_name, value) in header_iter {
let value_str = value.as_str().ok_or("Header value is not a string")?;
header_map.insert(
header_name.clone(),
Expr::from_primitive_string(value_str)
.map_err(|err| err.to_string())?,
);
}
Some(response) => {
let response_mapping_expr =
Expr::from_json_value(response).map_err(|err| err.to_string())?;

let _ =
HttpResponseMapping::try_from(&ResponseMapping(response_mapping_expr.clone()))
.map_err(|err| err.to_string())?;

header_map
},
body: Expr::from_json_value(response.get("body").ok_or("No body found")?)
.map_err(|err| err.to_string())?,
})),
Ok(Some(ResponseMapping(response_mapping_expr)))
}
None => Ok(None),
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,228 @@
use std::collections::HashMap;
use std::fmt::Display;

use crate::expression::Expr;
use crate::worker_binding::ResponseMapping;

#[derive(PartialEq, Debug, Clone)]
pub struct HttpResponseMapping {
pub body: Expr, // ${function.return}
pub status: Expr, // "200" or if ${response.body.id == 1} "200" else "400"
pub headers: HashMap<String, Expr>,
}

impl Display for HttpResponseMapping {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let response_mapping: ResponseMapping = self.clone().into();
let expr_json = Expr::to_json_value(&response_mapping.0).unwrap();

write!(f, "{}", expr_json)
}
}

impl From<HttpResponseMapping> for ResponseMapping {
fn from(http_response_mapping: HttpResponseMapping) -> Self {
ResponseMapping(Expr::Record(
vec![
("body".to_string(), Box::new(http_response_mapping.body)),
("status".to_string(), Box::new(http_response_mapping.status)),
(
"headers".to_string(),
Box::new(Expr::Record(
http_response_mapping
.headers
.into_iter()
.map(|(key, value)| (key, Box::new(value)))
.collect::<Vec<(String, Box<Expr>)>>(),
)),
),
]
.into_iter()
.collect(),
))
}
}

impl TryFrom<&ResponseMapping> for HttpResponseMapping {
type Error = String;

fn try_from(response_mapping: &ResponseMapping) -> Result<Self, Self::Error> {
let mut headers = HashMap::new();
let generic_expr = &response_mapping.0;

match generic_expr {
Expr::Record(obj) => {
let mut body = None;
let mut status = None;

for (key, value) in obj {
match key.as_str() {
"body" => body = Some(value),
"status" => status = Some(value),
"headers" => {
if let Expr::Record(headers_obj) = value.as_ref().clone() {
for (header_key, header_value) in headers_obj {
headers
.insert(header_key.clone(), header_value.as_ref().clone());
}
} else {
return Err("headers must be an object".to_string());
}
}
_ => return Err(format!("Unknown key: {}", key)),
}
}

match (body, status) {
(Some(body), Some(status)) => Ok(HttpResponseMapping {
body: body.as_ref().clone(),
status: status.as_ref().clone(),
headers,
}),
(None, _) => Err("body is required in http response mapping".to_string()),
(_, None) => Err("status is required in http response mapping".to_string()),
}
}
_ => Err("response mapping must be a record".to_string()),
}
}
}

#[cfg(test)]
mod tests {
use super::*;
use crate::expression::Expr;
use crate::worker_binding::ResponseMapping;

#[test]
fn test_try_from_response_mapping() {
let response_mapping = ResponseMapping(Expr::Record(
vec![
(
"body".to_string(),
Box::new(Expr::Variable("function.return".to_string())),
),
(
"status".to_string(),
Box::new(Expr::Literal("200".to_string())),
),
(
"headers".to_string(),
Box::new(Expr::Record(vec![(
"Content-Type".to_string(),
Box::new(Expr::Literal("application/json".to_string())),
)])),
),
]
.into_iter()
.collect(),
));

let http_response_mapping = HttpResponseMapping::try_from(&response_mapping).unwrap();

assert_eq!(
http_response_mapping.body,
Expr::Variable("function.return".to_string())
);
assert_eq!(
http_response_mapping.status,
Expr::Literal("200".to_string())
);
assert_eq!(http_response_mapping.headers.len(), 1);
assert_eq!(
http_response_mapping.headers.get("Content-Type").unwrap(),
&Expr::Literal("application/json".to_string())
);
}

#[test]
fn test_try_from_response_mapping_missing_body() {
let response_mapping = ResponseMapping(Expr::Record(
vec![
(
"status".to_string(),
Box::new(Expr::Literal("200".to_string())),
),
(
"headers".to_string(),
Box::new(Expr::Record(vec![(
"Content-Type".to_string(),
Box::new(Expr::Literal("application/json".to_string())),
)])),
),
]
.into_iter()
.collect(),
));

let result = HttpResponseMapping::try_from(&response_mapping);

assert_eq!(
result,
Err("body is required in http response mapping".to_string())
);
}

#[test]
fn test_try_from_response_mapping_missing_status() {
let response_mapping = ResponseMapping(Expr::Record(
vec![
(
"body".to_string(),
Box::new(Expr::Variable("function.return".to_string())),
),
(
"headers".to_string(),
Box::new(Expr::Record(vec![(
"Content-Type".to_string(),
Box::new(Expr::Literal("application/json".to_string())),
)])),
),
]
.into_iter()
.collect(),
));

let result = HttpResponseMapping::try_from(&response_mapping);

assert_eq!(
result,
Err("status is required in http response mapping".to_string())
);
}

#[test]
fn test_try_from_response_mapping_headers_not_object() {
let response_mapping = ResponseMapping(Expr::Record(
vec![
(
"body".to_string(),
Box::new(Expr::Variable("worker.response".to_string())),
),
(
"status".to_string(),
Box::new(Expr::Literal("200".to_string())),
),
(
"headers".to_string(),
Box::new(Expr::Literal("application/json".to_string())),
),
]
.into_iter()
.collect(),
));

let result = HttpResponseMapping::try_from(&response_mapping);

assert_eq!(result, Err("headers must be an object".to_string()));
}

#[test]
fn test_try_from_response_mapping_not_record() {
let response_mapping = ResponseMapping(Expr::Literal("200".to_string()));

let result = HttpResponseMapping::try_from(&response_mapping);

assert_eq!(result, Err("response mapping must be a record".to_string()));
}
}
4 changes: 4 additions & 0 deletions golem-worker-service-base/src/api_definition/http/mod.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
pub use http_api_definition::*;
pub use http_oas_api_definition::get_api_definition_from_oas;
pub(crate) use http_response_mapping::HttpResponseMapping;

mod http_api_definition;
mod http_oas_api_definition;

mod http_response_mapping;
6 changes: 6 additions & 0 deletions golem-worker-service-base/src/evaluator/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@ pub enum EvaluationError {
Message(String),
}

impl From<String> for EvaluationError {
fn from(string: String) -> Self {
EvaluationError::Message(string)
}
}

impl Display for EvaluationError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Expand Down
Loading
Loading