How to handle Action data filter

[VicNgu]

I'm looking at the action data filter and understand how the fromStateData, results, and toStateData are used.

I'm seeing some ambiguity on how to handle a specific edge case. From one of the examples, the results are filtering elements from the action output and placing it into the toStateData element:

{
"actions":[
    {
       "functionRef": "breadAndPastaTypesFunction",
       "actionDataFilter": {
          "results": "${ [ .breads[0], .pasta[1] ] }",
          "toStateData": "${ .itemsToBuyAtStore }"
       }
    }
 ]
}

In this case where the value of results is an array of string, what should happen if toStateData is not specified (since toStateData is optional)?

[cdavernas]

In the spec, you can read the following regarding the toStateDataFilter:

If not specified denotes the top-level state data element

In other words, the whole action result would be merged back into the stateData, which would then become the array of string, in your example.

[VicNgu]

From what I understand, the stateData should be a JSON object. If the array of strings need to be merged into the top level of the stateData, what would be the key for the action data?

Something like this would be invalid JSON:

{
	["bread", "pasta"]
}

[cdavernas]

Good point! @tsurdilo?

[tsurdilo]

@cdavernas @VicNgu
It's pretty hard for spec to define this rule correctly since we do not control the result type.
If you have ideas like how to do this, like set unique id of action/event def and use that as a key, or something else, i am all ears.

For now what is expected is that user understands the result and can use jq to manipulate it, so for your example if result is
{
"breads": ["baguette", "brioche", "rye"],
"pasta": [ "penne", "spaghetti", "ravioli"]
}
then you can use jq:

"results": "${ {"thingstobuy": [.breads[0], .pasta[1] ] } }",

in which case toStateData can still be optional

....jq is pretty magic hehe

[VicNgu]

In that case, I will make a check in my code to see if "results" is a map.
If it's a map, then toStateData is optional.
Else, we will return an error if toStateData is empty.

My thought is that since these filters are all optional, then the user should have some understanding on how to apply them correctly.

[ricardozanini]

I agree with @tsurdilo. Sounds like an implementation detail, but would be nice if the spec could suggest some best practices. Merging the payload is a little bit tricky. The response:

{
    "breads": ["baguette", "brioche", "rye"],
    "pasta": [ "penne", "spaghetti", "ravioli"]
}

Your current workflow payload contains:

{
    "workflowdata": {
         { "attr1": "value1" }
    }
}

How are you going to merge the array in this case? [ "penne", "spaghetti", "ravioli"]. We are using the function reference name, IIRC to be the key in this case.

I think we could develop a little bit more in the spec. wdyt guys?

[VicNgu]

Wouldn't the data merging section https://github.com/serverlessworkflow/specification/blob/main/specification.md#data-merging just apply here?

I would think it would look like

{
    "workflowdata": {
         { "attr1": "value1" }
    }
    "breads": ["baguette", "brioche", "rye"],
    "pasta": [ "penne", "spaghetti", "ravioli"]
}

[ricardozanini]

no, I want only the array. I forgot to add the filter query.

{
    "workflowdata": {
         { "attr1": "value1" },
          [ "penne", "spaghetti", "ravioli"] 
    }
}

And this is invalid.