You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@kzecchini brought up an issue where the sklearn model node requires a hardcoded datadict key that is used by the sklearn preprocessing node, but not necessarily other nodes which may connect to the sklearn model node from upstream. His example is a TF node that creates embeddings, which he wants to cluster using sklearn.
One option is to have a custom key converter / datadict mapping node to rename the datadict key. This seems like overkill for creating a new node.The other option is adding a node config parameter to specify the key to write to the data dict.
We have a lot of redundant code in each of our custom nodes to get upstream data with a specific key and assign it to a variable. We write this code enough that we should maybe abstract this functionality to primrose itself.
would take data_object.data_dict[upstream_data_dict_key_1]
copy it to data_object.data_dict[new_data_key_1]
but then in your code, you still have to use get_upstream_data and then use the new key.
would assign data_object.data_dict[upstream_data_dict_key_1] to object_1
The question is now how does this new object get passed to your custom node like the data_object does?
Maybe add *args to AbstracNode.run? AbstracNode.run(self, data_object, *args) and methods in AbstracNode that are run on init?
I'm just brainstorming some ideas here. Feel free to weigh in @kzecchini
The text was updated successfully, but these errors were encountered:
@kzecchini brought up an issue where the sklearn model node requires a hardcoded datadict key that is used by the sklearn preprocessing node, but not necessarily other nodes which may connect to the sklearn model node from upstream. His example is a TF node that creates embeddings, which he wants to cluster using sklearn.
One option is to have a custom key converter / datadict mapping node to rename the datadict key. This seems like overkill for creating a new node.The other option is adding a node config parameter to specify the key to write to the data dict.
We have a lot of redundant code in each of our custom nodes to get upstream data with a specific key and assign it to a variable. We write this code enough that we should maybe abstract this functionality to primrose itself.
One option:
would take
data_object.data_dict[upstream_data_dict_key_1]
copy it to
data_object.data_dict[new_data_key_1]
but then in your code, you still have to use get_upstream_data and then use the new key.
Option 2:
would assign
data_object.data_dict[upstream_data_dict_key_1]
toobject_1
The question is now how does this new object get passed to your custom node like the data_object does?
Maybe add
*args
toAbstracNode.run
?AbstracNode.run(self, data_object, *args)
and methods in AbstracNode that are run on init?I'm just brainstorming some ideas here. Feel free to weigh in @kzecchini
The text was updated successfully, but these errors were encountered: