initial prototype of flow testing plugin #4

HiroyasuNishiyama · 2021-08-21T05:23:54Z

This PR is a very small prototype implementation to confirm the design direction of the flow tester.
This prototype implements the flow test configuration node and part of the flow test UI for each node.
The actions send and match are only partially implemented tentatively.

The following modifications to Node-RED are required for this plugin to work:
https://github.com/node-red-hitachi/node-red/tree/flow-test-support

[Sample Flow]

[{"id":"7fa23d74517a5fe3","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"943944af8e27e8f3","type":"tab","label":"フロー 1","disabled":false,"info":""},{"id":"b61a827ba9dce0a2","type":"flow-test-config","name":"Test Suite 0","tests":[{"id":"T0","name":"New Test 0","suite":"b61a827ba9dce0a2","actions":{"setup":{"_global_":[{"kind":"send","value":"XYZ","target":"5448bc8e27249d0f"}]},"cleanup":{"_global_":[]},"recv":{"5448bc8e27249d0f":[]},"stub":{"5448bc8e27249d0f":[]},"send":{"5448bc8e27249d0f":[{"kind":"match","value":"XYZ.","target":"1ab07dedd80a7894"}]}}}]},{"id":"1ab07dedd80a7894","type":"inject","z":"943944af8e27e8f3","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":120,"wires":[["5448bc8e27249d0f"]]},{"id":"da3cdd8d6de6bcdd","type":"debug","z":"943944af8e27e8f3","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":400,"y":120,"wires":[]},{"id":"5448bc8e27249d0f","type":"function","z":"943944af8e27e8f3","name":"","func":"msg.payload += \".\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":260,"y":180,"wires":[["da3cdd8d6de6bcdd"]]}]

knolleary

Thanks for making a start on this. Some initial comments here - I haven't dug into the UI bits too much, more on the overall approach.

knolleary · 2021-08-27T11:00:02Z

nodes/flow-test-config.html

@@ -0,0 +1,61 @@
+<!--


Please do not add the license header. We're moving away from doing that as the consensus is having the top-level LICENSE file is sufficient.

removed license from code.

knolleary · 2021-08-27T11:02:10Z

src/flow-tester.js

+        checks = [];
+        checkSend = {};
+
+        RED.hooks.add("onSend.flow-test", (events, done) => {


We need to discuss how the runtime side is going to work. I don't believe runtime hooks are enough here - although they will play a part.

I would like to understand what you are planning here, or if this is just a place holder to show the concept.

The purpose of this runtime hook is to insert code in the middle to inspect the message.
In the initial design, the idea of using a pluggable message router was proposed.

knolleary · 2021-08-27T11:02:50Z

src/flow-tester.html

+
+    function invokeAction(kind, opt) {
+        return $.ajax({
+            url: "/flow-tester/executeAction/"+ kind,


Do not start the url with / - otherwise it will not work if the user changes httpAdminRoot.

removed the /.

knolleary · 2021-08-27T11:05:37Z

src/flow-tester.html

+            RED.events.on("nodes:remove", handleNodeRemove);
+            RED.events.on("nodes:change", handleNodeChange);
+
+            RED.events.on("nodes:dialog-open", handleNodeDialogOpen);


As I mentioned in the original design discussion, we need a proper design for how plugins can contribute panes to the editor dialog. I've not seen any discussion on that.

I don't think this event mechanism is the right approach. I have something in mind and I will get that written up in the next few days.

Thank you. I think being able to hook various dialogs would be useful also in other cases.

knolleary · 2021-08-27T11:07:32Z

src/flow-tester.js

+        },
+        onadd: () => {
+            const routeAuthHandler = RED.auth.needsPermission("flow-tester.write");
+            RED.httpAdmin.post(


It would be good to have some documentation for each of these actions to clearly define what it is meant to do - even if not yet implemented or just a placeholder.

added simple descriptions.

linux-foundation-easycla · 2021-11-15T05:38:12Z

The committers are authorized under a signed CLA.

✅ Hiroyasu Nishiyama (0a46727, bddb096, 7dbbe64, c88a601)

HiroyasuNishiyama · 2021-11-15T06:16:26Z

I have committed the current flow-tester code.
This commit is intended to implement most of the basic functionality.
However, I think the UI, actions, and other features need to be refined.

UI Improvements,
Enhancement to actions support,
Runtime support including:
- notification of node's setting panel extension,
- global configuration node not referenced from other node
...

I would appreciate it if you could comment.

KazuhiroItoh · 2022-03-18T06:57:10Z

I have committed a prototype version of GUI Testing.
It can be used by installing the GUI test addon in the flow testing plugin.

I have attached a sample flow to execute each action of the GUI Testing.
./node-red-contrib-flow-tester-gui-addon/examples/GUI-Test-Example.json

knolleary · 2022-03-23T11:57:58Z

Please remove the GUI testing from this Pull-Request. I have said multiple times the GUI testing work should not be part of the core flow testing plugin.

Raise a separate pull-request with the GUI work in. By adding it to this PR you are blocking any progress being made to get the core of the flow testing framework moving forward.

knolleary · 2022-03-23T12:39:00Z

@HiroyasuNishiyama we again have the problem that this is a huge PR which has had months of development work happen without any feedback or discussion over important details. The bigger the PR, the more time I have to find to spend understanding it. That then takes longer for me to do the review, and you continue to work on it and make it even bigger (such as the 'action extension' apis and the gui testing work).

I had hopped our regular status chats would help us do things in smaller iterations with better feedback and communication. Rather than end up with huge pieces of work that will take me days to understand properly.

The fact I have to run a hitachi fork for Node-RED to even try out the plugin makes it harder still.

So let us focus on one bit at a time.

Following the initial design discussion, we identified some requirements on the core of Node-RED. The main one was adding custom edit panes to the node dialogs. In September last year we merged node-red/node-red#3130 - and I called out Flow Testing as being the reason for doing that work. I documented it here: node-red/designs#63 - it would appear I didn't do a good enough job to make you aware of it.

I think that removes the need for the nodes:dialog-prepare and nodes:dialog-resize events.

If there are other core changes required we need a PR to be raised so they can be discussed.

knolleary · 2022-03-23T14:49:04Z

If I have understood the implementation notes correctly, to run a test hooks are added to the message handling path. The existing flows are not stopped or modified otherwise.

If you have a flow that starts with an MQTT In node, and you want to test different messages it could send, how does that work? Does it block any "real" messages the MQTT In node wants to send? Does it stop that node or disable it?

This was one of the big open questions I had with how we would implement flow testing - how we manage the flows when we want to run a test. I had assumed we would need to stop the flows and restart them in a 'test' mode - but I wasn't sure how that would work and it doesn't look like that is the approach you have taken.

KazuhiroItoh · 2022-03-29T02:03:32Z

Please remove the GUI testing from this Pull-Request. I have said multiple times the GUI testing work should not be part of the core flow testing plugin.

I'm sorry. Removed GUI test from this PR.
I have created a separate PR for GUI testing.

initial prototype of flow testing plugin

0a46727

knolleary reviewed Aug 27, 2021

View reviewed changes

HiroyasuNishiyama added 3 commits September 1, 2021 20:06

remove license

bddb096

add description of runtime actions

7dbbe64

merge initial prototype

c88a601

add action addon with examples & other fixes

1f457f6

HiroyasuNishiyama mentioned this pull request Dec 13, 2021

Add implementation note of flow-testing prototype node-red/designs#65

Open

HiroyasuNishiyama and others added 2 commits December 22, 2021 20:53

update addon mechanism & refine sidebar UI

be3f75f

add GUI test action addon

959b7e8

Remove the GUI　test addon

75048a6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

initial prototype of flow testing plugin #4

initial prototype of flow testing plugin #4

HiroyasuNishiyama commented Aug 21, 2021

knolleary left a comment

knolleary Aug 27, 2021

HiroyasuNishiyama Sep 1, 2021

knolleary Aug 27, 2021

HiroyasuNishiyama Sep 1, 2021

knolleary Aug 27, 2021

HiroyasuNishiyama Sep 1, 2021

knolleary Aug 27, 2021

HiroyasuNishiyama Sep 1, 2021

knolleary Aug 27, 2021

HiroyasuNishiyama Sep 1, 2021

linux-foundation-easycla bot commented Nov 15, 2021 •

edited

Loading

HiroyasuNishiyama commented Nov 15, 2021

KazuhiroItoh commented Mar 18, 2022

knolleary commented Mar 23, 2022

knolleary commented Mar 23, 2022

knolleary commented Mar 23, 2022

KazuhiroItoh commented Mar 29, 2022

initial prototype of flow testing plugin #4

Are you sure you want to change the base?

initial prototype of flow testing plugin #4

Conversation

HiroyasuNishiyama commented Aug 21, 2021

knolleary left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linux-foundation-easycla bot commented Nov 15, 2021 • edited Loading

HiroyasuNishiyama commented Nov 15, 2021

KazuhiroItoh commented Mar 18, 2022

knolleary commented Mar 23, 2022

knolleary commented Mar 23, 2022

knolleary commented Mar 23, 2022

KazuhiroItoh commented Mar 29, 2022

linux-foundation-easycla bot commented Nov 15, 2021 •

edited

Loading