From ce8ca305699a6425a0a9ed1e4cb3259aac0cf060 Mon Sep 17 00:00:00 2001 From: Marc Abramowitz Date: Sun, 31 Mar 2024 12:24:54 -0700 Subject: [PATCH] Add FE tests w/ Robot Framework MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit After submitting GH-37 as a potential fix for GH-35, I wondered if there was a way to add FE tests that would detect that sort of issue. I found that [Robot Framework] is a popular tool for this, and it has a [SeleniumLibrary] that can be used to interact with web pages. This commit adds a simple test case that opens a browser, inputs text into the `st_keyup` input field, and waits for the page to contain some expected text. The test case is run by a shell script that uses the `robot` command to run the test case. The test case is run by a shell script that uses the `robot` command to run the test case. This test fails with the code on `main` when using Streamlit 1.32.2, because of GH-35: ``` (.venv) abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (fe-tests-robot-framework) $ pip freeze | grep '^streamlit' streamlit==1.32.2 (.venv) abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (fe-tests-robot-framework) $ ./run-robot-tests ============================================================================== Robot ============================================================================== Robot.Test Cases ============================================================================== Interact with st_keyup | FAIL | Element 'css:iframe[title="st_keyup.st_keyup"]' not visible after 5 seconds. ------------------------------------------------------------------------------ Robot.Test Cases | FAIL | 1 test, 0 passed, 1 failed ============================================================================== Robot | FAIL | 1 test, 0 passed, 1 failed ============================================================================== Output: /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/output.xml Log: /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/log.html Report: /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/report.html ``` But it passes with the code in GH-37: ``` (.venv) abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (gh-35-not-rendering-with-streamlit-1.32.2●) $ git branch fe-tests-robot-framework * gh-35-not-rendering-with-streamlit-1.32.2 main (.venv) abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (gh-35-not-rendering-with-streamlit-1.32.2●) $ git checkout -b gh-35-not-rendering-with-streamlit-1.32.2-with-fe-tests-robot-framework Switched to a new branch 'gh-35-not-rendering-with-streamlit-1.32.2-with-fe-tests-robot-framework' (.venv) abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (gh-35-not-rendering-with-streamlit-1.32.2-with-fe-tests-robot-framework●) $ git merge fe-tests-robot-framework Merge made by the 'ort' strategy. .gitignore | 5 ++++- run-robot-tests | 3 +++ tests/robot/resources/app_keywords.robot | 16 ++++++++++++++++ tests/robot/test_cases.robot | 16 ++++++++++++++++ 4 files changed, 39 insertions(+), 1 deletion(-) create mode 100755 run-robot-tests create mode 100644 tests/robot/resources/app_keywords.robot create mode 100644 tests/robot/test_cases.robot (.venv) abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (gh-35-not-rendering-with-streamlit-1.32.2-with-fe-tests-robot-framework) $ ./run-robot-tests ============================================================================== Robot ============================================================================== Robot.Test Cases ============================================================================== Interact with st_keyup | PASS | ------------------------------------------------------------------------------ Robot.Test Cases | PASS | 1 test, 1 passed, 0 failed ============================================================================== Robot | PASS | 1 test, 1 passed, 0 failed ============================================================================== Output: /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/output.xml Log: /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/log.html Report: /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/report.html ``` [Robot Framework]: https://robotframework.org/ [SeleniumLibrary]: https://robotframework.org/SeleniumLibrary/SeleniumLibrary.html --- .gitignore | 5 ++++- run-robot-tests | 3 +++ tests/robot/resources/app_keywords.robot | 16 ++++++++++++++++ tests/robot/test_cases.robot | 16 ++++++++++++++++ 4 files changed, 39 insertions(+), 1 deletion(-) create mode 100755 run-robot-tests create mode 100644 tests/robot/resources/app_keywords.robot create mode 100644 tests/robot/test_cases.robot diff --git a/.gitignore b/.gitignore index a51753e..78025ec 100644 --- a/.gitignore +++ b/.gitignore @@ -32,6 +32,9 @@ latest_logs archived_logs current-screenshot.png +# Robot framework FE tests +tests/robot/output/ + ######################################################################## # OSX - https://github.com/github/gitignore/blob/master/Global/macOS.gitignore ######################################################################## @@ -75,4 +78,4 @@ package-lock.json .direnv/ .envrc -.streamlit/ \ No newline at end of file +.streamlit/ diff --git a/run-robot-tests b/run-robot-tests new file mode 100755 index 0000000..409e858 --- /dev/null +++ b/run-robot-tests @@ -0,0 +1,3 @@ +#!/bin/sh + +robot --outputdir=tests/robot/output --variable=app_url:http://localhost:8501 tests/robot/ diff --git a/tests/robot/resources/app_keywords.robot b/tests/robot/resources/app_keywords.robot new file mode 100644 index 0000000..fac26de --- /dev/null +++ b/tests/robot/resources/app_keywords.robot @@ -0,0 +1,16 @@ +*** Settings *** + +Library SeleniumLibrary + +*** Variables *** + +${st_keyup_iframe_locator} css:iframe[title="st_keyup.st_keyup"] + +*** Keywords *** + +Input text into st_keyup + [Arguments] ${text} + Wait until element is visible ${st_keyup_iframe_locator} + Select frame ${st_keyup_iframe_locator} + Input text css:input[type="text"] ${text} + Unselect frame diff --git a/tests/robot/test_cases.robot b/tests/robot/test_cases.robot new file mode 100644 index 0000000..9b756ef --- /dev/null +++ b/tests/robot/test_cases.robot @@ -0,0 +1,16 @@ +*** Settings *** + +Resource resources/app_keywords.robot + +*** Variables *** + +${app_url} http://localhost:8501 + +*** Test Cases *** + +Interact with st_keyup + Open browser ${app_url} Chrome + Input text into st_keyup adjunta + Wait until page contains URB San Joaquin + Wait until page contains Jard De Adjuntas + Wait until page contains Colinas Del Gigante