Skip to content

Commit

Permalink
Add FE tests w/ Robot Framework
Browse files Browse the repository at this point in the history
After submitting blackaryGH-37 as a potential fix for blackaryGH-35, I wondered if there was a
way to add FE tests that would detect that sort of issue. I found that [Robot
Framework] is a popular tool for this, and it has a [SeleniumLibrary] that can
be used to interact with web pages. This commit adds a simple test case that
opens a browser, inputs text into the `st_keyup` input field, and waits for the
page to contain some expected text. The test case is run by a shell script that
uses the `robot` command to run the test case.  The test case is run by a shell
script that uses the `robot` command to run the test case.

This test fails with the code on `main` when using Streamlit 1.32.2, because of
blackaryGH-35:

```
(.venv)
abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (fe-tests-robot-framework)
$ pip freeze | grep '^streamlit'
streamlit==1.32.2

(.venv)
abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (fe-tests-robot-framework)
$ ./run-robot-tests
==============================================================================
Robot
==============================================================================
Robot.Test Cases
==============================================================================
Interact with st_keyup                                                | FAIL |
Element 'css:iframe[title="st_keyup.st_keyup"]' not visible after 5 seconds.
------------------------------------------------------------------------------
Robot.Test Cases                                                      | FAIL |
1 test, 0 passed, 1 failed
==============================================================================
Robot                                                                 | FAIL |
1 test, 0 passed, 1 failed
==============================================================================
Output:  /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/output.xml
Log:     /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/log.html
Report:  /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/report.html
```

But it passes with the code in blackaryGH-37:

```
(.venv)
abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (blackarygh-35-not-rendering-with-streamlit-1.32.2●)
$ git branch
  fe-tests-robot-framework
* blackarygh-35-not-rendering-with-streamlit-1.32.2
  main

(.venv)
abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (blackarygh-35-not-rendering-with-streamlit-1.32.2●)
$ git checkout -b blackarygh-35-not-rendering-with-streamlit-1.32.2-with-fe-tests-robot-framework
Switched to a new branch 'blackarygh-35-not-rendering-with-streamlit-1.32.2-with-fe-tests-robot-framework'

(.venv)
abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (blackarygh-35-not-rendering-with-streamlit-1.32.2-with-fe-tests-robot-framework●)
$ git merge fe-tests-robot-framework
Merge made by the 'ort' strategy.
 .gitignore                               |  5 ++++-
 run-robot-tests                          |  3 +++
 tests/robot/resources/app_keywords.robot | 16 ++++++++++++++++
 tests/robot/test_cases.robot             | 16 ++++++++++++++++
 4 files changed, 39 insertions(+), 1 deletion(-)
 create mode 100755 run-robot-tests
 create mode 100644 tests/robot/resources/app_keywords.robot
 create mode 100644 tests/robot/test_cases.robot

(.venv)
abramowi at Marcs-MBP-3 in ~/Code/OpenSource/streamlit-keyup (blackarygh-35-not-rendering-with-streamlit-1.32.2-with-fe-tests-robot-framework)
$ ./run-robot-tests
==============================================================================
Robot
==============================================================================
Robot.Test Cases
==============================================================================
Interact with st_keyup                                                | PASS |
------------------------------------------------------------------------------
Robot.Test Cases                                                      | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Robot                                                                 | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Output:  /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/output.xml
Log:     /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/log.html
Report:  /Users/abramowi/Code/OpenSource/streamlit-keyup/tests/robot/output/report.html
```

[Robot Framework]: https://robotframework.org/
[SeleniumLibrary]: https://robotframework.org/SeleniumLibrary/SeleniumLibrary.html
  • Loading branch information
msabramo committed Mar 31, 2024
1 parent da7a18a commit ce8ca30
Show file tree
Hide file tree
Showing 4 changed files with 39 additions and 1 deletion.
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ latest_logs
archived_logs
current-screenshot.png

# Robot framework FE tests
tests/robot/output/

########################################################################
# OSX - https://github.com/github/gitignore/blob/master/Global/macOS.gitignore
########################################################################
Expand Down Expand Up @@ -75,4 +78,4 @@ package-lock.json
.direnv/
.envrc

.streamlit/
.streamlit/
3 changes: 3 additions & 0 deletions run-robot-tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh

robot --outputdir=tests/robot/output --variable=app_url:http://localhost:8501 tests/robot/
16 changes: 16 additions & 0 deletions tests/robot/resources/app_keywords.robot
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
*** Settings ***

Library SeleniumLibrary

*** Variables ***

${st_keyup_iframe_locator} css:iframe[title="st_keyup.st_keyup"]

*** Keywords ***

Input text into st_keyup
[Arguments] ${text}
Wait until element is visible ${st_keyup_iframe_locator}
Select frame ${st_keyup_iframe_locator}
Input text css:input[type="text"] ${text}
Unselect frame
16 changes: 16 additions & 0 deletions tests/robot/test_cases.robot
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
*** Settings ***

Resource resources/app_keywords.robot

*** Variables ***

${app_url} http://localhost:8501

*** Test Cases ***

Interact with st_keyup
Open browser ${app_url} Chrome
Input text into st_keyup adjunta
Wait until page contains URB San Joaquin
Wait until page contains Jard De Adjuntas
Wait until page contains Colinas Del Gigante

0 comments on commit ce8ca30

Please sign in to comment.