Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bug] Flaky test: test_autocomplete_shelf_filter #400

Closed
nemesifier opened this issue Aug 2, 2024 · 0 comments · Fixed by #429
Closed

[bug] Flaky test: test_autocomplete_shelf_filter #400

nemesifier opened this issue Aug 2, 2024 · 0 comments · Fixed by #429
Labels
bug

Comments

@nemesifier
Copy link
Member

nemesifier commented Aug 2, 2024 •
edited
Loading

Describe the bug
The following test: test_project.tests.test_selenium.TestAutocompleteFilter.test_autocomplete_shelf_filter it's failing 10% of times on CI.

Output:

======================================================================
FAIL: test_autocomplete_shelf_filter (test_project.tests.test_selenium.TestAutocompleteFilter)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/unittest/case.py", line 591, in run
    self._callTestMethod(testMethod)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/unittest/case.py", line 549, in _callTestMethod
    method()
  File "/home/runner/work/openwisp-utils/openwisp-utils/tests/test_project/tests/test_selenium.py", line 727, in test_autocomplete_shelf_filter
    self.assertIn(horror_shelf.name, self.web_driver.page_source)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/unittest/case.py", line 1112, in assertIn
    self.fail(self._formatMessage(msg, standardMsg))
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/unittest/case.py", line 675, in fail
    raise self.failureException(msg)
AssertionError: 'Horror' not found in '<html lang="en-us" dir="ltr"><head>\n<title>Select book to change | OpenWISP Admin</title>\n<link rel="stylesheet" type="text/css" href="/static/admin/css/base.css">\n\n\n  \n  \n\n\n  \n    <link rel="stylesheet" type="text/css" href="/static/admin/css/openwisp.css" media="all">\n  \n    <link rel="stylesheet" type="text/css" href="/static/menu-test.css" media="all">\n  \n    <link rel="icon" type="image/x-icon" href="/static/ui/openwisp/images/favicon.png">\n  \n\n\n  \n   <style>\n    *, *:before, *:after {\n      transition:none!important;\n    }\n   </style>\n  \n\n  <link rel="stylesheet" type="text/css" href="/static/admin/css/changelists.css">\n  \n  <link rel="stylesheet" type="text/css" href="/static/admin/css/ow-filters.css">\n  \n    <script src="/admin/jsi18n/"></script>\n  \n  \n    <script src="/static/admin/js/ow-filter.js"></script>\n  \n  <link href="/static/admin/css/vendor/select2/select2.min.css" media="screen" rel="stylesheet">\n<link href="/static/django-admin-autocomplete-filter/css/autocomplete-fix.css" media="screen" rel="stylesheet">\n<link href="/static/admin/css/ow-auto-filter.css" media="screen" rel="stylesheet">\n<link href="/static/admin/css/autocomplete.css" media="screen" rel="stylesheet">\n  \n\n\n\n\n<script src="/static/admin/js/vendor/jquery/jquery.min.js"></script>\n<script src="/static/admin/js/vendor/select2/select2.full.min.js"></script>\n<script src="/static/admin/js/vendor/select2/i18n/en.js"></script>\n<script src="/static/admin/js/jquery.init.js"></script>\n<script src="/static/admin/js/core.js"></script>\n<script src="/static/admin/js/autocomplete.js"></script>\n<script src="/static/admin/js/admin/RelatedObjectLookups.js"></script>\n<script src="/static/django-admin-autocomplete-filter/js/autocomplete_filter_qs.js"></script>\n<script src="/static/admin/js/actions.js"></script>\n<script src="/static/admin/js/ow-auto-filter.js"></script>\n<script src="/static/admin/js/urlify.js"></script>\n<script src="/static/admin/js/prepopulate.js"></script>\n<script src="/static/admin/js/vendor/xregexp/xregexp.min.js"></script>\n\n\n    <meta name="viewport" content="user-scalable=no, width=device-width, initial-scale=1.0, maximum-scale=1.0">\n    <link rel="stylesheet" type="text/css" href="/static/admin/css/responsive.css">\n    \n\n<meta name="robots" content="NONE,NOARCHIVE">\n</head>\n\n<body class=" app-test_project model-book change-list" data-admin-utc-offset="0">\n\n<!-- Container -->\n<div id="container" class="">\n  \n    <div class="menu-backdrop"></div>\n    <div id="menu">\n      <div class="menu-bar">\n        \n  <div class="head">\n    <span tabindex="0" role="button" aria-label="Minimize menu" class="hamburger">\n    <span></span></span>\n    <h1 id="site-name"><a href="/admin/">OpenWISP</a></h1>\n  </div>\n\n      </div>\n      \n      \n<button class="menu-toggle" title="Minimize menu">\n  <span></span>\n</button>\n<nav class="nav">\n   \n  <a class="menu-item" href="/admin" aria-label="Home">\n    <span class="ow-dashboard-icon icon"></span>\n    <span class="label">Home</span>\n  </a>\n    \n  <a class="menu-item" href="/admin/test_project/shelf/" aria-label="Shelfs">\n    <span class="shelf icon"></span>\n    <span class="label">Shelfs</span>\n  </a>\n    \n  <div class="menu-group">\n    <div id="mg-control-[32](https://github.com/openwisp/openwisp-utils/actions/runs/10221026912/job/28282626459#step:6:33)" aria-label="Authentication And Authorization" tabindex="0" role="button" aria-expanded="false" aria-controls="mg-dropdown-32" class="mg-head">\n      <div class="mg-heading">\n        <span class="auth icon"></span>\n        <span class="label">Authentication And Authorization</span>\n      </div>\n      <span class="mg-arrow"></span>\n    </div>\n    <div id="mg-dropdown-32" role="region" aria-labelledby="mg-control-32" class="mg-dropdown">\n      <div class="mg-dropdown-label">Authentication And Authorization</div>\n      <ul class="mg-list">\n        \n        <li>\n          <a class="mg-link" href="/admin/auth/user/" aria-label="Users" tabindex="-1">\n            <span class="user icon"></span>\n            <span class="label">Users</span>\n          </a>\n        </li>\n        \n        <li>\n          <a class="mg-link" href="/admin/auth/user/add/" aria-label="Add user" tabindex="-1">\n            <span class="add-user icon"></span>\n            <span class="label">Add user</span>\n          </a>\n        </li>\n        \n      </ul>\n    </div>\n  </div>\n    \n  <div class="menu-group">\n    <div id="mg-control-[33](https://github.com/openwisp/openwisp-utils/actions/runs/10221026912/job/28282626459#step:6:34)" aria-label="Docs" tabindex="0" role="button" aria-expanded="false" aria-controls="mg-dropdown-33" class="mg-head">\n      <div class="mg-heading">\n        <span class="docs icon"></span>\n        <span class="label">Docs</span>\n      </div>\n      <span class="mg-arrow"></span>\n    </div>\n    <div id="mg-dropdown-33" role="region" aria-labelledby="mg-control-33" class="mg-dropdown">\n      <div class="mg-dropdown-label">Docs</div>\n      <ul class="mg-list">\n        \n        <li>\n          <a class="mg-link" href="https://openwisp.org/" aria-label="OpenWISP" tabindex="-1">\n            <span class="link icon"></span>\n            <span class="label">OpenWISP</span>\n          </a>\n        </li>\n        \n        <li>\n          <a class="mg-link" href="https://openwisp.org/thecode.html" aria-label="Code" tabindex="-1">\n            <span class="code icon"></span>\n            <span class="label">Code</span>\n          </a>\n        </li>\n        \n      </ul>\n    </div>\n  </div>\n    \n  <a class="menu-item" href="/admin/openwisp-system-info/" aria-label="System info">\n    <span class="ow-info-icon icon"></span>\n    <span class="label">System info</span>\n  </a>\n   \n</nav>\n\n      \n    </div>\n  \n  <div id="main-content" data-select2-id="main-content">\n    \n    <!-- Header -->\n    \n      <div id="header">\n        \n        \n          <div id="ow-user-tools">\n          \n  <div class="account">\n    <div title="account" id="account-btn" role="button" tabindex="0" aria-label="account button" aria-expanded="false" aria-controls="account-dropdown" class="account-button toggle-btn tooltip-sm">\n      <span class="user"></span>\n      <strong>Tester</strong>\n    </div>\n    <ul id="account-dropdown" role="list" aria-labelledby="account-btn" class="account-menu hide">\n      <li class="account-menu-username">\n        <span class="user"></span>\n        <strong>Tester</strong>\n      </li>\n      \n      <li>\n        <a tabindex="-1" href="/admin/password_change/">\n          <span class="password"></span>\n          Change password\n        </a>\n      </li>\n      \n      <li>\n        <a tabindex="-1" class="menu-link" href="/admin/logout/">\n          <span class="logout"></span>\n          Log out\n        </a>\n      </li>\n    </ul>\n  </div>\n\n          </div>\n        \n      </div>\n    \n    <!-- END Header -->\n    \n<div class="breadcrumbs">\n<a href="/admin/">Home</a>\n› <a href="/admin/test_project/">Test_Project</a>\n› Books\n</div>\n\n    \n    <div class="main" id="main" data-select2-id="main">\n      \n      <div class="content" data-select2-id="9">\n        \n          \n        \n        \n  \n    <div id="ow-changelist-filter">\n      <div class="filters-bottom" data-select2-id="8">\n        <span class="left-arrow force-inactive">\n          <img src="/static/ui/openwisp/images/left-arrow.svg" alt="left">\n        </span>\n        <span class="right-arrow force-inactive">\n          <img src="/static/ui/openwisp/images/right-arrow.svg" alt="right">\n        </span>\n        <div aria-label="changelist filters region" tabindex="0" class="ow-filter-slider">\n          <div class="slider" data-select2-id="7">\n            \n<div class="ow-filter auto-filter" data-select2-id="6">\n  \n\n    <h3> By shelf </h3>\n\n    <ul data-select2-id="5">\n        <li data-select2-id="4"><select name="shelf__id" data-dropdown-css-class="ow2-autocomplete-dropdown" data-empty-label="-" id="id-shelf__id-dal-filter" class="admin-autocomplete select2-hidden-accessible" data-ajax--cache="true" data-ajax--delay="250" data-ajax--type="GET" data-ajax--url="/admin/ow-auto-filter/" data-app-label="test_project" data-model-name="book" data-field-name="shelf" data-theme="admin-autocomplete" data-allow-clear="true" data-placeholder="" lang="en" data-select2-id="id-shelf__id-dal-filter" tabindex="-1" aria-hidden="true">\n  <option value="" data-select2-id="2"></option>\n\n</select><span class="select2 select2-container select2-container--admin-autocomplete select2-container--below select2-container--open select2-container--focus" dir="ltr" data-select2-id="1" style="width: 300px;"><span class="selection"><span class="select2-selection select2-selection--single" role="combobox" aria-haspopup="true" aria-expanded="true" tabindex="0" aria-disabled="false" aria-labelledby="select2-id-shelf__id-dal-filter-container" aria-owns="select2-id-shelf__id-dal-filter-results"><span class="select2-selection__rendered" id="select2-id-shelf__id-dal-filter-container" role="textbox" aria-readonly="true"><span class="select2-selection__placeholder">All</span></span><span class="select2-selection__arrow" role="presentation"><b role="presentation"></b></span></span></span><span class="dropdown-wrapper" aria-hidden="true"></span></span></li>\n        <li></li>\n    </ul>\n  <div class="auto-filter-choices" style="min-height: 14.75rem;"></div>\n</div>\n\n\n<div class="ow-filter name">\n  \n    \n      <div id="name" aria-label="by name" tabindex="0" aria-controls="choices-" role="button" aria-expanded="false" class="filter-title">\n        <h3>\n           By name \n        </h3>\n        <div class="selected-option" tabindex="0" title="All">\n          All\n        </div>\n      </div>\n    \n  \n    \n  \n    \n  \n  <div id="choices-name" role="region" aria-labelledby="name" class="filter-options">\n    <ul>\n    \n      <li>\n        <a class="selected" title="All" href="?">\n          All\n        </a>\n      </li>\n    \n      <li>\n        <a title="Book 1" href="?name=Book+1">\n          Book 1\n        </a>\n      </li>\n    \n      <li>\n        <a title="Book 2" href="?name=Book+2">\n          Book 2\n        </a>\n      </li>\n    \n    </ul>\n  </div>\n</div>\n\n          </div>\n        </div>\n      </div>\n      <div class="filters-top">\n        <h2>Filter</h2>\n        <div class="filters-control">\n          \n          \n        </div>\n      </div>\n    </div>\n  \n\n        <!-- Content -->\n        <div id="content" class="">\n          <div class="title-wrapper">\n            \n            <h1>Select book to change</h1>\n            \n            \n        <ul class="object-tools">\n          \n            \n\n\n  \n  <li>\n    \n    <a href="/admin/test_project/book/add/" class="addlink">\n      Add book\n    </a>\n  </li>\n  \n\n\n          \n        </ul>\n    \n          </div>\n          \n  <div id="content-main">\n    \n        <ul class="object-tools">\n          \n            \n\n\n  \n  <li>\n    \n    <a href="/admin/test_project/book/add/" class="addlink">\n      Add book\n    </a>\n  </li>\n  \n\n\n          \n        </ul>\n    \n    \n    <div class="module filtered" id="changelist">\n      <div class="changelist-form-container">\n        \n\n<div id="toolbar"><form id="changelist-search" method="get">\n<div><!-- DIV needed for valid HTML -->\n<label for="searchbar"><img src="/static/admin/img/search.svg" alt="Search"></label>\n<input type="text" size="40" name="q" value="" id="searchbar">\n<input type="submit" value="Search">\n\n\n</div>\n\n</form></div>\n\n\n          \n        <form id="changelist-form" method="post" novalidate=""><input type="hidden" name="csrfmiddlewaretoken" value="yVhu8j4FiqQLFulF602QRJpBNuNT0iHn4cAablwW0i9T8akZpgYcWfNvgahXTosv">\n        \n        \n          \n<div class="actions">\n  \n    \n    <label>Action: <select name="action" required="">\n  <option value="" selected="">---------</option>\n\n  <option value="delete_selected">Delete selected books</option>\n\n</select></label><input type="hidden" name="select_across" value="0" class="select-across">\n    \n    \n    <button type="submit" class="button" title="Run the selected action" name="index" value="0">Go</button>\n    \n    \n    \n        <span class="action-counter" data-actions-icnt="2">0 of 2 selected</span>\n        \n    \n    \n  \n</div>\n\n          \n\n\n<div class="results">\n<table id="result_list">\n<thead>\n<tr>\n\n<th scope="col" class="action-checkbox-column">\n   \n   <div class="text"><span><input type="checkbox" id="action-toggle"></span></div>\n   <div class="clear"></div>\n</th>\n<th scope="col" class="column-__str__">\n   \n   <div class="text"><span>Book</span></div>\n   <div class="clear"></div>\n</th>\n</tr>\n</thead>\n<tbody>\n\n\n<tr><td class="action-checkbox"><input type="checkbox" name="_selected_action" value="ada7a586-[36](https://github.com/openwisp/openwisp-utils/actions/runs/10221026912/job/28282626459#step:6:37)44-4f8a-8c1d-c4f808c78650" class="action-select"></td><th class="field-__str__"><a href="/admin/test_project/book/ada7a586-3644-4f8a-8c1d-c4f808c78650/change/">Book 1</a></th></tr>\n\n\n<tr><td class="action-checkbox"><input type="checkbox" name="_selected_action" value="65cd30a7-9ebf-4621-9890-5bc6ca4f6d75" class="action-select"></td><th class="field-__str__"><a href="/admin/test_project/book/65cd30a7-9ebf-4621-9890-5bc6ca4f6d75/change/">Book 2</a></th></tr>\n\n</tbody>\n</table>\n</div>\n\n\n          \n        \n        \n\n<p class="paginator">\n\n2 books\n\n\n</p>\n\n        </form>\n      </div>\n    </div>\n  </div>\n  \n\n          \n          <br class="clear">\n        </div>\n        <!-- END Content -->\n        \n<div id="ow-loading" class="ow-overlay">\n  <div class="ow-overlay-inner">\n    <div class="ow-loading-spinner"></div>\n  </div>\n</div>\n\n\n\n    <script src="/static/admin/js/menu.js"></script>\n\n\n    \n        <script src="/static/dummy.js"></script>\n    \n\n\n      </div>\n    </div>\n    </div>\n</div>\n<!-- END Container -->\n\n\n<span class="select2-container select2-container--admin-autocomplete select2-container--open" style="position: absolute; top: 154.297px; left: [37](https://github.com/openwisp/openwisp-utils/actions/runs/10221026912/job/28282626459#step:6:38)5.594px;"><span class="select2-dropdown ow2-autocomplete-dropdown select2-dropdown--below" dir="ltr" style="width: 222px;"><span class="select2-search select2-search--dropdown"><input class="select2-search__field" type="search" tabindex="0" autocomplete="off" autocorrect="off" autocapitalize="none" spellcheck="false" role="searchbox" aria-autocomplete="list" aria-controls="select2-id-shelf__id-dal-filter-results"></span><span class="select2-results"><ul class="select2-results__options" role="listbox" id="select2-id-shelf__id-dal-filter-results" aria-expanded="true" aria-hidden="false"><li class="select2-results__option loading-results" role="option" aria-disabled="true" data-select2-id="3">Searching…</li></ul></span></span></span></body></html>'

Steps To Reproduce
Run in a loop until it fails

#!/bin/bash

while true; do
    # Run the Selenium test command
    SELENIUM_HEADLESS=1 ./manage.py test test_project.tests.test_selenium.TestAutocompleteFilter.test_autocomplete_shelf_filter
    
    # Check the exit code of the command
    if [ $? -ne 0 ]; then
        # If the command returns a non-zero exit code, break the loop
        break
    fi
done

# Send the beep command
beep

Expected behavior
Test always passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[tests] Fixed flaky selenium test: test_autocomplete_shelf_filter youhaveme9/openwisp-utils
1 participant
@nemesifier