Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] No truncation callout for Discover when query default record count isn't exactly 500 #8114

Open
Swiddis opened this issue Sep 10, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@Swiddis
Copy link
Contributor

Swiddis commented Sep 10, 2024

Describe the bug
Following this comment on #8060: Discover infinite scrolling queries are currently hardcoded to assume that for every available query language, running it with no limit configured will truncate the records after exactly 500 records. This is true for DQL and Lucene queries, but not for PPL or SQL, which have default limits of 200.

To Reproduce
Steps to reproduce the behavior:

  1. Create an index that has more than 500 records, using e.g. this docker-compose to generate data.
  2. Enable query enhancements
  3. Run a DQL query on the index -- notice that there's a truncated search callout indicating the records are truncated
image
  1. Do the same thing with a PPL query, where the default limit is 200
  2. Observe there's no such callout
image

Expected behavior
Depending on the selected language, there should be a callout for either 200 or 500 records as a default limit (possibly even pulling from a configured setting somewhere). It's maybe not sufficient to add the callout for 200 or 500 both with the same hardcoding, as this would lead to cases where it says it's truncated when more data is still being pulled (albeit only for some intermediate render steps).

It's worth noting that due to this hardcoded check, the callout also shows up if you run a SQL query with LIMIT 500 or PPL with HEAD 500 as the returned records matches the expected truncation threshold, when no truncation is occurring. These callouts ideally shouldn't show up if the query returns exactly 200 or 500 records without truncation (although knowing this ahead of time may be equivalent to the halting problem).

OpenSearch Version
2.16.0

Dashboards Version
Current main, pre-2.17 but post-2.16

Plugins
Observability, Reporting, Query Workbench. The SQL plugin on the backend is important.

Screenshots
See above.

Host/Environment (please complete the following information):

  • OS: Mac OS
  • Browser and version: Firefox 129.0.2

Additional context
N/A

@Swiddis Swiddis added bug Something isn't working untriaged labels Sep 10, 2024
@LDrago27 LDrago27 assigned LDrago27 and kavilla and unassigned LDrago27 Sep 17, 2024
@dblock dblock removed the untriaged label Sep 30, 2024
@dblock
Copy link
Member

dblock commented Sep 30, 2024

[Catch All Triage - 1, 2, 3, 4]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants