Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return max 100 for Get All and add paged response #23

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

doingnz
Copy link
Contributor

@doingnz doingnz commented Feb 19, 2023

A long running Clima app will have many thousands of records. Trying to return them all in response to HTTP GET is not practical and will exhaust resources.

This change limits the number of rows that can be returned in one HTTP GET to 100. It adds a second HTTP endpoint that returns data in pages of 50 records. Arbitrarily chosen limits. Need to experiment on what are reasonable limits. Ideally in the future the web API would include headers to aid paging. Current Maple only supports a single URL parameter which is being used here to select the page.

URL for paged access is : http://192.168.2.112:5417/gettemperaturepage/1

Note records are returned with the most recent record first on page 1.

@doingnz
Copy link
Contributor Author

doingnz commented Feb 19, 2023

Note that Database initialisation does not recreate the database each time the application started. This is to enable ongoing testing with ever increasing database as would be expected in the real world. May neend a web endpoint to erase the data and certainly need an upper limit on the number of records stored in the SQLite DB.... How should one determine what is the upper limit on the number of rows that can be saved before storage is exhausted or other resources exhausted.

What is the practical bounding factor for upper limit of the rows in Meadow for each of the two projects ?

@adrianstevens adrianstevens changed the base branch from main to develop January 13, 2024 21:33
@adrianstevens
Copy link
Contributor

Thx as always @doingnz - I'll take a look and get this merged

@adrianstevens
Copy link
Contributor

@jorgedevs for viz

@doingnz
Copy link
Contributor Author

doingnz commented Jan 14, 2024

This is from way back before the folders/apps were restructured. Applies to the demo application that writes data to SQLite DB and makes a Maple server available to query the data where returning all records from a local SQLite DB.

I think there is a bigger question of what is going to be the reference application for Clima?

Currently the "root" level application is a simple test application to read the sensor data and not do anything with it. This is useful to test tha Clima Hardware works, but is very far from a reference weather station application nor reference Meadow IOT application. Meadow may have bits of IoT tech that work some of the time on an engineers bench, but I cannot get them to all work together as a working solution with OTA, IOT data logging etc. and gave up a long time ago on SQLite as there is no mechanism to read out the DB file when the wheels fall off and the DB is corrupt.

It is probably simpler to delete this PR as no longer relevant, and then draw up a specification for the Clima Weather solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants