SIMSBIOHUB-648: Create Sampling Period Page #1446

mauberti-bc · 2024-12-05T03:22:01Z

Links to Jira Tickets

{Include a link to all applicable Jira tickets}

Description of Changes

The main goal of this PR is to separate the sampling site and sampling period CRUD operations. Previously sampling periods were created/updated as part of creating/updating a sampling site. Now, sampling periods have their own create/edit pages/components.

All of the changes are around that concept, handling all of the code that naturally broke as a result of the related database schema changes, and workflow changes, etc.

Testing Notes

New DB Schema

1 new migration
- See JSDoc notes in the migration.
1 new procedure
- Check constraint for the sample_period table.

Picture of the relevant tables, and how they look after running the migration:

New Functionality

Separate table/page for creating, editing, deleting sampling periods (via the Manage Sampling Information page).
- New endpoints/services/repositories/interfaces

Updated Functionality

Updated the 3 sampling columns in the Observations table. Behaviour is similar to before, but underlying data is different.
Updated the sampling side bar (from the Manage Observations page). Behaviour is similar to before, but underlying data is different.
Updated Analytics tab (below the map on the Survey page). Behaviour is similar to before, but underlying data is different.
Updated the sampling information portion of the existing Observation CSV import backend code. The rest of the Observation CSV import code should work as before.

github-actions · 2024-12-05T03:22:12Z

Openshift URLs for the PR Deployment:

codecov · 2024-12-05T03:23:29Z

Codecov Report

Attention: Patch coverage is 52.71318% with 488 lines in your changes missing coverage. Please review.

Project coverage is 46.39%. Comparing base (7344aa4) to head (5a10cc9).

Files with missing lines	Patch %	Lines
...lds/AutocompleteSearch/AutocompleteSearchField.tsx	0.00%	61 Missing ⚠️
api/src/repositories/sample-period-repository.ts	27.84%	53 Missing and 4 partials ⚠️
api/src/services/observation-services/utils.ts	64.61%	39 Missing and 7 partials ⚠️
...formation/periods/SamplePeriodDataGridEditCell.tsx	2.32%	42 Missing ⚠️
api/src/repositories/analytics-repository.ts	41.93%	32 Missing and 4 partials ⚠️
api/src/repositories/technique-repository.ts	2.77%	35 Missing ⚠️
...g-information/sites/SampleSiteDataGridEditCell.tsx	0.00%	32 Missing ⚠️
...ts/data-grid/taxonomy/TaxonomyDataGridEditCell.tsx	0.00%	31 Missing ⚠️
...s/sample-site-repository/sample-site-repository.ts	76.47%	14 Missing and 6 partials ⚠️
...autocomplete/AsyncAutocompleteDataGridEditCell.tsx	0.00%	19 Missing ⚠️
... and 28 more

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1446      +/-   ##
==========================================
+ Coverage   46.07%   46.39%   +0.32%     
==========================================
  Files         910      913       +3     
  Lines       23796    24080     +284     
  Branches     3512     3604      +92     
==========================================
+ Hits        10963    11171     +208     
- Misses      12215    12275      +60     
- Partials      618      634      +16

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

NickPhura · 2024-12-18T18:51:47Z

A few things I noticed:

When creating a period, if you select a site, and then remove the period from it, the validation is probably failing because the form won't submit, but there is no error message shown.

When editing an existing sample period, I can't select or change the sample sites.

Nit: I can save a period with a start and end date that are exactly the same (with no time).

If I pick the same times, then the save stops working, though there is no error message about why.

I have 1 sampling site, 1 technique, and 3 periods. The techniques aren't being combined in the observations table when adding a record:

mauberti-bc · 2024-12-18T22:43:18Z

I disallowed changing the site because at that point you may as well create a new period for the correct site and delete the wrong one. Seems along the lines of turning a skateboard into a car instead of just buying a car. Maybe worth allowing though?
We do want to allow a period that starts and stops on the same day with no time, in case the time is not known. I agree this seems off but is realistic data. (The duration column is null for those periods because there isn't an obvious way to represent the unknown duration)

Fix some api tests. Remove some deprecated code.

mauberti-bc · 2025-01-07T00:05:07Z

There's a failing request when expanding a sampling site in the left side panel of the observations page. Otherwise haven't found any issues

Fix some existing req.query parsing logic.

…ompleteSearchField.tsx.

Remove unused api param from delete observations endpoint. Minor tweak to find endpoint filter type (allow them to be undefined as they were meant to be)

- Rename/refactor sample method migrations to come after the upgrade postgres migration.

MacQSL · 2025-01-17T18:35:09Z

Heres some screenshots of some of the issues I found:

When deleting the sampling site from the "Manage Observations" page it throws a failed to execute SQL error: column "survey_sample_site_id" does not exist
The sampling techniques table renders the wrong page when the rows overflow into the next page. (renders page 2 vs rendering page 1) when 11 sampling techniques http://localhost:7100/admin/projects/2/surveys/4/sampling
The survey page Sampling Information (Technique tab) says 11 records but renders nothing on page 2 http://localhost:7100/admin/projects/2/surveys/4/details

- Fix techniques pagination (on both survey technique page, and manage sampling information page). - Fix delete sample sites sql error. - Update APIError -> HTTPError code.

NickPhura · 2025-01-17T20:38:12Z

Heres some screenshots of some of the issues I found:

Pushed up some changes that should address those 3 issues.

database/src/procedures/tr_before_survey_sample_period_check_minimum_data_requirements.ts

api/src/repositories/sample-site-repository/utils.ts

api/src/services/geo-service.ts

…le (instead of separate trigger).

app/src/features/surveys/sampling-information/techniques/edit/EditTechniquePage.tsx

app/src/features/surveys/sampling-information/techniques/create/CreateTechniquePage.tsx

database/src/migrations/20250117000001_survey_sample_period_check_constraints.ts

Fix error message.

Add unit tests for sample period matching functions.

Add unit tests

sonarqubecloud · 2025-01-20T21:39:48Z

Quality Gate failed

Failed conditions
8.9% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

MacQSL · 2025-01-20T22:21:49Z

api/src/services/import-services/utils/datetime.ts

@@ -21,15 +22,15 @@ interface ICaptureStub {
 * @returns {string | undefined}
 */
 export const formatTimeString = (time?: string | null): string | undefined => {
-  const fullTime = dayjs(time, 'HH:mm:ss');
+  const fullTime = dayjs(time, DefaultTimeFormat);
  const shortTime = dayjs(time, 'HH:mm');


nit: This can be updated to the new DefaultTimeFormatNoSeconds

mauberti-bc added 7 commits December 4, 2024 11:06

sampling period table on manage page

86a5d9b

remove periods tab from sampling technique table

7220297

add periods

788948a

sampling period create page

6688295

remove methods from creating sample site workflow

bf39de1

api changes

997235e

fix test

3c06e67

mauberti-bc added the Early Feedback Welcome PR is not finished, but early review feedback is welcomed label Dec 5, 2024

mauberti-bc and others added 5 commits December 6, 2024 15:53

Merge branch 'dev' of github.com:bcgov/biohubbc into SIMSBIOHUB-648

e4869c8

edit sampling period

346b31c

Merge branch 'dev' into SIMSBIOHUB-648

ef6d51c

move sample-method/{methodId}/sample-period api to survey/sample-period

4082c92

fix update endpoint

0f3c204

mauberti-bc added Ready For Review PR is ready for review and removed Early Feedback Welcome PR is not finished, but early review feedback is welcomed labels Dec 10, 2024

add missing backend tests

c377633

mauberti-bc marked this pull request as ready for review December 12, 2024 20:40

Merge remote-tracking branch 'origin/dev' into SIMSBIOHUB-648

d812318

NickPhura added 7 commits December 20, 2024 16:40

Create/Edit Sampling Period forms working.

b62e870

WIP: save point

0c9416e

Manage observation Sampling controls working

075d358

Refactor sampling information utils

9031a40

Fix analytics backend.

185e6a8

Fix some api tests. Remove some deprecated code.

Remove some deprecated code.

1d41b79

Update most sampling "location" to sampling "site"

c002d2f

NickPhura added 2 commits January 13, 2025 13:03

Add endpoint unit tests.

c577490

Fix some existing req.query parsing logic.

Add prop to automatically run the search when using the generic Autoc…

9678292

…ompleteSearchField.tsx.

NickPhura requested review from NickPhura and MacQSL January 13, 2025 21:21

NickPhura added 6 commits January 13, 2025 16:31

Add endpoint unit tests.

3397082

Remove unused api param from delete observations endpoint. Minor tweak to find endpoint filter type (allow them to be undefined as they were meant to be)

Merge remote-tracking branch 'origin/dev' into SIMSBIOHUB-648

620554e

- Add new migration to drop n8n schema.

669477f

- Rename/refactor sample method migrations to come after the upgrade postgres migration.

ignore-skip

4c907ae

Fix bug in findTechniques repo function

8f6adf1

Enhancements

6a4f6ed

NickPhura mentioned this pull request Jan 17, 2025

SIMSBIOHUB-647: Create Observations Page #1444

Open

Address PR feedback

f41d843

- Fix techniques pagination (on both survey technique page, and manage sampling information page). - Fix delete sample sites sql error. - Update APIError -> HTTPError code.