Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

column_name added as "null" when test modified #89

Open
JanPeterDatakind opened this issue May 17, 2023 · 3 comments · May be fixed by #109
Open

column_name added as "null" when test modified #89

JanPeterDatakind opened this issue May 17, 2023 · 3 comments · May be fixed by #109

Comments

@JanPeterDatakind
Copy link
Contributor

Describe the bug
If a test which takes parameters is edited through the UI and the column is left blank (intentionally), the resulting entry in the dot.configured_tests tables has its column_name set to "null" which breaks the execution of the dot run/ creation of test results

To Reproduce
Steps to reproduce the behavior on the demo ScanProject1 project:

  1. Go to test screen
  2. Edit test "duplicate flight record" and leave column name parameter empty
  3. Go to DBeaver/ DB management software of your choice and inspect the dot.configured_tests table
  4. See value "null" in the column_name column

Expected behavior
A clear and concise description of what you expected to happen.
Instead of adding "null", this field should be left empty in the DB if it has been left empty in the UI

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@bkowshik
Copy link

image

Edit test "duplicate flight record" and leave column name parameter empty.

Don't understand what is column name parameter referring to as I don't see column name for the Duplicate flight record test. Is this referring to the UUID column?

@bkowshik
Copy link

bkowshik commented Jan 4, 2025

Was able to replicate this scenario when adding a new test. The value of column_name is null for the new test.

select *
from dot.configured_tests
where test_type = 'expression_is_true';

test_activated|project_id  |test_id                             |scenario_id   |priority|description                                  |impact|proposed_remediation|entity_id          |test_type         |column_name|column_description|test_parameters                                                                                                                |date_added                   |date_modified                |last_updated_by|
--------------+------------+------------------------------------+--------------+--------+---------------------------------------------+------+--------------------+-------------------+------------------+-----------+------------------+-------------------------------------------------------------------------------------------------------------------------------+-----------------------------+-----------------------------+---------------+
true          |nsso_ayush  |842e0a0a-d746-3015-a998-8b9033e40755|INCONSISTENT-1|       8|Household size (hhsize) is a positive number |      |                    |all_nsso_ayush_data|expression_is_true|null       |                  |{"expression": "hhsize is not null and hhsize > 0"}                                                                            |2025-01-05 00:30:54.873 +0530|2025-01-05 00:30:54.873 +0530|false          |
true          |ScanProject1|368b65ac-b5e4-37a9-902d-5f385f94a9a0|INCONSISTENT-1|       8|Price is a positive number for direct flights|      |                    |all_flight_data    |expression_is_true|           |                  |{"name": "t_direct_flights_positive_price", "condition": "stops = 'non-stop'", "expression": "price is not null and price > 0"}|2025-01-02 09:12:42.207 +0530|2025-01-03 07:03:29.038 +0530|Lorenzo        |

@bkowshik
Copy link

bkowshik commented Jan 5, 2025

One learning I had today is that the null value is a string and not the traditional NULL value.

-- Query by the default NULL value does not return any rows.
SELECT * FROM dot.configured_tests WHERE column_name IS NULL;
test_activated|project_id|test_id|scenario_id|priority|description|impact|proposed_remediation|entity_id|test_type|column_name|column_description|test_parameters|date_added|date_modified|last_updated_by|
--------------+----------+-------+-----------+--------+-----------+------+--------------------+---------+---------+-----------+------------------+---------------+----------+-------------+---------------+


-- But, querying by the string 'null' value returns the new test.
SELECT * FROM dot.configured_tests WHERE column_name = 'null';
test_activated|project_id|test_id                             |scenario_id   |priority|description                           |impact|proposed_remediation|entity_id          |test_type |column_name|column_description|test_parameters                                                                                                 |date_added                   |date_modified                |last_updated_by|
--------------+----------+------------------------------------+--------------+--------+--------------------------------------+------+--------------------+-------------------+----------+-----------+------------------+----------------------------------------------------------------------------------------------------------------+-----------------------------+-----------------------------+---------------+
true          |nsso_ayush|768d80c6-136e-376a-b048-3d30f1c650f5|INCONSISTENT-1|       5|Household size is not too large either|      |                    |all_nsso_ayush_data|custom_sql|null       |                  |{"query": "SELECT hhsize, COUNT(*) samples\nFROM public.nsso_ayush\nWHERE hhsize > 20\nGROUP BY 1\nORDER BY 1;"}|2025-01-05 11:07:13.481 +0530|2025-01-05 11:07:13.481 +0530|false          |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants