Add comprehensive test cases

[spwoodcock]

Is your feature request related to a problem? Please describe.

• osm-fieldwork has a lot of the core functionality needed in other tools (FMTM)
• There are some test cases here, but we need to cover a few more things that are important not to break.

Describe the solution you'd like

• Add more test cases. In particular starting with basemapper.py first, then additional modules.
• OdkCentral.py already has some good test cases, but could do with more, as it underpins FMTM.

[owolabioromidayo]

Hello!
I want to work on this. It would help me understand the codebase some more.

[donaldte]

greeting ! I will be contributing here

[valentina-buoro]

Hello 👋🏼 @spwoodcock , can I work on this issue too ?

[owolabioromidayo]

Hi @spwoodcock ,

I am writing tests for the make_data_extract.py and filter_data.py modules.

However, I am unable to create a database connection through the URI passed into the MakeDataExtract class which is passed into the PostgresClient class from osm-rawdata.

There are two outcomes from my testing:

1. I use "underpass" as the dbUri which configures the client to use https://api-prod.raw-data.hotosm.org/v1/ . This seems to be down and I can't establish a connection.

2. I use URIs based on the docker-compose configurations.
   

In this case, I get an error from the queryRemote method of the PostgresClient class which states that self.url does not exist.
This is because self.url is only declared when the dbUri is "underpass".
It seems the PostgresClient class only allows for either a localhost connection or a remote connection through underpass.

And localhost doesn't work for the docker-compose configuration.

How should I work around this?

[spwoodcock]

Hi @owolabioromidayo , thanks for the feedback.

Don't worry about updating the tests for make data extract, as it may be removed in future in favour of osm-rawdata.

As for filter data, you could update the code as part of a PR to make it configurable for connections other than localhost (so that testing can be carried out using the container based database).

[spwoodcock]

@valentina-buoro sure thing 😄

[robsavoye]

Currently all the real functionality of make_data_extract.py is now in osm-rawdata, so all this does now is filter data. So this would be a test for the FilterData() class only.

[owolabioromidayo]

Okay, got it.

The parse method of the FilterData class creates self.tags and self.keep attributes which have been commented out of usage in the cleanData method with a FIXME comment.

What should be done concerning that?

[robsavoye]

Ignore those. :-) Some of that is from past experiments, and may be brought back to life later, but commented out to reduce introducing weird errors. Eventually we'll clean that up, fixed and added or deleted.

[owolabioromidayo]

Okay :-)

I made a PR with 2 basic tests for filter_data.py now. I also fixed a tiny bug in it.
owolabioromidayo#3

Would appreciate any feedback.

[RonaldRonnie]

hi @spwoodcock , i would like to contribute and work on this issue. I request to be assigned to it. Surely i will continue to understand the entire code base

[spwoodcock]

Sure thing @RonaldRonnie 👍

[RonaldRonnie]

Hi @spwoodcock , am i advised to use pytest-mock while carrying out test???

[spwoodcock]

Hi!

While it's possible to do so, I don't think we need it.

We can use real data files where needed, then for OdkCentral.py we run the tests via docker compose so have access to the real API of ODK Central, meaning we can send requests and get real responses instead of having to mock 👍

[RonaldRonnie]

thank you @spwoodcock for the knowledge and guidance

[RonaldRonnie]

Hello @spwoodcock , I have made a PR onto this issue and I will grateful for the feedback.
#280