🐛 Source OpenWeather: Update openweather onecall api to 3.0 #16136

pierre-borckmans · 2022-08-30T17:22:51Z

What

The openweather onecall api only works with the 3.0 version of the api, unless I'm missing something as explained here:
https://openweathermap.org/price

How

This PR simply changes the base openweather URL to use the 3.0 api

CLAassistant · 2022-08-30T17:22:56Z

All committers have signed the CLA.

marcosmarxm · 2022-08-30T20:12:18Z

@pierre-borckmans please update the Dockerfile version and the doc too

marcosmarxm · 2022-08-30T20:12:45Z

/test connector=connectors/source-openweather

pierre-borckmans · 2022-08-31T08:33:35Z

@marcosmarxm Done 👍

pierre-borckmans · 2022-08-31T08:38:31Z

@marcosmarxm More details can be found here, which seems to confirm my observations about OpenWeahter deprecating the 2.5 API for the OneCall endpoint:

https://community.home-assistant.io/t/cannot-setup-openweathermap/450817
home-assistant/core#76877

TL;DR: When creating new API keys (which I did when testing this), only the 3.0 API is supported.

marcosmarxm · 2022-08-31T17:20:01Z

/test connector=connectors/source-openweather

🕑 connectors/source-openweather https://github.com/airbytehq/airbyte/actions/runs/2965779311
❌ connectors/source-openweather https://github.com/airbytehq/airbyte/actions/runs/2965779311
🐛 https://gradle.com/s/rj63ky2o3clmy

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestSpec::test_additional_properties_is_true[inputs0] - ...
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contai...
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
FAILED test_incremental.py::TestIncremental::test_read_sequential_slices[inputs0]
FAILED test_incremental.py::TestIncremental::test_state_with_abnormally_large_values[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestFullRefresh.test_sequential_reads because not found in the config
=================== 6 failed, 22 passed, 1 skipped in 25.80s ===================

pierre-borckmans · 2022-08-31T17:48:43Z

@marcosmarxm I see one of the tests fails because of ***"type": "TRACE", "trace": ***"type": "ERROR", "emitted_at": 1661967159125.733, "error": ***"message": "Invalid API key. Please see http://openweathermap.org/faq#error401 for more info.", "internal_message": "401 Client Error: Unauthorized for url: https://api.openweathermap.org/data/3.0/onecall?appid=****&lat=45.7733258&lon=4.8398336&units=standard"

Possibly the app id that is used for this test is an old one, which still works with the 2.5 API and not with the 3.0.
When creating a new API key (with a newly registered user), it will work with the 3.0 API, and not with the 2.5.

What do you think is the best way to move forward?

vincentkoc · 2022-09-04T12:14:15Z

/test connector=connectors/source-openweather

marcosmarxm · 2022-09-05T20:22:03Z

/test connector=connectors/source-openweather

🕑 connectors/source-openweather https://github.com/airbytehq/airbyte/actions/runs/2995852781
❌ connectors/source-openweather https://github.com/airbytehq/airbyte/actions/runs/2995852781
🐛 https://gradle.com/s/cibmdvh6yat2q

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestSpec::test_additional_properties_is_true[inputs0] - ...
FAILED test_core.py::TestConnection::test_check[inputs0] - AssertionError: as...
FAILED test_core.py::TestBasicRead::test_read[inputs0] - docker.errors.Contai...
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
FAILED test_incremental.py::TestIncremental::test_read_sequential_slices[inputs0]
FAILED test_incremental.py::TestIncremental::test_state_with_abnormally_large_values[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestFullRefresh.test_sequential_reads because not found in the config
=================== 6 failed, 22 passed, 1 skipped in 26.16s ===================

marcosmarxm · 2022-09-05T22:07:54Z

/test connector=connectors/source-openweather

🕑 connectors/source-openweather https://github.com/airbytehq/airbyte/actions/runs/2996268234
✅ connectors/source-openweather https://github.com/airbytehq/airbyte/actions/runs/2996268234
Python tests coverage:

Name                                      Stmts   Miss  Cover
-------------------------------------------------------------
source_openweather/source.py                 22      0   100%
source_openweather/extra_validations.py      22      0   100%
source_openweather/__init__.py                2      0   100%
source_openweather/streams.py                33      5    85%
-------------------------------------------------------------
TOTAL                                        79      5    94%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       121     25    79%   21-23, 29-31, 36-43, 48-61, 208-216
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       110     48    56%   23-26, 32, 36, 39-64, 67-69, 72-74, 77-79, 82-84, 87-89, 92-110, 144-146
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1322    463    65%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestFullRefresh.test_sequential_reads because not found in the config
======================== 28 passed, 1 skipped in 30.48s ========================

marcosmarxm · 2022-09-05T22:20:53Z

/publish connector=connectors/source-openweather

🕑 Publishing the following connectors:
connectors/source-openweather
https://github.com/airbytehq/airbyte/actions/runs/2996316062

Connector	Did it publish?	Were definitions generated?
connectors/source-openweather	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…q#16136) * Update openweather onecall api to 3.0 * Bump version and update docs * correct backwards compatibility spec * auto-bump connector version [ci skip] Co-authored-by: marcosmarxm <[email protected]> Co-authored-by: Octavia Squidington III <[email protected]>

github-actions bot added the area/connectors Connector related issues label Aug 30, 2022

octavia-squidington-iii added bounty community connectors/source/openweather labels Aug 30, 2022

github-actions bot added the area/documentation Improvements or additions to documentation label Aug 31, 2022

pierre-borckmans changed the title ~~Update openweather onecall api to 3.0~~ 🐛 Source OpenWeather: Update openweather onecall api to 3.0 Aug 31, 2022

pierre-borckmans added 2 commits August 31, 2022 13:19

Update openweather onecall api to 3.0

35b1a7a

Bump version and update docs

a06f70f

marcosmarxm force-pushed the patch-2 branch from b036e14 to a06f70f Compare August 31, 2022 17:19

pierre-borckmans added 2 commits September 2, 2022 17:55

Merge branch 'master' into patch-2

13a2352

Merge branch 'master' into patch-2

61cd20f

correct backwards compatibility spec

6153bc4

marcosmarxm approved these changes Sep 5, 2022

View reviewed changes

octavia-squidington-iii and others added 2 commits September 5, 2022 22:45

auto-bump connector version [ci skip]

16c36eb

Merge branch 'master' into patch-2

50633f9

marcosmarxm merged commit ebd0f30 into airbytehq:master Sep 6, 2022

pierre-borckmans deleted the patch-2 branch September 6, 2022 13:43

sajarin added internal and removed bounty labels Sep 6, 2022

octavia-squidington-iii mentioned this pull request Sep 12, 2022

Bump Airbyte version from 0.40.4 to 0.40.5 #16610

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Source OpenWeather: Update openweather onecall api to 3.0 #16136

🐛 Source OpenWeather: Update openweather onecall api to 3.0 #16136

pierre-borckmans commented Aug 30, 2022

CLAassistant commented Aug 30, 2022 •

edited

Loading

marcosmarxm commented Aug 30, 2022

marcosmarxm commented Aug 30, 2022

pierre-borckmans commented Aug 31, 2022

pierre-borckmans commented Aug 31, 2022 •

edited

Loading

marcosmarxm commented Aug 31, 2022 •

edited by github-actions bot

Loading

pierre-borckmans commented Aug 31, 2022 •

edited

Loading

vincentkoc commented Sep 4, 2022

marcosmarxm commented Sep 5, 2022 •

edited by github-actions bot

Loading

marcosmarxm commented Sep 5, 2022 •

edited by github-actions bot

Loading

marcosmarxm commented Sep 5, 2022 •

edited by github-actions bot

Loading

🐛 Source OpenWeather: Update openweather onecall api to 3.0 #16136

🐛 Source OpenWeather: Update openweather onecall api to 3.0 #16136

Conversation

pierre-borckmans commented Aug 30, 2022

What

How

CLAassistant commented Aug 30, 2022 • edited Loading

marcosmarxm commented Aug 30, 2022

marcosmarxm commented Aug 30, 2022

pierre-borckmans commented Aug 31, 2022

pierre-borckmans commented Aug 31, 2022 • edited Loading

marcosmarxm commented Aug 31, 2022 • edited by github-actions bot Loading

Build Failed

pierre-borckmans commented Aug 31, 2022 • edited Loading

vincentkoc commented Sep 4, 2022

marcosmarxm commented Sep 5, 2022 • edited by github-actions bot Loading

Build Failed

marcosmarxm commented Sep 5, 2022 • edited by github-actions bot Loading

Build Passed

marcosmarxm commented Sep 5, 2022 • edited by github-actions bot Loading

CLAassistant commented Aug 30, 2022 •

edited

Loading

pierre-borckmans commented Aug 31, 2022 •

edited

Loading

marcosmarxm commented Aug 31, 2022 •

edited by github-actions bot

Loading

pierre-borckmans commented Aug 31, 2022 •

edited

Loading

marcosmarxm commented Sep 5, 2022 •

edited by github-actions bot

Loading

marcosmarxm commented Sep 5, 2022 •

edited by github-actions bot

Loading

marcosmarxm commented Sep 5, 2022 •

edited by github-actions bot

Loading