Skip to content
This repository has been archived by the owner on Oct 24, 2023. It is now read-only.

Unable to launch Airflow DBT tutorial using Docker #27

Open
cbuffett opened this issue Apr 13, 2022 · 0 comments
Open

Unable to launch Airflow DBT tutorial using Docker #27

cbuffett opened this issue Apr 13, 2022 · 0 comments

Comments

@cbuffett
Copy link

cbuffett commented Apr 13, 2022

Out of the box, there appear to be several issues with running the Airflow DBT examples using Docker.

  1. The 1.0.x release of dbt should be installed using pip install dbt-core or pip install dbt-<connector>, e.g., pip install dbt-postgres. This applies both to the Dockerfile and to requirements.txt
  2. Dependency resolution complains about a few packages, ultimately resulting in Airflow failing to start up. I've been able to resolve this by pinning dbt-postgres<1.0.0, wtforms==2.3.3 , and werkzeug<1.0.0 in requirements.txt, which is required by Airflow 1.10.9.
  3. The dbt_project.yml is missing the config-version: 2 setting, which prevents DAGs from executing
  4. There is a typo in airflow/ge_tutorials_dag_with_great_expectations.py, though it looks like Fix syntax error in ge_tutorials_dag_with_great_expectations.py #16 addresses this
webserver_1     | Traceback (most recent call last):
webserver_1     |   File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 243, in process_file
webserver_1     |     m = imp.load_source(mod_name, filepath)
webserver_1     |   File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
webserver_1     |     module = _load(spec)
webserver_1     |   File "<frozen importlib._bootstrap>", line 696, in _load
webserver_1     |   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
webserver_1     |   File "<frozen importlib._bootstrap_external>", line 724, in exec_module
webserver_1     |   File "<frozen importlib._bootstrap_external>", line 860, in get_code
webserver_1     |   File "<frozen importlib._bootstrap_external>", line 791, in source_to_code
webserver_1     |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
webserver_1     |   File "/usr/local/airflow/dags/ge_tutorials_dag_with_great_expectations.py", line 21
webserver_1     |     "owner":` "Airflow",
webserver_1     |             ^
webserver_1     | SyntaxError: invalid syntax```
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant