Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Airbyte metadata ingestion pipeline - filtering the internal ID instead of pipeline name #18224

Open
SChalupaBraiins opened this issue Oct 10, 2024 · 0 comments

Comments

@SChalupaBraiins
Copy link

Affected module
Airbyte connector or/and Ingestion Framework

Describe the bug
The Pipeline Filter (Include, Exclude) in Airbyte Metadata Ingestion does filter the Airbyte internal IDs, not the pipeline name (connection name in Airbyte terminology).
This is useless, internal IDs cannot be defined by regexps and changes on every environments or re-deploy.

To Reproduce

Filtering by Pipeline names does not work.
Filtering works by ID strings like 'd3720495-8668-432d-b5cf-cc350b94af2b', you can extract them from Airbyte connection URL

Expected behavior
Using Airbyte pipeline name for filtering is necessary. The name is retrieved by Metadata Ingestion already, OpenMetadata use it as the pipeline name in UI.

Version:

  • OS: Linux
  • Python version:
  • OpenMetadata version: 1.5.5
  • OpenMetadata Ingestion package version: docker.getcollate.io/openmetadata/ingestion:1.5.5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant