Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][catalog] Streampark start job ship catalogstore plugin and database crud. #3987

Open
wants to merge 4 commits into
base: dev
Choose a base branch
from

Conversation

Mrart
Copy link
Contributor

@Mrart Mrart commented Aug 22, 2024

What changes were proposed in this pull request

Issue Number: close #3946

Brief change log

  1. ship catalogstore jar in shims
  2. add database curd.

Verifying this change

This change added tests and can be verified as follows:

Does this pull request potentially affect one of the following parts

  • Dependencies (does it add or upgrade a dependency): (yes )

pom.xml Show resolved Hide resolved
@ChengJie1053
Copy link
Member

There are some bugs please fix

@Mrart
Copy link
Contributor Author

Mrart commented Aug 29, 2024

I have add unit test, Waiting Ci

ChengJie1053
ChengJie1053 previously approved these changes Sep 2, 2024
Copy link
Member

@ChengJie1053 ChengJie1053 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wolfboys
Copy link
Member

wolfboys commented Sep 5, 2024

Hi:

User history job uses Flink's default in-memory catalog. How can we migrate or upgrade these existing jobs to use the catalog?

@Mrart Mrart force-pushed the 3946 branch 2 times, most recently from 41cb3e9 to 0b52ffe Compare September 5, 2024 08:40
@Mrart
Copy link
Contributor Author

Mrart commented Sep 5, 2024

Hi:

User history job uses Flink's default in-memory catalog. How can we migrate or upgrade these existing jobs to use the catalog?

We do not have the problem of historical job migration, because we do not save the catalog now, the catalog in memory is used in the sql of each job, each time the job is started, a new catalog will be created, if the migration needs to register in the catalogstore first, Remove the job sql to create catalog.

@Mrart Mrart force-pushed the 3946 branch 2 times, most recently from 21eff12 to a0b5d7d Compare September 6, 2024 05:24
@Mrart
Copy link
Contributor Author

Mrart commented Sep 10, 2024

@wolfboys @SbloodyS Can you help review this pr?

@Mrart
Copy link
Contributor Author

Mrart commented Sep 11, 2024

image image add Test test pic

Copy link

sonarcloud bot commented Sep 11, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
4.2% Duplication on New Code (required ≤ 3%)

See analysis details on SonarCloud

@Mrart Mrart requested a review from wolfboys September 11, 2024 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature][catalog] Streampark start job ship catalogstore plugin.
3 participants