You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Brought up by @AndreGodinho7 in the Delta User's slack channel.
When we are migrating from Table A (call it "last stable") and we need to move to the new "stable" table, while not "interrupting" the streams until we are "ready". If there is a new set of "columns" then those can get picked up by the streaming applications without problems. However, if there is a completely re-written table structure, and we need to use a new "uuid" - then the change is more complex. We want to shoot for less complicated.
Consider that there are live customers "streaming data customers"
You want to create a "new" table to "replace" the old "tableA"
You want to ensure there is a "backup plan" - so you intercept the "table" and "redirect"
This is not a simple established pattern. It is a full "table aliasing" concept.
talk about ways to get this to work (with Trino for example - you can "redirect" tables to new locations) - with managed tables and a table UUID that changes with "Delta" - we will need a more "seamless" means to handle the redirection.
Include an official writeup (blog post) to go along with the pattern.
The text was updated successfully, but these errors were encountered:
Brought up by @AndreGodinho7 in the Delta User's slack channel.
When we are migrating from Table A (call it "last stable") and we need to move to the new "stable" table, while not "interrupting" the streams until we are "ready". If there is a new set of "columns" then those can get picked up by the streaming applications without problems. However, if there is a completely re-written table structure, and we need to use a new "uuid" - then the change is more complex. We want to shoot for less complicated.
This is not a simple established pattern. It is a full "table aliasing" concept.
talk about ways to get this to work (with Trino for example - you can "redirect" tables to new locations) - with managed tables and a table UUID that changes with "Delta" - we will need a more "seamless" means to handle the redirection.
Include an official writeup (blog post) to go along with the pattern.
The text was updated successfully, but these errors were encountered: