Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'migrate' task to support migration to the specmesh way #170

Open
bluemonk3y opened this issue Jul 20, 2023 · 0 comments
Open

Add 'migrate' task to support migration to the specmesh way #170

bluemonk3y opened this issue Jul 20, 2023 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@bluemonk3y
Copy link
Collaborator

Automation tasks should be used to help existing installations migrate to SpecMesh.

This could look like:

  1. User write simple-app with proposed domain id 'somedomain-id'
  • channel names are completed using the proposed naming. i.e. _public.somedata
  • channels are annotated with migrate-source tags.

i.e.

<snip>
  _protected.retail.subway.food.purchase:
    bindings:
      kafka:
    publish:
      tags: [
        name: "migrate-source: existing-topic.blah-stuff.withmessy-name"
      ]
<snip>
  1. User applies extract command to the spec and an existing cluster (should be production)
    simple-template.app is enriched with the topic config, and schema. Schema file is downloaded to the /schema folder.
<snip>
  _protected.retail.subway.food.purchase:
    bindings:
      kafka:
        partitions: 10
        configs:
          cleanup.policy: delete
          retention.ms: 9900000
    publish:
      tags: [
        name: "migrate-source: existing-topic.blah-stuff.withmessy-name"
      ]
        payload:
          $ref: "/schema/simple.schema_demo._public.user_info.proto"
<snip>
  1. User applies migrate task. This task will
  • create a topic (if it doesn't exist) and configure it with associated settings
  • copy topic offsets from the old to new topic
  • copy topic data from the old topic to the new topic
  1. Client apps should be pointed to the new topic. They should also use the new domain-id/credentaisls so that ACLs are enforced
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants