Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NDNts-aux snapshot #6

Merged
merged 15 commits into from
Oct 8, 2024
Merged

NDNts-aux snapshot #6

merged 15 commits into from
Oct 8, 2024

Conversation

CAWorks-ChrisA
Copy link
Contributor

Includes snapshot program parts 1,2,3 and some leftover debug code (when all testing is good will remove before merge)

  • Part 1 (yjsAdaptor): Check the state vector for the snapshot interval, and when the interval is up, publish and update the persistent storage on the blind fetch pointer of the latest snapshot.

  • Part 2 (syncAgent + namespace): An additional checker for the snapshot keyword for the existing syncAgent endpoint, and respond by checking persistent storage with the name as the key.

  • Detailed explanation of how this is achieved is in the slides and the doc attached below.
  • If needed, will move it to a separate function or a separate class/endpoint.
  • Part 3 (yjsAdaptor): A callback registered to the syncAgent to handle the topic 'snapshot'. When a member receives a snapshot via blobs, it checks it against its persistent storage blind fetch pointer. If the received snapshot is newer, it will override the pointer with the snapshot it just received.
  • Again, this is a rough overview, please consult the slides / docs

Lastly, a minor debug code is added to fetchBlob(), because I found that under heavier loads it is prone to lag and therefore cause problems to the synchronization process.

This pull request is meant to be a work-in-progress, as there are many debug codes to remove and unsafe calls to fix.

Attached are the slides and doc for documentation.
https://docs.google.com/presentation/d/15q8wTChtQtKzaAA1anvDJa1ZNr1DS1Kc67kW6XYghzM/edit?usp=sharing
https://docs.google.com/document/d/1s4pnJS4vSAP_EJD-P0XP6J5osoz8nNgEfLs-HOCDL6U/edit?usp=sharing

@zjkmxy zjkmxy self-assigned this Aug 14, 2024
@zjkmxy
Copy link
Collaborator

zjkmxy commented Aug 14, 2024

Copy link
Collaborator

@zjkmxy zjkmxy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically good to me

src/adaptors/yjs-ndn-adaptor.ts Outdated Show resolved Hide resolved
src/adaptors/yjs-ndn-adaptor.ts Outdated Show resolved Hide resolved
src/adaptors/yjs-ndn-adaptor.ts Show resolved Hide resolved
src/sync-agent/namespace.ts Outdated Show resolved Hide resolved
@CAWorks-ChrisA CAWorks-ChrisA marked this pull request as ready for review September 28, 2024 22:44
src/adaptors/yjs-ndn-adaptor.ts Outdated Show resolved Hide resolved
src/adaptors/yjs-ndn-adaptor.ts Outdated Show resolved Hide resolved
src/adaptors/yjs-ndn-adaptor.ts Outdated Show resolved Hide resolved
src/adaptors/yjs-ndn-adaptor.ts Outdated Show resolved Hide resolved
src/adaptors/yjs-ndn-adaptor.ts Show resolved Hide resolved
src/adaptors/yjs-ndn-adaptor.ts Show resolved Hide resolved
src/adaptors/yjs-ndn-adaptor.ts Show resolved Hide resolved
Copy link
Collaborator

@zjkmxy zjkmxy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove other Adam Chen Injection XXX as well. These comments do not explain the purpose of the code and I think is unnecessary.

Copy link
Collaborator

@zjkmxy zjkmxy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now. Let me test this weekend and merge

@zjkmxy zjkmxy merged commit edcff4c into UCLA-IRL:main Oct 8, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants