Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DynamoDB Integration #30

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

dheerajoruganty
Copy link
Collaborator

Description

This PR delivers end-to-end functionality to integrate DynamoDB into the FMBench Orchestrator, enabling persistent logging of run data directly within a DynamoDB table. From initial setup to full integration, the orchestrator now logs each run in a structured, persistent, and queryable format, ensuring data retention and traceability.

Summary of Changes

  • DynamoDB Table Creation and Management:
    • Introduced the get_or_create_table method, which checks for an existing DynamoDB table or creates a new one if absent.
    • Configured the table with a composite primary key, using run_uid as the partition key and timestamp as the sort key, allowing multiple entries per run_uid to capture each run's stages and updates over time.
    • Enabled PAY_PER_REQUEST billing to handle varying workloads without pre-set capacity.

Testing and Validation

  • Functional Testing: Validated the DynamoDB table creation and retrieval under various scenarios, including successful retrieval of an existing table, creating a new table, and handling IAM permission errors gracefully.
  • Integration Testing: Confirmed that FMBench can log runs with run_uid and timestamp values, successfully writing data to DynamoDB

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@dheerajoruganty dheerajoruganty changed the title Complete DynamoDB Integration for Persistent Logging in FMBench Orchestrator DynamoDB Integration for Persistent Logging in FMBench Orchestrator Nov 11, 2024
@dheerajoruganty dheerajoruganty changed the title DynamoDB Integration for Persistent Logging in FMBench Orchestrator DynamoDB Integration in FMBench Orchestrator Nov 12, 2024
@dheerajoruganty dheerajoruganty changed the title DynamoDB Integration in FMBench Orchestrator DynamoDB Integration Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant