Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming Deck Squash #6162

Closed
wants to merge 1 commit into from
Closed

Streaming Deck Squash #6162

wants to merge 1 commit into from

Conversation

Future-Outlier
Copy link
Member

@Future-Outlier Future-Outlier commented Jan 13, 2025

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

  • added: For new features.
  • changed: For changes in existing functionality.
  • deprecated: For soon-to-be-removed features.
  • removed: For features being removed.
  • fixed: For any bug fixed.
  • security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Summary by Bito

Implementation of enhanced deck URI handling and real-time deck functionality in Flyte, including DeckUri support in node execution transformers and a new DeckStatus system. The changes improve deck URI handling during task execution while maintaining backward compatibility with older Flytekit versions. The implementation introduces robust deck file existence checks and enhanced deck generation capabilities.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 2

Signed-off-by: Future-Outlier <[email protected]>
@flyte-bot
Copy link
Collaborator

flyte-bot commented Jan 13, 2025

Code Review Agent Run #00f3fb

Actionable Suggestions - 2
  • flytepropeller/pkg/controller/nodes/task/handler.go - 2
    • Consider complete OutputInfo field initialization · Line 193-195
    • Consider adding nil check for metadata · Line 450-450
Review Details
  • Files reviewed - 3 · Commit Range: e984f6a..e984f6a
    • flyteadmin/pkg/repositories/transformers/node_execution.go
    • flyteadmin/pkg/repositories/transformers/node_execution_test.go
    • flytepropeller/pkg/controller/nodes/task/handler.go
  • Files skipped - 0
  • Tools
    • Golangci-lint (Linter) - ✖︎ Failed
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo

Copy link

codecov bot commented Jan 13, 2025

Codecov Report

Attention: Patch coverage is 39.75904% with 50 lines in your changes missing coverage. Please review.

Project coverage is 37.01%. Comparing base (b8fb68d) to head (e984f6a).

Files with missing lines Patch % Lines
...lytepropeller/pkg/controller/nodes/task/handler.go 39.02% 40 Missing and 10 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6162      +/-   ##
==========================================
- Coverage   37.02%   37.01%   -0.01%     
==========================================
  Files        1317     1317              
  Lines      132534   132591      +57     
==========================================
+ Hits        49067    49085      +18     
- Misses      79221    79252      +31     
- Partials     4246     4254       +8     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.25% <100.00%> (+0.02%) ⬆️
unittests-flytecopilot 30.99% <ø> (ø)
unittests-flytectl 62.29% <ø> (-0.05%) ⬇️
unittests-flyteidl 7.23% <ø> (ø)
unittests-flyteplugins 53.85% <ø> (ø)
unittests-flytepropeller 42.61% <39.02%> (-0.03%) ⬇️
unittests-flytestdlib 55.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@flyte-bot
Copy link
Collaborator

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Feature Improvement - Enhanced Deck URI Handling and Real-time Deck Support

node_execution.go - Added DeckUri support in node execution transformers

node_execution_test.go - Added test cases for DeckUri functionality

handler.go - Implemented real-time deck functionality with backward compatibility

Comment on lines +193 to +195
p.execInfo.OutputInfo = &handler.OutputInfo{
OutputURI: outputPath,
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider complete OutputInfo field initialization

Consider initializing OutputInfo with both OutputURI and DeckURI fields consistently. The current implementation only sets OutputURI which may lead to inconsistent state if DeckURI was previously set.

Code suggestion
Check the AI-generated fix before applying
Suggested change
p.execInfo.OutputInfo = &handler.OutputInfo{
OutputURI: outputPath,
}
p.execInfo.OutputInfo = &handler.OutputInfo{
OutputURI: outputPath,
DeckURI: nil,
}

Code Review Run #00f3fb


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

return DeckUnknown, regErrors.Wrapf(err, "failed to read task template")
}

if template.GetMetadata().GetGeneratesDeck() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding nil check for metadata

Consider adding error handling for the case when GetMetadata() returns nil. Currently, if template.GetMetadata() returns nil, the code will panic when calling GetGeneratesDeck().

Code suggestion
Check the AI-generated fix before applying
Suggested change
if template.GetMetadata().GetGeneratesDeck() {
metadata := template.GetMetadata()
if metadata == nil {
return DeckUnknown, nil
}
if metadata.GetGeneratesDeck() {

Code Review Run #00f3fb


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants