Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

guide: update comparing exps #4651

Merged
merged 7 commits into from
Jun 30, 2023
Merged

guide: update comparing exps #4651

merged 7 commits into from
Jun 30, 2023

Conversation

dberenbaum
Copy link
Collaborator

Updates to the guide for comparing experiments.

Main points:

  • Focus on high-level workflow of how to review, compare, and manage experiments
  • Add VS Code examples
  • Deemphasize detailing lots of different CLI options

@shcheklein shcheklein temporarily deployed to dvc-org-exp-compare-1pcs47e2en June 22, 2023 21:52 Inactive
@github-actions
Copy link
Contributor

github-actions bot commented Jun 22, 2023

Link Check Report

There were no links to check!

@dberenbaum dberenbaum marked this pull request as ready for review June 26, 2023 15:59
@dberenbaum dberenbaum requested a review from a team as a code owner June 26, 2023 15:59
@shcheklein shcheklein temporarily deployed to dvc-org-exp-compare-1pcs47e2en June 26, 2023 15:59 Inactive
@dberenbaum dberenbaum requested a review from a team June 26, 2023 15:59
@shcheklein shcheklein had a problem deploying to dvc-org-exp-compare-1pcs47e2en June 26, 2023 15:59 Failure
}
}
```
## Compare plots
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feels like it should be more prominent tbh, part of the compare functionality ... I wonder if should include some screenshots/previews at least.

Also, is it the right title for end users - "compare plots"? We are still comparing metrics, right? Just in a different format. What is the easiest for people to understand?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feels like it should be more prominent tbh, part of the compare functionality ... I wonder if should include some screenshots/previews at least.

Added a more expansive section. PTAL.

I didn't want to repeat https://dvc.org/doc/user-guide/experiment-management/visualizing-plots#comparing-plots, but I think it's better even if it's slightly repetitive.

Also, is it the right title for end users - "compare plots"? We are still comparing metrics, right? Just in a different format. What is the easiest for people to understand?

🤔 I think it's easy for people to understand regardless of the title. Plots aren't a novel concept and neither is comparing them as part of experiment tracking, so no strong opinion from me, but I doubt it's too important.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plots aren't a novel concept and neither is comparing them as part of experiment tracking, so no strong opinion from me, but I doubt it's too important.

Yep, I think my question is more about even - do we need a separate section at all for this? (since it's nothing special - should it be just part of the main section, w/o subtitles, etc).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, makes sense. I don't quite have a great idea for how to reorganize it to bring it together with the table, but open to ideas.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, nw, it's not a blocker at all...

@shcheklein shcheklein temporarily deployed to dvc-org-exp-compare-1pcs47e2en June 26, 2023 18:03 Inactive
Copy link
Contributor

@daavoo daavoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are Studio tabs planned as a followup?

@shcheklein shcheklein temporarily deployed to dvc-org-exp-compare-1pcs47e2en June 28, 2023 18:21 Inactive
@dberenbaum
Copy link
Collaborator Author

Are Studio tabs planned as a followup?

Good question. I didn't plan on it, but I now added a section here where I mention it. Since we have guides for sharing experiments and comparing in Studio, I'm not sure I want to cover it in depth, and I'm also not sure I want it as another "tab" since otherwise this guide is focused on the local experience and the need for Studio feels like a slightly different use. We also have more space to explain these differences compared to get started.

@daavoo
Copy link
Contributor

daavoo commented Jun 30, 2023

Good question. I didn't plan on it, but I now added a section here where I mention it. Since we have guides for sharing experiments and comparing in Studio, I'm not sure I want to cover it in depth, and I'm also not sure I want it as another "tab" since otherwise this guide is focused on the local experience and the need for Studio feels like a slightly different use. We also have more space to explain these differences compared to get started.

No strong opinion, I am just not sure whether it helps or hurts to have separate comparing experiments in the Studio guide (i.e. https://dvc-org-exp-compare-1pcs47e2en.herokuapp.com/doc/studio/user-guide/projects-and-experiments/visualize-and-compare).

@dberenbaum
Copy link
Collaborator Author

No strong opinion, I am just not sure whether it helps or hurts to have separate comparing experiments in the Studio guide

It's linked here. Do you think we should consolidate them into one page? Let's follow up and discuss how we want to structure the dvc/studio content, especially with the upcoming changes of dvcx.

@dberenbaum
Copy link
Collaborator Author

Merging but happy to keep making improvements as follow ups.

@shcheklein shcheklein temporarily deployed to dvc-org-exp-compare-1pcs47e2en June 30, 2023 13:28 Inactive
@dberenbaum dberenbaum merged commit e5ff7d1 into main Jun 30, 2023
4 checks passed
@dberenbaum dberenbaum deleted the exp-compare branch June 30, 2023 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants