Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature(parallized deploy): allow configuration of the deployer to run tasks in parallel (per leaf node) #74

Closed
wants to merge 2 commits into from
Closed

feature(parallized deploy): allow configuration of the deployer to run tasks in parallel (per leaf node) #74

wants to merge 2 commits into from

Conversation

dakaneye
Copy link

Summary of Changes

  • Configuration: Adds a deployThreads configuration.
  • MavenRepositoryDeployer: establishes this class as a Singleton
  • MavenRepositoryDeploymentCallable: for a given leaf node, this does the deployment, returning a new DeploymentResult object to collect for the overall summary

@dakaneye dakaneye force-pushed the parallelized-deploys branch 2 times, most recently from 01a7658 to bfeb2f6 Compare October 19, 2024 22:17
@mosabua
Copy link
Member

mosabua commented Oct 20, 2024

Please change commit messages to follow https://trino.io/development/process#pull-request-and-commit-guidelines-

mosabua
mosabua requested changes Oct 20, 2024
View reviewed changes
Copy link
Member

@mosabua mosabua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a first look .. once its compiles I can do some testing ..

.gitignore
@@ -10,3 +10,4 @@ release.properties
local-cache
maven-repository-provisioner.log
.checkstyle
**/*.iml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isnt that covered by the .idea thing already?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm not 100% sure how it ended up in git tracking, but I had one generated when I tried to add some checkstyle plugins to my intellij.

...n-repository-provisioner/src/main/java/com/simpligility/maven/provisioner/Configuration.java Outdated Show resolved Hide resolved
...ovisioner/src/test/java/com/simpligility/maven/provisioner/helpers/RepositorySystemImpl.java Outdated Show resolved Hide resolved
...ovisioner/src/test/java/com/simpligility/maven/provisioner/helpers/RepositorySystemImpl.java Show resolved Hide resolved
...ovisioner/src/test/java/com/simpligility/maven/provisioner/helpers/RepositorySystemImpl.java Outdated Show resolved Hide resolved
...ovisioner/src/test/java/com/simpligility/maven/provisioner/helpers/RepositorySystemImpl.java Outdated Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
...n-repository-provisioner/src/main/java/com/simpligility/maven/provisioner/Configuration.java Show resolved Hide resolved
.../src/test/java/com/simpligility/maven/provisioner/MavenRepositoryDeploymentCallableTest.java Show resolved Hide resolved
...ovisioner/src/test/java/com/simpligility/maven/provisioner/helpers/RepositorySystemImpl.java
@@ -0,0 +1,120 @@
package com.simpligility.maven.provisioner.helpers;

import org.eclipse.aether.RepositorySystem;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should really move to the newer version of aether or the maven resolver replacement .. but that can follow later cc @cstamas

@dakaneye dakaneye force-pushed the parallelized-deploys branch 3 times, most recently from e1dbb79 to 3ca7165 Compare October 20, 2024 05:15
@dakaneye
Apply some whitespace and line-wrapping changes
a77cc0c 
This change uses the Intellij auto-formatter to format whitespace
and line wrapping according to the checkstyle guidelines laid out
in the maven-repository-provisioner/src/conf/maven_checks.xml

Signed-off-by: Samuel Dacanay <[email protected]>
@dakaneye
Run the MavenDeployer in threads-per-leaf node
6fb0809 
This change allows for the configuration of the provisioner/deployer
with a thread count. It makes the MavenRepositoryDeployer a singleton
and copies the leaf-node deployment code into a Callable class.

Signed-off-by: Samuel Dacanay <[email protected]>
@cstamas
Copy link
Contributor

cstamas commented Oct 22, 2024

Folks, unsure what you want to achieve, but parallel deploy in Resolver 1.9+ is a thing. All you need is to "batch" the deploy request. See https://issues.apache.org/jira/browse/MRESOLVER-319

@mosabua
Copy link
Member

mosabua commented Oct 22, 2024

Folks, unsure what you want to achieve, but parallel deploy in Resolver 1.9+ is a thing. All you need is to "batch" the deploy request. See https://issues.apache.org/jira/browse/MRESOLVER-319

I vaguely remember now .. but yes.. for that to be used we have to upgrade this project to the new resolver stuff .. which arguably would be good.

@mosabua mosabua closed this Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mosabua mosabua mosabua requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dakaneye @mosabua @cstamas