Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added global fetchSize variables to DAOs #877

Merged
merged 3 commits into from
Dec 17, 2024

Conversation

zack-rma
Copy link
Collaborator

Added global fetchSize variables to JooqDao and appropriate DAOs

Copy link
Contributor

@MikeNeilson MikeNeilson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent idea. Some thoughts on naming but not enough to assert a denial if others dissagree with me.

@@ -69,6 +68,8 @@
public abstract class JooqDao<T> extends Dao<T> {
protected static final int ORACLE_CURSOR_TYPE = -10;
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
public static final int FETCH_SIZE = 1000;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this should be DEFAULT_FETCH_SIZE and DEFAULT_SMALL_FETCH_SIZE or the like.

Over time we will likely find that we need to fine tune this for different queries though this is a solid baseline, especially over the apparent default of 10.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If wanting to tune this, I recommend that this variable be changed to public static getDefaultFetchSize method that supports a System property override if System property changes are easily done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt we'd want to do a global override of the fetch size. If it's tweaked, it's going to be per query based on that queries specific need.

@@ -69,6 +68,8 @@
public abstract class JooqDao<T> extends Dao<T> {
protected static final int ORACLE_CURSOR_TYPE = -10;
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
public static final int FETCH_SIZE = 1000;
public static final int HALF_FETCH_SIZE = FETCH_SIZE / 2;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldnt code this as a half fetch size but instead as Mike called out above. SMALL and set it to 500. I think the relationship between the two values in the code is arbitrary.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the naming and set the small fetch size to 500

@adamkorynta
Copy link
Collaborator

The default size can get set via properties on the connection pool. See connectionProperties https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html

@MikeNeilson
Copy link
Contributor

I'd have to check in more detail, but there are two pool classes in Tomcat by default and I believe we use the other one. May have a similar setting though.

@zack-rma zack-rma requested a review from MikeNeilson November 26, 2024 23:12
Copy link
Contributor

@MikeNeilson MikeNeilson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent improvement of the software in general. Thanks for taking the time to address performance fixes with the effort of related work and making an appropriate refactor that improves performance and clarity.

@zack-rma zack-rma requested a review from adamkorynta December 12, 2024 16:48
@adamkorynta adamkorynta merged commit dc4e4fe into USACE:develop Dec 17, 2024
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants