Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HPCC4J-542 DFSClient: Create JUnit for read retry #648

Merged
merged 2 commits into from
Oct 13, 2023

Conversation

jpmcmu
Copy link
Contributor

@jpmcmu jpmcmu commented Oct 11, 2023

  • Added file part failure retry test
  • Fixed retry issue on initial connection

Signed-off-by: James McMullan [email protected]

Type of change:

  • This change is a bug fix (non-breaking change which fixes an issue).
  • This change is a new feature (non-breaking change which adds functionality).
  • This change is a breaking change (fix or feature that will cause existing behavior to change).

Checklist:

  • I have created a corresponding JIRA ticket for this submission
  • My code follows the code style of this project.
    • I have applied the Eclipse code-format template provided.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly, or...
    • I have created a JIRA ticket to update the documentation.
    • Any new interfaces or exported functions are appropriately commented.
  • I have read the HPCC Systems CONTRIBUTORS document (https://github.com/hpcc-systems/HPCC-Platform/wiki/Guide-for-contributors).
  • The change has been fully tested:
    • This change does not cause any existing JUnits to fail.
    • I have include JUnit coverage to test this change
    • I have performed system test and covered possible regressions and side effects.
  • I have given due consideration to all of the following potential concerns:
    • Scalability
    • Performance
    • Security
    • Thread-safety
    • Premature optimization
    • This change fixes the problem, not just the symptom

Testing:

@jpmcmu jpmcmu requested a review from rpastrana October 11, 2023 13:39
- Added file part failure retry test
- Fixed retry issue on initial connection

Signed-off-by: James McMullan [email protected]
@jpmcmu
Copy link
Contributor Author

jpmcmu commented Oct 12, 2023

@rpastrana squashed and pushed up code review changes. The first block of the test is the original test that had resulted in the crash

Copy link
Member

@rpastrana rpastrana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpmcmu a few comments

*/
public void setCopyIP(int copyIndex, String copyIP)
{
if (copyIndex < 0 || copyIndex >= copyLocations.length) return;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's move the return statement to the next line, for optics and future maintenance, but more importantly, do we need to throw in this situation?

{
List<String> copyLocationsList = new ArrayList<>(Arrays.asList(copyLocations));
copyLocationsList.add(index, copyIP);
copyLocations = copyLocationsList.toArray(new String[0]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems a little bit extraneous, not sure what type copyLocations is, but are we not able to append the copyIP directly? and if not, are we using the most appropriate datastructure to track copyIPs, copypaths?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, the existing type is a simple array; I had considered moving this to a List but other parts of the code are expecting an array, so it would require the internal list be converted to an Array at those locations. I thought it would be better to do the conversion here in the add() function because this unlikely to be used outside of test cases.

*/
public void add(int index, String copyIP, String copyPath)
{
List<String> copyLocationsList = new ArrayList<>(Arrays.asList(copyLocations));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if either index, copyIP, or copyPath are invalid, we should throw

try
{
if (getUseSSL())
log.debug("Attempting to connect to file part : '" + dataPart.getThisPart() + "' Copy: '"
+ (getFilePartCopy() + 1) + "' on IP: '" + getIP() + "'");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worth reporting the filename and filepart path as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, we don't have access to the filename at this point, but I can log the file path.

// So we don't care as much about individual packet latency or connection time overhead
sock.setPerformancePreferences(0, 1, 2);
sock.connect(new InetSocketAddress(this.getIP(), this.dataPart.getPort()), this.connectTimeout);
throw new HpccFileException("Bad file part addr " + this.getIP(), e);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this be due to invalid path as well?

Copy link
Member

@rpastrana rpastrana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpmcmu approved

@rpastrana rpastrana merged commit 905b095 into hpcc-systems:candidate-9.4.x Oct 13, 2023
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants