Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HPCC4J-618 WIP Add WsECL testsuite #719

Open
wants to merge 1 commit into
base: candidate-9.6.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -486,12 +486,27 @@ private void setProtocol(String protocol_)
* @param port_
* the new port
*/
private void setPort(String port_)
public void setPort(String port_)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpmcmu I'm making this change so we can copy the wseclwatch connection, and simply edit the port to the wsecl port.
I'm not sure if we'd be better off creating a brand new connection based off of the eclwatch connection

@asselitx ignore this change

{
if (port_ != null && port_.length() > 0)
boolean hasChanged = false;

if (port_ != null && !port_.isEmpty())
{
if (port != null && !port.equals(port_))
hasChanged = true;

port = port_;
}
else
{
if (port != null && !port.isEmpty())
hasChanged = true;

port = "";
}

if(baseUrl != null && !baseUrl.isEmpty() && hasChanged)
constructUrl();
}

/**
Expand All @@ -516,7 +531,7 @@ private void setURIPath(String path)
/**
* Construct url.
*/
private void constructUrl()
public void constructUrl()
{
baseUrl = new StringBuffer();
baseUrl.append(protocol).append(protDelimiter);
Expand Down
181 changes: 181 additions & 0 deletions wsclient/src/test/java/org/hpccsystems/ws/client/WSECLTests.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
/*##############################################################################
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asselitx this is the wsecl test suite, any wsecl related tests would be hosted here


HPCC SYSTEMS software Copyright (C) 2024 HPCC Systems®.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
############################################################################## */

package org.hpccsystems.ws.client;

import static org.junit.Assume.assumeTrue;

import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.util.List;

import org.hpccsystems.ws.client.utils.Connection;
import org.hpccsystems.ws.client.wrappers.gen.wsworkunits.WUPublishWorkunitResponseWrapper;
import org.hpccsystems.ws.client.wrappers.wsworkunits.QueryResultWrapper;
import org.hpccsystems.ws.client.wrappers.wsworkunits.WorkunitWrapper;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;

public class WSECLTests extends BaseRemoteTest
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BaseRemoteTests initializes wseclwatch connection, based on optional env vars

{
private static HPCCWsWorkUnitsClient wswuclient = null;
private static boolean hasPublishedQuery = false;
private final static String wsECLPort = System.getProperty("wseclport", "8002");
private final static String eclScriptName = "SimpleFunction.ecl";
private static Connection wseclConn = null;

@BeforeClass
public static void setup() throws Exception
{
wswuclient = wsclient.getWsWorkunitsClient(); //for publishing queries
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this method runs before the tests.
Here we try to publish an ecl query, then confirm it was actually published, and setup the wsecl connection

Assert.assertNotNull(wswuclient);

try
{
InputStream resourceStream = BaseRemoteTest.class.getClassLoader().getResourceAsStream(eclScriptName);
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();

byte[] buffer = new byte[4096];
int bytesRead = resourceStream.read(buffer);
while (bytesRead > -1) {
byteArrayOutputStream.write(buffer, 0, bytesRead);
bytesRead = resourceStream.read(buffer);
}

byte[] eclData = byteArrayOutputStream.toByteArray();
String ecl = new String(eclData, "UTF-8");

WorkunitWrapper wu = new WorkunitWrapper();
wu.setECL(ecl);
wu.setJobname(eclScriptName);
wu.setCluster(thorclustername);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My experience has been with queries deployed to roxies, so this question may be naive. Are we able to test all the sample XML, XSD and WSDL features by targeting thor? I may need a quick explanation or pointer to relevant docs explaining how thor handles soap features if its different from roxie.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current assumption is that the target engine doesn't affect the presence/absence of the wsecl features for req/resp/xsd/wsdl. But targeting thor is not necessary, it was just convenient. We can target Roxie if preferred/necessary but you'll need to target a configurable (or discoverable) roxie name.


//wswuclient.createAndRunWUFromECLAndGetResults(wu);
WUPublishWorkunitResponseWrapper resp = wswuclient.publishWUFromEclWrapped(wu);
System.out.println("Finished publishing query" + resp.toString());
List<QueryResultWrapper> queries = wswuclient.listQueries(wu.getWuid(), wu.getJobname(), wu.getCluster(), null, null, null, null, null, null);
for (QueryResultWrapper query : queries)
{
if (query.getName().equalsIgnoreCase(eclScriptName))
{
hasPublishedQuery = true;
return;
}
}
}
catch (Exception e)
{
System.out.println("Could not publish ECL query: " + e.getLocalizedMessage());
}

wseclConn = wsclient.getConnection();
wseclConn.setPort(wsECLPort);
wseclConn.setSocketTimeoutMilli(15000);
wseclConn.setWriteTimeoutMilli(15000);
}

@Test
public void testWsECLGetWSDL()
{
assumeTrue("WsECL connection not available", wseclConn != null);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We ignore the test if any of these conditions fail

Copy link
Collaborator

@asselitx asselitx Jul 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might there be a way to common-up these 'assumeTrue' pre-checks, or always run one test with them first that all other tests are contingent upon? After finishing the review, this might not be as important as I thought. It looks like I won't be creating multiple new functions that need to follow the same pattern, rather filling in the meat of these four here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ofcourse, we can create a new method with all necessary prereq tests and manually call it directly before each test, or simply annotate it as
"BeforeEach":

https://stackoverflow.com/questions/63819472/how-can-i-use-the-beforeeach-method-for-testing-in-java

assumeTrue("WsECL connection port appears invalid", wseclConn.getPort().equals(wsECLPort));
assumeTrue("Cannot test WsECL published query WSDL feature without published queries!", hasPublishedQuery);

try
{
String wsdlURI = "/WsEcl/definitions/query/" + thorclustername + "/" + eclScriptName + "/main/" + eclScriptName + ".wsdl";

String wsdlResponse = wseclConn.sendGetRequest(wsdlURI);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simple http get request based on the wsecl connection setup above, and the target URI constructed here.

Assert.assertNotNull("Unexpected Null response", wsdlResponse);
//TODO determine good way to confirm success/failure
//Assert.assertArrayEquals(expectedWsdlResponse, wsdlResponse)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can add expected responses under tests/resources and compare here

}
catch (Exception e)
{
Assert.fail("Could not fetch WsECL query wsdl: " + e.getLocalizedMessage());
}
}

@Test
public void testWsECLGetSampleReq()
{
assumeTrue("WsECL connection not available", wseclConn != null);
assumeTrue("WsECL connection port appears invalid", wseclConn.getPort().equals(wsECLPort));
assumeTrue("Cannot test WsECL published query WSDL feature without published queries!", hasPublishedQuery);

try
{
//http://127.0.0.1:8002/WsEcl/example/request/query/<roxie>/<query>
String sampleReqURI = "/WsEcl/example/request/query" + thorclustername + "/" + eclScriptName;
String sampleReqResponse = wseclConn.sendGetRequest(sampleReqURI);
Assert.assertNotNull("Unexpected Null response", sampleReqResponse);
//TODO determine good way to confirm success/failure
//Assert.assertArrayEquals(expectedWsdlResponse, wsdlResponse)
}
catch (Exception e)
{
Assert.fail("Could not fetch WsECL query wsdl: " + e.getLocalizedMessage());
}
}

@Test
public void testWsECLGetSampleResp()
{
assumeTrue("WsECL connection not available", wseclConn != null);
assumeTrue("WsECL connection port appears invalid", wseclConn.getPort().equals(wsECLPort));
assumeTrue("Cannot test WsECL published query WSDL feature without published queries!", hasPublishedQuery);

try
{
//http://127.0.0.1:8002/WsEcl/example/response/query/<roxie>/<query>
String sampleRespURI = "/WsEcl/example/response/query" + thorclustername + "/" + eclScriptName;
String sampleRespResponse = wseclConn.sendGetRequest(sampleRespURI);
Assert.assertNotNull("Unexpected Null response", sampleRespResponse);
//TODO determine good way to confirm success/failure
//Assert.assertArrayEquals(expectedWsdlResponse, wsdlResponse)
}
catch (Exception e)
{
Assert.fail("Could not fetch WsECL query wsdl: " + e.getLocalizedMessage());
}
}

@Test
public void testWsECLGetSchema()
{
assumeTrue("WsECL connection not available", wseclConn != null);
assumeTrue("WsECL connection port appears invalid", wseclConn.getPort().equals(wsECLPort));
assumeTrue("Cannot test WsECL published query WSDL feature without published queries!", hasPublishedQuery);

try
{
String xsdURI = "/WsEcl/definitions/query/" + thorclustername + "/" + eclScriptName + "/main/" + eclScriptName + ".xsd";
String xsdResponse = wseclConn.sendGetRequest(xsdURI);
Assert.assertNotNull("Unexpected Null response", xsdResponse);
//TODO determine good way to confirm success/failure
//Assert.assertArrayEquals(expectedWsdlResponse, wsdlResponse)
}
catch (Exception e)
{
Assert.fail("Could not fetch WsECL query xsd: " + e.getLocalizedMessage());
}
}

}
3 changes: 3 additions & 0 deletions wsclient/src/test/resources/RoxieEchoPersonInfo.ecl
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import $.esdl_example;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

abandoning this ecl query since it requires esdl_example, opting for the simple ecl query below

request := dataset([], esdl_example.t_RoxieEchoPersonInfoRequest) : stored('RoxieEchoPersonInfoRequest', few);
output(request, named('RoxieEchoPersonInfoResponse'));
19 changes: 19 additions & 0 deletions wsclient/src/test/resources/SimpleFunction.ecl
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
MyFunc(STRING DataIn, STRING1 SearchChar) := FUNCTION
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the content of the query matters, we can figure out how to publish RoxieEchoPersonInfo instead.


StrLen := LENGTH(TRIM(dataIn));
ds := DATASET([{DataIn}], {STRING chars});

OutRec := RECORD
UNSIGNED1 flag;
END;

OutRec Xform(ds L, INTEGER C) := TRANSFORM
SELF.flag := IF(L.chars[C] = SearchChar, 1, 0);
END;

n := NORMALIZE(ds, StrLen, Xform(LEFT, COUNTER));

RETURN COUNT(n(flag=1));
END;

OUTPUT(MyFunc('abc~xyz~def~fred', '~'));
Loading