KNOX-3058: Avoid 404 When Topology Is Being Redeployed

[pzampino]

What changes were proposed in this pull request?

Modified error handling when a topology is being redeployed, such that the response is not HTTP 404 'Not Found', but rather HTTP 503 'Service Unavailable'. The 503 response is much more likely to be retried by clients than is a 404 response.

Added a Jetty ErrorHandler that checks whether or not the topology being requested is in a Set marked as inactive. Topology names are add to this inactive set when the associated topology is deactivated, and removed from this set when the topology is reactivated. In the case of topology deletion, the topology is marked as inactive, but then removed from the inactive set because we know it's being deleted.

How was this patch tested?

I deployed a test topology with a demo LDAP provider and the Knox Token service.

I then ran the following script with 'https://localhost:8443/gateway/demo/knoxtoken/api/v2/token' and one of the demo LDAP username/pwd combinations, and piped the output to a file. This script outputs only the HTTP response status code for each invocation.

#!/bin/sh
#
#

ENDPOINT=$1
echo "Endpoint: $ENDPOINT"

if [ ! -z "$2" ] ; then
  USER=$2
fi

if [ ! -z "$3" ] ; then
  PWD=$3
fi

for i in {1..100000}
do
  curl -o /dev/null -s -w "%{http_code}\n" -ku ${USER}:${PWD} ${ENDPOINT}
done

Example:
~/bin/resp-test.sh 'https://localhost:8443/gateway/demo/knoxtoken/api/v2/token' sam sam-password > ~/response-code-test.txt &
While this script is running, I copied the test topology to the conf/topologies directory, then subsequently "touched" the test topology to trigger redeployment many times over several minutes. Finally, I deleted the test topology.

Following this, I reviewed the output to verify that there were initially 404 responses, followed by some 503 responses while the topology was being activated. Then, I verified that there were no additional 404 responses until that time at which I deleted the topology. I also verified the periodic 503 responses which are expected, and the normal 200 responses in between.

[lmccay]

Are we able to add a test for this?
See my other comments as well.

[moresandeep]

Just a thought, can we specify in the error message that the topology is inactive? This will be helpful in cases where we are debugging and knox the request failed because topology was inactive and not due to 500 error.

[pzampino]

We could certainly log that fact, but I think the error messages (e.g., Service Unavailable) are pretty standard. Given the separation between status code and error message though, I could be persuaded otherwise.

[lmccay]

LGTM
+1