Concurrency issue using DefaultHttpFetcher: Invalid use of SingleClientConnManager

[GoogleCodeExporter]

What steps will reproduce the problem?

Use ab to generate 2 (or more) concurrent post requests against the 
example-consumer application:
ab -n 2 -c 2 -p post_data -T 'application/x-www-form-urlencoded' 
http://localhost:8080/lso2/
where post_data contains the following:
openid=yourdomain.com&stage=discovery

What is the expected output? What do you see instead?

Caused by: java.lang.IllegalStateException: Invalid use of 
SingleClientConnManager: connection still allocated.
Make sure to release the connection before allocating another one.
    at org.apache.http.impl.conn.SingleClientConnManager.getConnection(SingleClientConnManager.java:199)
    at org.apache.http.impl.conn.SingleClientConnManager$1.getConnection(SingleClientConnManager.java:173)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:356)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
    at com.google.step2.http.DefaultHttpFetcher.fetch(DefaultHttpFetcher.java:76)
    at com.google.step2.discovery.UrlHostMetaFetcher.getHostMeta(UrlHostMetaFetcher.java:58)


What version of the product are you using? On what operating system?
Latest snapshot

Please provide any additional information below.
The problem appears to stem from the use of DefaultHttpClient in the default 
fetcher. The httpclient docs indicate the DefaultHttpClient is threadsafe: 
http://hc.apache.org/httpcomponents-client-dev/tutorial/html/httpagent.html

However, other information suggests using ThreadSafeClientConnManager 
http://stackoverflow.com/questions/1281219/best-practice-to-use-httpclient-in-mu
ltithreaded-environment

Updating to the latest httpclient and httpcore libraries didn’t resolve the 
problem. Switching to ThreadSafeClientConnManager resolves the issue.  I have 
an HttpFetcher implementation which I can contribute.

Original issue reported on code.google.com by [email protected] on 17 Apr 2011 at 7:49

[GoogleCodeExporter]

I am experiencing the same problems, is there any solutions/fix for this issue?

This is my stack trace:


SEVERE: Servlet.service() for servlet jsp threw exception
java.lang.IllegalStateException: Invalid use of SingleClientConnManager: 
connection still allocated.
Make sure to release the connection before allocating another one.
        at org.apache.http.impl.conn.SingleClientConnManager.getConnection(SingleClientConnManager.java:199)
        at org.apache.http.impl.conn.SingleClientConnManager$1.getConnection(SingleClientConnManager.java:173)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:356)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
        at com.google.step2.http.DefaultHttpFetcher.fetch(DefaultHttpFetcher.java:76)
        at com.google.step2.discovery.UrlHostMetaFetcher.getHostMeta(UrlHostMetaFetcher.java:58)
        at com.google.step2.discovery.Discovery2.discoverOpEndpointsForSite(Discovery2.java:190)
        at com.google.step2.discovery.Discovery2$1.newStyleDiscovery(Discovery2.java:116)
        at com.google.step2.discovery.Discovery2$1.newStyleDiscovery(Discovery2.java:111)
        at com.google.step2.discovery.Discovery2$FallbackDiscovery.get(Discovery2.java:411)
        at com.google.step2.discovery.Discovery2.discover(Discovery2.java:345)
        at com.google.step2.AuthRequestHelper.getDiscoveryInformation(AuthRequestHelper.java:115)
        at com.google.step2.AuthRequestHelper.generateRequest(AuthRequestHelper.java:216)

Thanks in advance.

Original comment by [email protected] on 29 Feb 2012 at 9:40

[GoogleCodeExporter]

This problem has just bitten us too:

@400000004f83e4f51ef65e64 Caused by: java.lang.IllegalStateException: Invalid 
use of SingleClientConnManager: connection still allocated.
@400000004f83e4f51ef6624c Make sure to release the connection before allocating 
another one.
@400000004f83e4f51ef66634       at 
org.apache.http.impl.conn.SingleClientConnManager.getConnection(SingleClientConn
Manager.java:199)
@400000004f83e4f51ef66634       at 
org.apache.http.impl.conn.SingleClientConnManager$1.getConnection(SingleClientCo
nnManager.java:173)
@400000004f83e4f51ef66a1c       at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirecto
r.java:356)
@400000004f83e4f51ef66e04       at 
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:6
41)
@400000004f83e4f51ef675d4       at 
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:5
76)
@400000004f83e4f51ef679bc       at 
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:5
54)
@400000004f83e4f51ef679bc       at 
com.google.step2.http.DefaultHttpFetcher.fetch(DefaultHttpFetcher.java:76)
@400000004f83e4f51ef67da4       at 
com.google.step2.discovery.UrlHostMetaFetcher.getHostMeta(UrlHostMetaFetcher.jav
a:58)
@400000004f83e4f51ef6818c       at 
com.google.step2.discovery.ParallelHostMetaFetcher$FetcherThread.call(ParallelHo
stMetaFetcher.java:103)
@400000004f83e4f51ef68574       at 
com.google.step2.discovery.ParallelHostMetaFetcher$FetcherThread.call(ParallelHo
stMetaFetcher.java:92)
@400000004f83e4f51ef6895c       at 
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
@400000004f83e4f51ef6895c       at 
java.util.concurrent.FutureTask.run(FutureTask.java:138)
@400000004f83e4f51ef68d44       at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
@400000004f83e4f51ef6912c       at 
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
@400000004f83e4f51ef6912c       at 
java.util.concurrent.FutureTask.run(FutureTask.java:138)
@400000004f83e4f51ef69514       at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:8
86)
@400000004f83e4f51ef69ce4       at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
@400000004f83e4f51ef69ce4       ... 1 more

We would be very interested in seeing a fix for this and we can second that 
this is fixed by using the ThreadSafeClientConnManager. Just do this to fix it:

public DefaultHttpFetcher() {
    // this follows redirects by default
    this.httpClient = new DefaultHttpClient();
    this.httpClient.getParams().setParameter(ClientPNames.CONNECTION_MANAGER_FACTORY_CLASS_NAME, ThreadSafeClientConnManager.class.getName());

Original comment by [email protected] on 23 Apr 2012 at 5:04