Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxy to use OpenAI API #6188

Open
gary-young opened this issue Jan 10, 2025 · 3 comments
Open

Proxy to use OpenAI API #6188

gary-young opened this issue Jan 10, 2025 · 3 comments
Labels
enhancement New feature or request

Comments

@gary-young
Copy link

I try to use the OpenAI API with OpenHands Agent GUI. However, due to some network problem, I (and some people) must use proxy to get service from OpenAI. How can I set the proxy?

I install the openhands with a standard installed docker:

Specifically, I install docker in root account and then use newgrp dockerto get the authority to use docker.
Then I install OpenHands by the standard instruction in the main page.

  docker pull docker.all-hands.dev/all-hands-ai/runtime:0.19-nikolaik

  docker run -it --rm --pull=always \
      -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.19-nikolaik \
      -e LOG_ALL_EVENTS=true \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v ~/.openhands-state:/.openhands-state \
      -p 3000:3000 \
      --add-host host.docker.internal:host-gateway \
      --name openhands-app \
      docker.all-hands.dev/all-hands-ai/openhands:0.19

I have tried to use HTTP_PROXY and HTTPS_PROXY enviroment variables but it seems useless. I also try this method (https://docs.docker.com/engine/cli/proxy/). Specifically I write the proxy in a ~/.docker/config.json file. However, I find this error.

04:57:34 - openhands:INFO: openhands_config.py:48 - Using config class None
INFO:     Started server process [10]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit)
INFO:     ('172.17.0.1', 39662) - "WebSocket /socket.io/?latest_event_id=0&conversation_id=63bab271fd3b492f9b3aaa3e2cd44e92&EIO=4&transport=websocket" [accepted]
04:57:40 - openhands:INFO: listen_socket.py:25 - sio:connect: xqorPJ8aNPKNeQodAAAB
04:57:40 - openhands:INFO: manager.py:214 - join_conversation:63bab271fd3b492f9b3aaa3e2cd44e92:xqorPJ8aNPKNeQodAAAB
04:57:40 - openhands:INFO: manager.py:369 - _get_event_stream:63bab271fd3b492f9b3aaa3e2cd44e92
04:57:40 - openhands:INFO: manager.py:351 - maybe_start_agent_loop:63bab271fd3b492f9b3aaa3e2cd44e92
04:57:40 - openhands:INFO: manager.py:354 - start_agent_loop:63bab271fd3b492f9b3aaa3e2cd44e92
04:57:40 - openhands:INFO: manager.py:369 - _get_event_stream:63bab271fd3b492f9b3aaa3e2cd44e92
04:57:40 - openhands:INFO: manager.py:372 - found_local_agent_loop:63bab271fd3b492f9b3aaa3e2cd44e92
04:57:42 - openhands:INFO: docker_runtime.py:133 - [runtime 63bab271fd3b492f9b3aaa3e2cd44e92] Starting runtime with image: docker.all-hands.dev/all-hands-ai/runtime:0.19-nikolaik
04:57:52 - openhands:INFO: docker_runtime.py:137 - [runtime 63bab271fd3b492f9b3aaa3e2cd44e92] Container started: openhands-runtime-63bab271fd3b492f9b3aaa3e2cd44e92. VSCode URL: None
04:57:52 - openhands:INFO: docker_runtime.py:145 - [runtime 63bab271fd3b492f9b3aaa3e2cd44e92] Waiting for client to become ready at http://host.docker.internal:39668...
04:57:52 - openhands:ERROR: session.py:116 - Error creating controller: 502 Server Error: Bad Gateway for url: http://host.docker.internal:39668/alive
Traceback (most recent call last):
  File "/app/openhands/runtime/utils/request.py", line 45, in send_request
    response.raise_for_status()
  File "/app/.venv/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: http://host.docker.internal:39668/alive

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/openhands/server/session/session.py", line 104, in initialize_agent
    await self.agent_session.start(
  File "/app/openhands/server/session/agent_session.py", line 93, in start
    await self._create_runtime(
  File "/app/openhands/server/session/agent_session.py", line 198, in _create_runtime
    await self.runtime.connect()
  File "/app/openhands/runtime/impl/docker/docker_runtime.py", line 148, in connect
    await call_sync_from_async(self._wait_until_alive)
  File "/app/openhands/utils/async_utils.py", line 18, in call_sync_from_async
    result = await coro
             ^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/utils/async_utils.py", line 17, in <lambda>
    coro = loop.run_in_executor(None, lambda: fn(*args, **kwargs))
                                              ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/runtime/impl/docker/docker_runtime.py", line 337, in _wait_until_alive
    self.check_if_alive()
  File "/app/openhands/runtime/impl/action_execution/action_execution_client.py", line 99, in check_if_alive
    with self._send_action_server_request(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/runtime/impl/action_execution/action_execution_client.py", line 96, in _send_action_server_request
    return send_request(self.session, method, url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/runtime/utils/request.py", line 51, in send_request
    raise RequestHTTPError(
openhands.runtime.utils.request.RequestHTTPError: 502 Server Error: Bad Gateway for url: http://host.docker.internal:39668/alive

My config.json file is :

{
        "proxies": {
                "default": {
                        "httpProxy": "http://host.docker.internal:7890",
                        "httpsProxy": "https://host.docker.internal:7890",
                        "noProxy": "http://localhost"
                }
        }
}

I use a clash in the host machine and the clash listen to the port 7890. In the host machine, I can successfully get google by setting HTTP_PROXY=http://127.0.0.1:7890

And I remove the proxy config the error disappears but I cannot get the OpenAI API.

Does anyone know how to solve the problem?
@gary-young gary-young added the enhancement New feature or request label Jan 10, 2025
@enyst
Copy link
Collaborator

enyst commented Jan 10, 2025

Please try in the UI settings, choose Advanced Settings, and set base URL to the proxy

@gary-young
Copy link
Author

Please try in the UI settings, choose Advanced Settings, and set base URL to the proxy

In fact I set up the clash on the host machine and I want to use the https://host.docker.internal:7890 as a tunnel. I have start the clash and make it listen to port 7890. So do you mean I should set the base URL as https://host.docker.internal:7890? I have try this and get another error:

07:14:06 - openhands:INFO: manage_conversations.py:42 - Initializing new conversation
07:14:06 - openhands:INFO: manage_conversations.py:44 - Loading settings
07:14:06 - openhands:INFO: manage_conversations.py:47 - Settings loaded
07:14:06 - openhands:INFO: manage_conversations.py:57 - Loading conversation store
07:14:06 - openhands:INFO: manage_conversations.py:61 - Conversation store loaded
07:14:06 - openhands:INFO: manage_conversations.py:67 - New conversation ID: 03a7e76c1f3442db96436ff6ab28f70e
07:14:06 - openhands:INFO: manage_conversations.py:73 - Saving metadata for conversation 03a7e76c1f3442db96436ff6ab28f70e
07:14:06 - openhands:INFO: manage_conversations.py:83 - Starting agent loop for conversation 03a7e76c1f3442db96436ff6ab28f70e
07:14:06 - openhands:INFO: manager.py:351 - maybe_start_agent_loop:03a7e76c1f3442db96436ff6ab28f70e
07:14:06 - openhands:INFO: manager.py:354 - start_agent_loop:03a7e76c1f3442db96436ff6ab28f70e
07:14:06 - openhands:INFO: manager.py:369 - _get_event_stream:03a7e76c1f3442db96436ff6ab28f70e
07:14:06 - openhands:INFO: manager.py:372 - found_local_agent_loop:03a7e76c1f3442db96436ff6ab28f70e
07:14:06 - openhands:INFO: manage_conversations.py:95 - Finished initializing conversation 03a7e76c1f3442db96436ff6ab28f70e
INFO:     172.17.0.1:48448 - "POST /api/conversations HTTP/1.1" 200 OK
INFO:     172.17.0.1:48448 - "GET /api/options/config HTTP/1.1" 200 OK
07:14:08 - openhands:INFO: docker_runtime.py:133 - [runtime 03a7e76c1f3442db96436ff6ab28f70e] Starting runtime with image: docker.all-hands.dev/all-hands-ai/runtime:0.19-nikolaik
INFO:     ('172.17.0.1', 48452) - "WebSocket /socket.io/?latest_event_id=-1&conversation_id=03a7e76c1f3442db96436ff6ab28f70e&EIO=4&transport=websocket" [accepted]
07:14:08 - openhands:INFO: listen_socket.py:25 - sio:connect: jAa_ELXE5BYV0gkZAAAB
07:14:09 - openhands:INFO: manager.py:214 - join_conversation:03a7e76c1f3442db96436ff6ab28f70e:jAa_ELXE5BYV0gkZAAAB
07:14:09 - openhands:INFO: manager.py:369 - _get_event_stream:03a7e76c1f3442db96436ff6ab28f70e
07:14:09 - openhands:INFO: manager.py:372 - found_local_agent_loop:03a7e76c1f3442db96436ff6ab28f70e
INFO:     172.17.0.1:48456 - "GET /api/settings HTTP/1.1" 200 OK
07:14:16 - openhands:INFO: docker_runtime.py:137 - [runtime 03a7e76c1f3442db96436ff6ab28f70e] Container started: openhands-runtime-03a7e76c1f3442db96436ff6ab28f70e. VSCode URL: None
07:14:16 - openhands:INFO: docker_runtime.py:145 - [runtime 03a7e76c1f3442db96436ff6ab28f70e] Waiting for client to become ready at http://host.docker.internal:37707...
07:16:17 - openhands:ERROR: session.py:116 - Error creating controller: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 507, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/http/client.py", line 1428, in getresponse
    response.begin()
  File "/usr/local/lib/python3.12/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/socket.py", line 707, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/util/util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 507, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/http/client.py", line 1428, in getresponse
    response.begin()
  File "/usr/local/lib/python3.12/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/socket.py", line 707, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/openhands/server/session/session.py", line 104, in initialize_agent
    await self.agent_session.start(
  File "/app/openhands/server/session/agent_session.py", line 93, in start
    await self._create_runtime(
  File "/app/openhands/server/session/agent_session.py", line 198, in _create_runtime
    await self.runtime.connect()
  File "/app/openhands/runtime/impl/docker/docker_runtime.py", line 148, in connect
    await call_sync_from_async(self._wait_until_alive)
  File "/app/openhands/utils/async_utils.py", line 18, in call_sync_from_async
    result = await coro
             ^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/utils/async_utils.py", line 17, in <lambda>
    coro = loop.run_in_executor(None, lambda: fn(*args, **kwargs))
                                              ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 418, in exc_check
    raise retry_exc.reraise()
          ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 185, in reraise
    raise self.last_attempt.result()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/runtime/impl/docker/docker_runtime.py", line 337, in _wait_until_alive
    self.check_if_alive()
  File "/app/openhands/runtime/impl/action_execution/action_execution_client.py", line 99, in check_if_alive
    with self._send_action_server_request(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/runtime/impl/action_execution/action_execution_client.py", line 96, in _send_action_server_request
    return send_request(self.session, method, url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/app/.venv/lib/python3.12/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/app/openhands/runtime/utils/request.py", line 43, in send_request
    response = session.request(method, url, timeout=timeout, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/adapters.py", line 682, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

@enyst
Copy link
Collaborator

enyst commented Jan 10, 2025

The error in your post doesn't seem related to setting that base URL, it never got to try to send a prompt to the LLM API. (that's when it's used)

It's docker. I'm not sure why it fails to connect. That connection in your error log is NOT to the LLM, it's to the sandbox where the agent is running. Are you running with the command in the readme and no other docker configuration modifications?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@enyst @gary-young