Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

proxy.pac: isInNet() & isResolvable() not working properly #210

Closed
fwa-wup opened this issue Feb 15, 2024 · 8 comments
Closed

proxy.pac: isInNet() & isResolvable() not working properly #210

fwa-wup opened this issue Feb 15, 2024 · 8 comments
Labels
bug fixed

Comments

@fwa-wup
Copy link

fwa-wup commented Feb 15, 2024 •
edited
Loading

Hi everyone!

We are using isInNet() to evaluate if a client is connect to our corporate network. Similar to this:

if(!isInNet(dnsResolve("server01.internal"), "172.16.0.1", "255.255.255.255") ) {
		return "DIRECT";
}

I also tested:

if(!isInNet("server01.internal", "172.16.0.1", "255.255.255.255") ) {
		return "DIRECT";
}

and

if(!isResolvable("server01.internal") ) {
	return "DIRECT";
}

Note: "server01.internal" can only be resolved internally

While there may be other ways to accomplish this, i am not able to change this due to internal reasons.

Unfortunately px doesn't support this type of expression and simply crashes. I am not an expert ins writing code. But this issue seems to be similar to #164

I guess it is it's failing to resolve the host and crashes instead of returning false and continuing the .pac-file?

Could #170 solve this issue?

I used firefox and apt for testing. Strangely enough, this works fine when using curl tho.

Can anyone help?

Configuration:

[proxy]
server = 
pac = http://mypacproxy.company.internal/proxy.pac
pac_encoding = iso-8859-1
port = 3128
listen = 127.0.0.1
gateway = 0
hostonly = 1
allow = 
noproxy = 
useragent = 
username = user@localdomain
auth = ANYSAFE

[client]

[settings]
log = 0
workers = 2
threads = 32
idle = 30
socktimeout = 20.0
proxyreload = 60
foreground = 0

Logfile:

px[130744]: ----------------------------------------
px[130744]: Exception occurred during processing of request from ('127.0.0.1', 46722)
px[130744]: Fatal Python error: PyEval_RestoreThread: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
px[130744]: Python runtime state: initialized
px[130744]: Thread 0x00007f59c69776c0 (most recent call first):
px[130744]:   File "/usr/lib/python3.11/tokenize.py", line 325 in read_or_stop
px[130744]:   File "/usr/lib/python3.11/tokenize.py", line 379 in detect_encoding
px[130744]:   File "/usr/lib/python3.11/tokenize.py", line 325 in open
px[130744]:   File "/usr/lib/python3.11/linecache.py", line 137 in updatecache
px[130744]:   File "/usr/lib/python3.11/linecache.py", line 46 in getlines
px[130744]:   File "/usr/lib/python3.11/linecache.py", line 30 in getline
px[130744]:   File "/usr/lib/python3.11/traceback.py", line 321 in line
px[130744]:   File "/usr/lib/python3.11/traceback.py", line 436 in _extract_from_extended_frame_gen
px[130744]:   File "/usr/lib/python3.11/traceback.py", line 702 in __init__
px[130744]:   File "/usr/lib/python3.11/traceback.py", line 124 in print_exception
px[130744]:   File "/usr/lib/python3.11/traceback.py", line 183 in print_exc
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 381 in handle_error
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 693 in process_request_thread
px[130744]:   File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58 in run
px[130744]:   File "/usr/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
px[130744]:   File "/usr/lib/python3.11/threading.py", line 982 in run
px[130744]:   File "/usr/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
px[130744]:   File "/usr/lib/python3.11/threading.py", line 1002 in _bootstrap
px[130744]: Current thread 0x00007f59c717c6c0 (most recent call first):
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/pac.py", line 74 in find_proxy_for_url
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/wproxy.py", line 258 in find_proxy_for_url
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/handler.py", line 255 in get_destination
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/handler.py", line 159 in do_curl
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/handler.py", line 230 in do_GET
px[130744]:   File "/usr/lib/python3.11/http/server.py", line 424 in handle_one_request
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/handler.py", line 128 in handle_one_request
px[130744]:   File "/usr/lib/python3.11/http/server.py", line 436 in handle
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 755 in __init__
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 361 in finish_request
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 691 in process_request_thread
px[130744]:   File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58 in run
px[130744]:   File "/usr/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
px[130744]:   File "/usr/lib/python3.11/threading.py", line 982 in run
px[130744]:   File "/usr/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
px[130744]:   File "/usr/lib/python3.11/threading.py", line 1002 in _bootstrap
px[130744]: Thread 0x00007f59c797d6c0 (most recent call first):
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/pac.py", line 74 in find_proxy_for_url
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/wproxy.py", line 258 in find_proxy_for_url
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/handler.py", line 255 in get_destination
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/handler.py", line 159 in do_curl
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/handler.py", line 248 in do_CONNECT
px[130744]:   File "/usr/lib/python3.11/http/server.py", line 424 in handle_one_request
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/handler.py", line 128 in handle_one_request
px[130744]:   File "/usr/lib/python3.11/http/server.py", line 436 in handle
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 755 in __init__
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 361 in finish_request
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 691 in process_request_thread
px[130744]:   File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58 in run
px[130744]:   File "/usr/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
px[130744]:   File "/usr/lib/python3.11/threading.py", line 982 in run
px[130744]:   File "/usr/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
px[130744]:   File "/usr/lib/python3.11/threading.py", line 1002 in _bootstrap
px[130744]: Thread 0x00007f59ca6d2040 (most recent call first):
px[130744]:   File "/usr/lib/python3/dist-packages/netaddr/ip/sets.py", line 194 in _compact_single_network
px[130744]:   File "/usr/lib/python3/dist-packages/netaddr/ip/sets.py", line 296 in add
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/config.py", line 147 in get_host_ips
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/main.py", line 42 in verify_request
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 315 in _handle_request_noblock
px[130744]:   File "/usr/lib/python3.11/socketserver.py", line 238 in serve_forever
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/main.py", line 81 in serve_forever
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/main.py", line 91 in start_httpds
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/main.py", line 115 in start_worker
px[130744]:   File "/usr/lib/python3.11/multiprocessing/process.py", line 108 in run
px[130744]:   File "/usr/lib/python3.11/multiprocessing/process.py", line 314 in _bootstrap
px[130744]:   File "/usr/lib/python3.11/multiprocessing/popen_fork.py", line 71 in _launch
px[130744]:   File "/usr/lib/python3.11/multiprocessing/popen_fork.py", line 19 in __init__
px[130744]:   File "/usr/lib/python3.11/multiprocessing/context.py", line 281 in _Popen
px[130744]:   File "/usr/lib/python3.11/multiprocessing/context.py", line 224 in _Popen
px[130744]:   File "/usr/lib/python3.11/multiprocessing/process.py", line 121 in start
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/main.py", line 153 in run_pool
px[130744]:   File "/usr/local/lib/python3.11/dist-packages/px/main.py", line 301 in main
px[130744]:   File "/usr/local/bin/px", line 8 in <module>
px[130744]: Extension modules: _quickjs, psutil._psutil_linux, psutil._psutil_posix (total: 3)
@genotrance
Copy link
Owner

It seems similar to #164 and I have a fix - I just need to test before I push out the code to Github. Testing and posting binaries takes a long time so hopefully you will be able to test the fix before that.

@genotrance genotrance added the bug label Feb 20, 2024
@genotrance
Copy link
Owner

Fixed in branch v091. Please see if you can test the changes.

Only file to change is pac.py - change it in your existing install of Px and see if the issue goes away. Thanks in advance!

@fwa-wup
Copy link
Author

fwa-wup commented Feb 23, 2024 •
edited
Loading

Works! Thank you. However it is complaining about "keyring.util.properties". Did I miss something at install?

I am using gnome-keyring.

log

Serving at :3128 proc MainProcess
Serving at :3128 proc Process-1
----------------------------------------
Exception occurred during processing of request from ('127.0.0.1', 60038)
----------------------------------------
Exception occurred during processing of request from ('127.0.0.1', 60060)
----------------------------------------
Exception occurred during processing of request from ('127.0.0.1', 60034)
Traceback (most recent call last):
  File "/usr/lib/python3.11/socketserver.py", line 691, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.11/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.11/socketserver.py", line 755, in __init__
    self.handle()
  File "/usr/lib/python3.11/http/server.py", line 436, in handle
    self.handle_one_request()
  File "/home/fwa/git/px/px/handler.py", line 128, in handle_one_request
    http.server.BaseHTTPRequestHandler.handle_one_request(self)
  File "/usr/lib/python3.11/http/server.py", line 424, in handle_one_request
    method()
  File "/home/fwa/git/px/px/handler.py", line 230, in do_GET
    self.do_curl()
  File "/home/fwa/git/px/px/handler.py", line 159, in do_curl
    ipport = self.get_destination()
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fwa/git/px/px/handler.py", line 255, in get_destination
    servers, netloc, path = STATE.wproxy.find_proxy_for_url(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fwa/git/px/px/wproxy.py", line 258, in find_proxy_for_url
    return parse_proxy(self.pac.find_proxy_for_url(url, netloc[0])), netloc, path
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Traceback (most recent call last):
  File "/home/fwa/git/px/px/pac.py", line 88, in find_proxy_for_url
    proxies = self._ctxt.eval("FindProxyForURL")(url, host)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_quickjs.JSException: ReferenceError: 'FindProxyForURL' is not defined
    at <eval> (<input>)

----------------------------------------
  File "/usr/lib/python3.11/socketserver.py", line 691, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.11/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.11/socketserver.py", line 755, in __init__
    self.handle()
  File "/usr/lib/python3.11/http/server.py", line 436, in handle
    self.handle_one_request()
  File "/home/fwa/git/px/px/handler.py", line 128, in handle_one_request
    http.server.BaseHTTPRequestHandler.handle_one_request(self)
  File "/usr/lib/python3.11/http/server.py", line 424, in handle_one_request
    method()
  File "/home/fwa/git/px/px/handler.py", line 230, in do_GET
    self.do_curl()
  File "/home/fwa/git/px/px/handler.py", line 159, in do_curl
    ipport = self.get_destination()
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fwa/git/px/px/handler.py", line 255, in get_destination
    servers, netloc, path = STATE.wproxy.find_proxy_for_url(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fwa/git/px/px/wproxy.py", line 258, in find_proxy_for_url
    return parse_proxy(self.pac.find_proxy_for_url(url, netloc[0])), netloc, path
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fwa/git/px/px/pac.py", line 88, in find_proxy_for_url
    proxies = self._ctxt.eval("FindProxyForURL")(url, host)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Traceback (most recent call last):
_quickjs.JSException: ReferenceError: 'FindProxyForURL' is not defined
    at <eval> (<input>)

  File "/usr/lib/python3.11/socketserver.py", line 691, in process_request_thread
    self.finish_request(request, client_address)
----------------------------------------
  File "/usr/lib/python3.11/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.11/socketserver.py", line 755, in __init__
    self.handle()
  File "/usr/lib/python3.11/http/server.py", line 436, in handle
    self.handle_one_request()
  File "/home/fwa/git/px/px/handler.py", line 128, in handle_one_request
    http.server.BaseHTTPRequestHandler.handle_one_request(self)
  File "/usr/lib/python3.11/http/server.py", line 424, in handle_one_request
    method()
  File "/home/fwa/git/px/px/handler.py", line 248, in do_CONNECT
    self.do_curl()
  File "/home/fwa/git/px/px/handler.py", line 159, in do_curl
    ipport = self.get_destination()
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fwa/git/px/px/handler.py", line 255, in get_destination
    servers, netloc, path = STATE.wproxy.find_proxy_for_url(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fwa/git/px/px/wproxy.py", line 258, in find_proxy_for_url
    return parse_proxy(self.pac.find_proxy_for_url(url, netloc[0])), netloc, path
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fwa/git/px/px/pac.py", line 88, in find_proxy_for_url
    proxies = self._ctxt.eval("FindProxyForURL")(url, host)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_quickjs.JSException: ReferenceError: 'FindProxyForURL' is not defined
    at <eval> (<input>)

----------------------------------------
Error initializing plugin EntryPoint(name='jeepney', value='keyring_jeepney', group='keyring.backends').
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/keyring/backend.py", line 208, in _load_plugins
    init_func = ep.load()
                ^^^^^^^^^
  File "/usr/lib/python3/dist-packages/importlib_metadata/__init__.py", line 203, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.11/dist-packages/keyring_jeepney.py", line 9, in <module>
    from keyring.util.properties import ClassProperty
ModuleNotFoundError: No module named 'keyring.util.properties'
Error initializing plugin EntryPoint(name='jeepney', value='keyring_jeepney', group='keyring.backends').
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/keyring/backend.py", line 208, in _load_plugins
    init_func = ep.load()
                ^^^^^^^^^
  File "/usr/lib/python3/dist-packages/importlib_metadata/__init__.py", line 203, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.11/dist-packages/keyring_jeepney.py", line 9, in <module>
    from keyring.util.properties import ClassProperty
ModuleNotFoundError: No module named 'keyring.util.properties'

@genotrance
Copy link
Owner

The keyring_jeepney module is no longer needed since it is obsolete - #187 - hasn't been updated since 2017 and doesn't work with newer versions of keyring. You can go ahead and uninstall it. While you are at it, maybe update all Px dependencies, if not already.

I'm wondering about this though:

_quickjs.JSException: ReferenceError: 'FindProxyForURL' is not defined
    at <eval> (<input>)

@fwa-wup
Copy link
Author

fwa-wup commented Feb 26, 2024 •
edited
Loading

I uninstalled keyring_jeepney and now it seems to work fine without any complaints.

ATM i cannot reproduce the error message.
I'll keep an eye on that.

@genotrance
Copy link
Owner

Thanks for verifying!

@genotrance genotrance added fixed and removed waiting labels Feb 26, 2024
@fwa-wup
Copy link
Author

fwa-wup commented Feb 28, 2024

After 3 days of daily use, i cannot reproduce the error message. You can close this Issue once 9.1 is released.
Thank you for your amazing work!

@genotrance
Copy link
Owner

v0.9.1 has been released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug fixed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@genotrance @fwa-wup