Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CHROMEDRIVER_CHROME_BINARY TypeError: Binary Location Must be a String #93

Open
KamarajuKusumanchi opened this issue Dec 29, 2023 · 3 comments
Open

CHROMEDRIVER_CHROME_BINARY TypeError: Binary Location Must be a String #93

KamarajuKusumanchi opened this issue Dec 29, 2023 · 3 comments

Comments

@KamarajuKusumanchi
Copy link

KamarajuKusumanchi commented Dec 29, 2023 •
edited
Loading

I am getting

TypeError: Binary Location Must be a String

when trying to run finance_dl.cli with schwab configuration. It is looking for os.getenv("CHROMEDRIVER_CHROME_BINARY") but could not find it. Could you please tell me how to fix the issue?

Steps to reproduce:
Create an environment file

% cat env_test_finance-dl.yml
name: test_finance-dl
channels:
  - defaults
dependencies:
  - python=3.12
  - pip
  - pip:
    - git+https://github.com/jbms/finance-dl

Create the environment

% conda env create -f env_test_finance-dl.yml

Activate the environment

% conda activate test_finance-dl

Create a configuration file

% cat finance_dl_config.py
import os
profile_dir = os.path.join(os.getenv('HOME'), '.cache', 'finance_dl')
data_dir = '/home/rajulocal/x/x5'
def CONFIG_schwab():
    return dict(
        module='finance_dl.schwab',
        credentials={
            'username': 'XXXXXX',
            'password': 'XXXXXX',
        },
        output_directory=os.path.join(data_dir, 'schwab'),
        profile_dir=profile_dir,
        headless=False,
        min_start_date='2023-11-01'
    )

Run

% python -m finance_dl.cli --config-module finance_dl_config --config schwab
Traceback (most recent call last):
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 411, in retry
    return func()
           ^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 430, in fetch
    with temp_scraper(scraper_class, **kwargs) as scraper:
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 393, in temp_scraper
    scraper = scraper_type(*args, download_dir=download_dir,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/schwab.py", line 87, in __init__
    super().__init__(use_seleniumrequests=True, **kwargs)
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 146, in __init__
    chrome_options.binary_location = os.getenv("CHROMEDRIVER_CHROME_BINARY")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/selenium/webdriver/chromium/options.py", line 52, in binary_location
    raise TypeError(self.BINARY_LOCATION_ERROR)
TypeError: Binary Location Must be a String
Waiting 0 seconds before retrying
Traceback (most recent call last):
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 411, in retry
    return func()
           ^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 430, in fetch
    with temp_scraper(scraper_class, **kwargs) as scraper:
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 393, in temp_scraper
    scraper = scraper_type(*args, download_dir=download_dir,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/schwab.py", line 87, in __init__
    super().__init__(use_seleniumrequests=True, **kwargs)
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 146, in __init__
    chrome_options.binary_location = os.getenv("CHROMEDRIVER_CHROME_BINARY")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/selenium/webdriver/chromium/options.py", line 52, in binary_location
    raise TypeError(self.BINARY_LOCATION_ERROR)
TypeError: Binary Location Must be a String
Waiting 0 seconds before retrying
Traceback (most recent call last):
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 411, in retry
    return func()
           ^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 430, in fetch
    with temp_scraper(scraper_class, **kwargs) as scraper:
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 393, in temp_scraper
    scraper = scraper_type(*args, download_dir=download_dir,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/schwab.py", line 87, in __init__
    super().__init__(use_seleniumrequests=True, **kwargs)
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 146, in __init__
    chrome_options.binary_location = os.getenv("CHROMEDRIVER_CHROME_BINARY")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/selenium/webdriver/chromium/options.py", line 52, in binary_location
    raise TypeError(self.BINARY_LOCATION_ERROR)
TypeError: Binary Location Must be a String
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/cli.py", line 94, in <module>
    main()
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/cli.py", line 90, in main
    module.run(**spec)
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/schwab.py", line 374, in run
    scrape_lib.run_with_scraper(SchwabScraper, **kwargs)
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 433, in run_with_scraper
    retry(fetch)
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 411, in retry
    return func()
           ^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 430, in fetch
    with temp_scraper(scraper_class, **kwargs) as scraper:
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 393, in temp_scraper
    scraper = scraper_type(*args, download_dir=download_dir,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/schwab.py", line 87, in __init__
    super().__init__(use_seleniumrequests=True, **kwargs)
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/finance_dl/scrape_lib.py", line 146, in __init__
    chrome_options.binary_location = os.getenv("CHROMEDRIVER_CHROME_BINARY")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/selenium/webdriver/chromium/options.py", line 52, in binary_location
    raise TypeError(self.BINARY_LOCATION_ERROR)
TypeError: Binary Location Must be a String

There is no CHROMEDRIVER_CHROME_BINARY environment variable set on my machine.

% ipython
Python 3.12.0 | packaged by Anaconda, Inc. | (main, Oct  2 2023, 17:29:18) [GCC 11.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.19.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: 
import os

In [2]: 
os.getenv("CHROMEDRIVER_CHROME_BINARY") is None
Out[2]: 
True

FWIW chromedriver-path shows

% chromedriver-path 
/opt/rajulocal/miniconda3/envs/test_finance-dl/lib/python3.12/site-packages/chromedriver_binary
@paulirish
Copy link

I got passed this with pip install -Iv selenium==4.2.0

@KamarajuKusumanchi
Copy link
Author

Thanks @paulirish . I am able to get past this error with selenium 4.2.0 . But it looks like the schwab scraper is broken. It opens the chrome window, enters the password but it is not able to login. It does not work even if I manually enter the password. Since it is a different issue, I will open another issue for it.

The original environment file

 % cat env_test_finance-dl.yml
name: test_finance-dl
channels:
  - defaults
dependencies:
  - python=3.12
  - pip
  - pip:
    - git+https://github.com/jbms/finance-dl

is pulling

 % conda list "^python$|selenium|chromedriver-binary" -n test_finance-dl   
# packages in environment at /opt/rajulocal/miniconda3/envs/test_finance-dl:
#
# Name                    Version                   Build  Channel
chromedriver-binary       122.0.6211.0.0           pypi_0    pypi
python                    3.12.0               h996f2a0_0  
selenium                  4.16.0                   pypi_0    pypi
selenium-requests         2.0.3                    pypi_0    pypi

The error does not show up with

 % cat env_test_finance-dl_v2.yml
name: test_finance-dl_v2
channels:
  - defaults
dependencies:
  - python=3.12
  - pip
  - pip:
    - selenium==4.2.0
    - chromedriver-binary==120.*
    - git+https://github.com/jbms/finance-dl

which is pulling

 % conda list "^python$|selenium|chromedriver-binary" -n test_finance-dl_v2
# packages in environment at /opt/rajulocal/miniconda3/envs/test_finance-dl_v2:
#
# Name                    Version                   Build  Channel
chromedriver-binary       120.0.6099.109.0          pypi_0    pypi
python                    3.12.0               h996f2a0_0  
selenium                  4.2.0                    pypi_0    pypi
selenium-requests         1.4.1                    pypi_0    pypi

I am specifying the chromedriver-binary version in the environment file since it has to match with my chrome's version.

 % dpkg -l google-chrome-stable                                            
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                 Version          Architecture Description
+++-====================-================-============-=================================
ii  google-chrome-stable 120.0.6099.199-1 amd64        The web browser from Google

The latest version of selenium is 4.16.0 (released on 2023-12-06 as per https://pypi.org/project/selenium/#history). It would be nice if finance-dl can support the latest version. If that is not possible, the correct selenium version should be specified in the dependencies.

@moritzj29
Copy link
Contributor

this is probably related to the fact that previously an empty CHROMEDRIVER_CHROME_BINARY was allowed but this is no longer the case (SeleniumHQ/selenium#12328)

in my case I use a remote driver anyway (selenium docker container) so I decided to skip the setting if a remote is specified: moritzj29@ae145aa

but since people here do not mention remote drivers, I guess there needs to be some further tweaking... maybe just skip the setting if the env does not exist?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@paulirish @KamarajuKusumanchi @moritzj29