Failed to scrape jobs for MonsterScraperUKEng

[benb0jangles]

I'm getting an error:
Terminal Command: funnel load -s settings.yaml
[2021-03-21 01:21:25,777] [INFO] JobFunnel: Scraping local providers with: ['IndeedScraperUKEng', 'MonsterScraperUKEng']
[2021-03-21 01:21:26,638] [ERROR] JobFunnel: Failed to scrape jobs for IndeedScraperUKEng
[2021-03-21 01:21:26,639] [INFO] MonsterScraperUKEng: No get() or set() will be done for Job attrs: ['REMOTENESS']
[2021-03-21 01:21:28,454] [ERROR] JobFunnel: Failed to scrape jobs for MonsterScraperUKEng
[2021-03-21 01:21:28,454] [INFO] JobFunnel: Completed all scraping, found 0 new jobs.
[2021-03-21 01:21:28,475] [INFO] JobFunnel: Done. View your current jobs in demo_job_search_results/demo_search.csv

Using:
Mac OS 10.12.6

[PaulMcInnis]

Thanks for opening an issue, can you confirm that you are able to obtain results (non advertisement results) for the search you are performing on the monster website?

Additionally, if you are open to sharing your search URLs from logs it would be very helpful to identify what the issue is.

We current have CI for the UK Indeed scraper but it only performs a basic search.

[PaulMcInnis]

I think we have just merged a PR that resolves this, feel free to try again on current master

[benb0jangles]

pip uninstall JobFunnel
pip install git+https://github.com/PaulMcInnis/JobFunnel.git
funnel load -s my_settings.yaml

[2021-04-02 10:24:34,081] [INFO] JobFunnel: Scraping local providers with: ['IndeedScraperUKEng', 'MonsterScraperUKEng']
[2021-04-02 10:24:35,154] [INFO] IndeedScraperUKEng: Found 3 pages of search results for query=HTML
[2021-04-02 10:24:36,456] [INFO] IndeedScraperUKEng: Scraped 126 job listings from search results pages
100%|##################################################################################################| 126/126 [03:09<00:00, 1.51s/it]
[2021-04-02 10:27:46,274] [INFO] MonsterScraperUKEng: No get() or set() will be done for Job attrs: ['REMOTENESS']
[2021-04-02 10:27:48,070] [ERROR] JobFunnel: Failed to scrape jobs for MonsterScraperUKEng
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/funnel", line 11, in
load_entry_point('JobFunnel==3.0.2', 'console_scripts', 'funnel')()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/jobfunnel/main.py", line 28, in main
job_funnel.run()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/jobfunnel/backend/jobfunnel.py", line 114, in run
scraped_jobs_dict = self.scrape()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/jobfunnel/backend/jobfunnel.py", line 244, in scrape
self._check_for_inter_scraper_validity(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/jobfunnel/backend/jobfunnel.py", line 220, in _check_for_inter_scraper_validity
raise ValueError(
ValueError: Inter-scraper key-id duplicate! ead06c46e9940d3d

Mac OS 10.12.6

[PaulMcInnis]

Duplicate #148