- Install dependencies
poetry install- Run Neptun-Scraper
scrapy crawl dockerhubDockerRegistrySpider -a query="python"- Run Neptun-Bot (WebUI, Rest-Interface, Scrapy-Daemon)
docker compose up --build- Access the WebUI:
http://localhost:8000- Access the DAEMON-Interface:
http://localhost:6802- Create a Scrapyd-Job
curl http://localhost:6802/schedule.json -d project=neptunscraper -d spider=dockerhubDockerRegistrySpider