forked from theanti9/PyCrawler
-
Notifications
You must be signed in to change notification settings - Fork 1
serenity-valley/PyCrawler
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
usage: PyCrawler.py [-h] [--dbname DBNAME] [--followextern] [--verbose] [--striphtml] [--downloadstatic] starturl crawldepth positional arguments: starturl The root URL to start crawling from. crawldepth Number of levels to crawl down to before quitting. Default is 10. optional arguments: -h, --help show this help message and exit --dbname DBNAME The db file to be created for storing crawl data. --followextern Follow external links(disabled by default). --verbose Be verbose while crawling. --striphtml Strip HTML tags from crawled content. --downloadstatic Download static content.
About
A simple python web crawler
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Python 100.0%