Skip to content

liuchen9494/web-crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

crawler

illinois-logo

This crawler is for UIUC Course Explorer.

Setup

# install the dependencies
npm i

Crawling the catelog

# create and sync the db
node model.js

# start crawling the catelog
node catalog.js --db

# or you wish to save to json file
node catelog.js --file=data.json

# or to specify the endpoint. This will only update part of data.
node catelog.js --db --endpoint=2018
node catelog.js --db --endpoint=2018/spring

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published