Skip to content

Latest commit

 

History

History
39 lines (24 loc) · 1.04 KB

README.md

File metadata and controls

39 lines (24 loc) · 1.04 KB

This repository is deprecated. Working on rewrite,... - Oto

Univizor Scrapers

List of all entities (Google Document)

Installation

How to run scraper?

Scraper Command
vsvo cd vsvo; ruby vsvo.rb > ./vsvo.log 2>&1
ung cd ung; ruby ung-library.rb > ./ung.log 2>&1
gea cd gea; ruby gea-college.rb > ./gea.log 2>&1

(*) gea: set REDOWNLOAD=true in gea-college.rb to fetch again

Database

Dumping latest structure on production.

mysqldump -d -h $DB_HOST -u $DB_USER -p$DB_PASS $DB_NAME \
--skip-comments --skip-add-drop-table > $DB_NAME.sql

Loading database localy.

mysql -h <host> -u <username> -p < univizor.sql

Python

mkvirtualenv --no-site-packages univizor
workon univizor
pip install --upgrade -r requirements.txt

Pipeline

Output

PDF file