Skip to content

A web-scraping framework written in Javascript, using PhantomJS and jQuery

License

Notifications You must be signed in to change notification settings

dmccarey/pjscrape

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

74 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Homepage: http://nrabinowitz.github.io/pjscrape/

Overview

pjscrape is a framework for anyone who's ever wanted a command-line tool for web scraping using Javascript and jQuery. Built for PhantomJS, it allows you to scrape pages in a fully rendered, Javascript-enabled context from the command line, no browser required.

Dependencies

Features

  • Client-side, Javascript-based scraping environment with full access to jQuery functions
  • Easy, flexible syntax for setting up one or more scrapers
  • Recursive/crawl scraping
  • Delay scrape until a "ready" condition occurs
  • Load your own scripts on the page before scraping
  • Modular architecture for logging and writing/formatting scraped items
  • Client-side utilities for common tasks
  • Growing set of unit tests

Please see http://nrabinowitz.github.io/pjscrape/ for usage, examples, and documentation.

Comments and questions welcomed at: nick (at) nickrabinowitz (dot) com.

About

A web-scraping framework written in Javascript, using PhantomJS and jQuery

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • JavaScript 83.3%
  • Python 10.7%
  • HTML 5.7%
  • Other 0.3%