Javascript web scraper

Sample web scraping project that uses phantomjs and casperjs to scrap contents of a website

Installation instructions

• install Phantomjs - http://phantomjs.org
• install Casperjs - http://casperjs.org

  NB binary installation recommended for Windows users

• execute the following commands from the project root folder

  $ npm install

Thats it, for more info please feel free checkout more resources online and also refer to the official casperjs and phantomjs websites

#Running the server and crawler service

• Start the server which provides the endpoint needed to save the scraped data:

  $ node server.js

The web scraping service is scheduled with node-cron to run after every three minutes, The service is triggered by executing the following command:

  $ node scraper.trigger.js