Extended and fully async version of scrape-it
npm install --save flamescraper
- Multiple pages parse per request
- Recursive subpage loading per each parameter with
subpagefield - Pagination infinity loop, where
Next pageparam detected and next page link is finded - You can select for each page: use
Puppeteerfor SPA sites or defaultweb request - If Convert and How is a string => make functions
- Async
PageConvertorcallback after each page (in pagination) for bulk processing/save results, but with full control after each step - Async
stopCallbackfor stop scraping if results are right for you (for example you scraped next pages last time)