simplecrawler Not imported
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
Import to Registry
This package is not yet in the greenflagged registry. Import it to start the review process.
1.1.9
Latest Version
BSD-2-Clause
License
70
Published Versions
4
Dependencies
Maintainers
cgiffardfredrikekelundkbychkov
Keywords
simplecrawlerspidercachequeuesimplecrawlereventemitter
Dependencies
| Package | Constraint |
|---|---|
| async | ^3.1.0 |
| iconv-lite | ^0.5.0 |
| robots-parser | ^2.1.1 |
| urijs | ^1.19.1 |
Published Versions
| Version |
|---|
| 1.1.9 |
| 1.1.8 |
| 1.1.7 |
| 1.1.6 |
| 1.1.5 |
| 1.1.4 |
| 1.1.3 |
| 1.1.2 |
| 1.1.1 |
| 1.1.0 |
| 1.0.5 |
| 1.0.4 |
| 1.0.3 |
| 1.0.2 |
| 1.0.1 |
| 1.0.0 |
| 0.7.0 |
| 0.6.2 |
| 0.6.1 |
| 0.6.0 |
| 0.5.4 |
| 0.5.3 |
| 0.5.2 |
| 0.5.1 |
| 0.5.0 |
| 0.4.2 |
| 0.4.1 |
| 0.4.0 |
| 0.3.14 |
| 0.3.13 |
| 0.3.12 |
| 0.3.11 |
| 0.3.10 |
| 0.3.9 |
| 0.3.8 |
| 0.3.7 |
| 0.3.6 |
| 0.3.5 |
| 0.3.4 |
| 0.3.3 |
| 0.3.2 |
| 0.3.1 |
| 0.3.0 |
| 0.2.10 |
| 0.2.9 |
| 0.2.8 |
| 0.2.7 |
| 0.2.6 |
| 0.2.5 |
| 0.2.4 |
| 0.2.3 |
| 0.2.2 |
| 0.2.1 |
| 0.2.0 |
| 0.1.7 |
| 0.1.6 |
| 0.1.5 |
| 0.1.4 |
| 0.1.3 |
| 0.1.2 |
| 0.1.1 |
| 0.1.0 |
| 0.0.10 |
| 0.0.9 |
| 0.0.8 |
| 0.0.7 |
| 0.0.6 |
| 0.0.5 |
| 0.0.4 |
| 0.0.3 |