Open-source crawlers

Full-featured, flexible and extensible. Run on any platform.
Crawl what you want, how you want.

Download | Features | User Feedback | Related

Available Crawlers

...

Web Crawler

Web crawler for the Internet or Intranet.

Download   More Info

...

File System Crawler

Crawl data from local disk, FTP, SFTP, WebDAV, HDFS, ...

Download   More Info

Go to top

Features

Why chose Norconex Crawlers? Click here or on a feature to learn more.

Universal Crawlers

Easy for developers to extend

Commercially supported

Modify document metadata

Easy to run

Embeddable

Ease of maintenance

Resumable upon system failure

Modular design

Cross-platform

Portable

Open-Source

Powerful

Easy to use

Good documentation

Event listeners

Logs are meaningful and verbose

Flexible

More features: Web Crawler | File System Crawler
Go to top

User Feedback

Real feedback from real Github users. Click a quote to view its origin.

Who uses Norconex Crawlers?
Go to top