Norconex Crawler Core

Open-Source Crawler Core Library

Documentation Download

Core Crawler library

The Norconex Crawler Core is a library containing features and reusable code shared between all crawlers. It is not an executable application.

If you are looking for a web crawler, filesystem crawler or else, check out Norconex Web Crawler or Norconex File System Crawler.

Use this library if you are interested to build a new crawler, or if you want to replace the Crawler Core version you have with your downloaded Crawler zip file with a more recent one.

Refer to the API documentation for many of the features that are shared features between all crawlers.

Latest news

Norconex Web Crawler 3.0.0 Released!
2022-01-05
The new major release of Norconex HTTP Collector it is finally here. Check out what's new. More...

Norconex HTTP Collector 3.0.0 Release Candidate 1
2021-10-10
Is this the last pre-release? Put it to the test and let us know! Includes applicable release candidates of core dependencies as well. More...

Norconex HTTP Collector 3.0.0 Milestone 2
2021-07-28
3.0.0 second milestone release. Includes applicable milestone releases of core dependencies as well. More...

Norconex HTTP Collector 3.0.0 Milestone 1
2021-03-01
A step closer to final release. Available with milestone releases of core dependencies as well. More...

Norconex HTTP Collector 3.0.0 snapshots available
2020-09-07
Development builds of upcoming version 3 now available to experiment with. More...