Package com.norconex.collector.fs

Class FilesystemCollector

java.lang.Object

com.norconex.collector.core.Collector

com.norconex.collector.fs.FilesystemCollector

public class FilesystemCollector
extends Collector

Main application class. In order to use it properly, you must first configure it, either by providing a populated instance of FilesystemCollectorConfig, or by XML configuration, loaded using a collector config loader. Instances of this class can hold several crawler, running at once. This is convenient when there are configuration setting to be shared amongst crawlers. When you have many crawler jobs defined that have nothing in common, it may be best to configure and run them separately, to facilitate troubleshooting. There is no fair rule for this, experimentation will help you.

Author:

Pascal Essiembre

Field Summary

Fields inherited from class com.norconex.collector.core.Collector

NORCONEX_ASCII

Constructor Summary

Constructors

Constructor	Description
FilesystemCollector()	Creates a non-configured Filesystem collector.
FilesystemCollector(FilesystemCollectorConfig collectorConfig)	Creates and configure an HTTP Collector with the provided configuration.

Method Summary

Modifier and Type	Method	Description
protected Crawler	createCrawler(CrawlerConfig config)
FilesystemCollectorConfig	getCollectorConfig()
static void	main(String[] args)	Invokes the Filesystem Collector from the command line.

Methods inherited from class com.norconex.collector.core.Collector

clean, destroyCollector, exportDataStore, fireStopRequest, get, getCrawlers, getEventManager, getId, getReleaseVersions, getStreamFactory, getTempDir, getVersion, getWorkDir, importDataStore, initCollector, isRunning, lock, start, stop, toString, unlock

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

FilesystemCollector

public FilesystemCollector()

Creates a non-configured Filesystem collector.

FilesystemCollector

public FilesystemCollector(FilesystemCollectorConfig collectorConfig)

Creates and configure an HTTP Collector with the provided configuration.

Parameters:

collectorConfig - HTTP Collector configuration

Method Details

main

public static void main(String[] args)

Invokes the Filesystem Collector from the command line.

Parameters:

args - Invoke it once without any arguments to get a list of command-line options.

getCollectorConfig

public FilesystemCollectorConfig getCollectorConfig()

Overrides:

getCollectorConfig in class Collector

createCrawler

protected Crawler createCrawler(CrawlerConfig config)

Specified by:

createCrawler in class Collector