com.norconex.collector.fs

Class FilesystemCollector

java.lang.Object

com.norconex.collector.core.AbstractCollector

com.norconex.collector.fs.FilesystemCollector

All Implemented Interfaces:

com.norconex.collector.core.ICollector, IJobSuiteFactory

public class FilesystemCollector
extends com.norconex.collector.core.AbstractCollector

Main application class. In order to use it properly, you must first configure it, either by providing a populated instance of FilesystemCollectorConfig, or by XML configuration, loaded using CollectorConfigLoader. Instances of this class can hold several crawler, running at once. This is convenient when there are configuration setting to be shared amongst crawlers. When you have many crawler jobs defined that have nothing in common, it may be best to configure and run them separately, to facilitate troubleshooting. There is no fair rule for this, experimentation will help you.

Author:

Pascal Essiembre

Constructor Summary

Constructors

Constructor and Description
FilesystemCollector() Creates a non-configured Filesystem collector.
FilesystemCollector(FilesystemCollectorConfig collectorConfig) Creates and configure an HTTP Collector with the provided configuration.

Method Summary

Modifier and Type	Method and Description
protected com.norconex.collector.core.crawler.ICrawler	createCrawler(com.norconex.collector.core.crawler.ICrawlerConfig config)
FilesystemCollectorConfig	getCollectorConfig()
static void	main(String[] args) Invokes the Filesystem Collector from the command line.

Methods inherited from class com.norconex.collector.core.AbstractCollector

createJobSuite, getCrawlers, getId, getJobSuite, getState, setCrawlers, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FilesystemCollector

public FilesystemCollector()

Creates a non-configured Filesystem collector.

FilesystemCollector

public FilesystemCollector(FilesystemCollectorConfig collectorConfig)

Creates and configure an HTTP Collector with the provided configuration.

Parameters:

collectorConfig - HTTP Collector configuration

Method Detail

main

public static void main(String[] args)

Invokes the Filesystem Collector from the command line.

Parameters:

args - Invoke it once without any arguments to get a list of command-line options.

getCollectorConfig

public FilesystemCollectorConfig getCollectorConfig()

Specified by:

getCollectorConfig in interface com.norconex.collector.core.ICollector

Overrides:

getCollectorConfig in class com.norconex.collector.core.AbstractCollector

createCrawler

protected com.norconex.collector.core.crawler.ICrawler createCrawler(com.norconex.collector.core.crawler.ICrawlerConfig config)

Specified by:

createCrawler in class com.norconex.collector.core.AbstractCollector