Uses of Interface
com.norconex.collector.http.robot.IRobotsTxtProvider

Packages that use IRobotsTxtProvider

Package	Description
com.norconex.collector.http.crawler
com.norconex.collector.http.robot.impl

Uses of IRobotsTxtProvider in com.norconex.collector.http.crawler

Methods in com.norconex.collector.http.crawler that return IRobotsTxtProvider

Modifier and Type	Method and Description
IRobotsTxtProvider	HttpCrawlerConfig.getRobotsTxtProvider()

Methods in com.norconex.collector.http.crawler with parameters of type IRobotsTxtProvider

Modifier and Type	Method and Description
void	HttpCrawlerConfig.setRobotsTxtProvider(IRobotsTxtProvider robotsTxtProvider)

Uses of IRobotsTxtProvider in com.norconex.collector.http.robot.impl

Classes in com.norconex.collector.http.robot.impl that implement IRobotsTxtProvider

Modifier and Type	Class and Description
class	StandardRobotsTxtProvider Implementation of IRobotsTxtProvider as per the robots.txt standard described at http://www.robotstxt.org/robotstxt.html.