Uses of Interface
com.norconex.collector.http.robot.IRobotsTxtProvider

Packages that use IRobotsTxtProvider

Package	Description
com.norconex.collector.http.crawler
com.norconex.collector.http.robot.impl

Uses of IRobotsTxtProvider in com.norconex.collector.http.crawler

Methods in com.norconex.collector.http.crawler that return IRobotsTxtProvider

Modifier and Type	Method	Description
IRobotsTxtProvider	HttpCrawlerConfig.getRobotsTxtProvider()

Methods in com.norconex.collector.http.crawler with parameters of type IRobotsTxtProvider

Modifier and Type	Method	Description
void	HttpCrawlerConfig.setRobotsTxtProvider(IRobotsTxtProvider robotsTxtProvider)

Uses of IRobotsTxtProvider in com.norconex.collector.http.robot.impl

Classes in com.norconex.collector.http.robot.impl that implement IRobotsTxtProvider

Modifier and Type	Class	Description
class	StandardRobotsTxtProvider	Implementation of IRobotsTxtProvider as per the robots.txt standard described at http://www.robotstxt.org/robotstxt.html.