Package | Description |
---|---|
com.norconex.collector.http.pipeline.importer | |
com.norconex.collector.http.robot | |
com.norconex.collector.http.robot.impl |
Modifier and Type | Method and Description |
---|---|
RobotsMeta |
HttpImporterPipelineContext.getRobotsMeta() |
Modifier and Type | Method and Description |
---|---|
void |
HttpImporterPipelineContext.setRobotsMeta(RobotsMeta robotsMeta) |
Modifier and Type | Method and Description |
---|---|
RobotsMeta |
IRobotsMetaProvider.getRobotsMeta(Reader document,
String documentUrl,
ContentType contentType,
Properties httpHeaders)
Extracts Robots meta information for a page, if any.
|
Modifier and Type | Method and Description |
---|---|
RobotsMeta |
StandardRobotsMetaProvider.getRobotsMeta(Reader document,
String documentUrl,
ContentType contentType,
Properties httpHeaders) |
Copyright © 2009–2021 Norconex Inc.. All rights reserved.