Package | Description |
---|---|
com.norconex.collector.http.crawler | |
com.norconex.collector.http.fetch | |
com.norconex.collector.http.fetch.impl | |
com.norconex.collector.http.fetch.impl.webdriver |
Modifier and Type | Method and Description |
---|---|
List<IHttpFetcher> |
HttpCrawlerConfig.getHttpFetchers()
Gets HTTP fetchers.
|
Modifier and Type | Method and Description |
---|---|
void |
HttpCrawlerConfig.setHttpFetchers(IHttpFetcher... httpFetchers)
Sets HTTP fetchers.
|
Modifier and Type | Method and Description |
---|---|
void |
HttpCrawlerConfig.setHttpFetchers(List<IHttpFetcher> httpFetchers)
Sets HTTP fetchers.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractHttpFetcher
Base class implementing the
AbstractHttpFetcher.accept(Doc, HttpMethod) method
using reference filters to determine if this fetcher will accept to fetch
a URL and delegating the HTTP method check to its own
AbstractHttpFetcher.accept(HttpMethod) abstract method. |
Modifier and Type | Method and Description |
---|---|
void |
HttpFetchClientResponse.addResponse(IHttpFetchResponse resp,
IHttpFetcher fetcher) |
Constructor and Description |
---|
HttpFetchClient(CachedStreamFactory streamFactory,
List<IHttpFetcher> httpFetchers,
int maxRetries,
long retryDelay) |
Modifier and Type | Class and Description |
---|---|
class |
GenericHttpFetcher
Default implementation of
IHttpFetcher , based on Apache HttpClient. |
class |
PhantomJSDocumentFetcher
Deprecated.
Since 3.0.0 use
WebDriverHttpFetcher |
Modifier and Type | Class and Description |
---|---|
class |
WebDriverHttpFetcher
Uses Selenium WebDriver support for using native browsers to crawl documents.
|
Copyright © 2009–2023 Norconex Inc.. All rights reserved.