Interface IHttpFetcher
-
- All Known Implementing Classes:
AbstractHttpFetcher,GenericHttpFetcher,PhantomJSDocumentFetcher,WebDriverHttpFetcher
public interface IHttpFetcherFetches HTTP resources.- Since:
- 3.0.0
- Author:
- Pascal Essiembre
-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description booleanaccept(Doc doc, HttpMethod httpMethod)IHttpFetchResponsefetch(CrawlDoc doc, HttpMethod httpMethod)Performs an HTTP request for the supplied document reference and HTTP method.StringgetUserAgent()
-
-
-
Method Detail
-
getUserAgent
String getUserAgent()
-
accept
boolean accept(Doc doc, HttpMethod httpMethod)
-
fetch
IHttpFetchResponse fetch(CrawlDoc doc, HttpMethod httpMethod) throws HttpFetchException
Performs an HTTP request for the supplied document reference and HTTP method.
For each HTTP method supported, implementors should do their best to populate the document and its
CrawlDocInfowith as much information they can.Unsupported HTTP methods should return an HTTP response with the
CrawlState.UNSUPPORTEDstate. To prevent userse having to configure multiple HTTP clients, implementors should try to support both theGETandHEADmethods. POST is only used in special cases and is often not used during a crawl session.A
nullmethod is treated as aGET.- Parameters:
doc- document to fetch or to use to make the request.httpMethod- HTTP method- Returns:
- an HTTP response
- Throws:
HttpFetchException- problem when fetching the document- See Also:
HttpFetchResponseBuilder.unsupported()
-
-