com.norconex.collector.http.url

Interface ILinkExtractor

All Known Implementing Classes:

GenericLinkExtractor, RegexLinkExtractor, TikaLinkExtractor, XMLFeedLinkExtractor

public interface ILinkExtractor

Responsible for finding links in documents. Links are URLs to be followed with possibly contextual information about that URL (the "a" tag attributes, and text).

Implementing classes also implementing IXMLConfigurable should make sure to name their XML tag "extractor", normally nested in linkExtractors tags.

Author:

Pascal Essiembre

Method Summary

Modifier and Type	Method and Description
boolean	accepts(String url, ContentType contentType) Whether this link extraction should be executed for the given URL and/or content type.
Set<Link>	extractLinks(InputStream input, String reference, ContentType contentType) Extracts links from a document.

Method Detail

extractLinks

Set<Link> extractLinks(InputStream input,
                       String reference,
                       ContentType contentType)
                throws IOException

Extracts links from a document.

Parameters:

input - the document input stream

reference - document reference (URL)

contentType - the document content type

Returns:

a set of links

Throws:

IOException - problem reading the document

accepts

boolean accepts(String url,
                ContentType contentType)

Whether this link extraction should be executed for the given URL and/or content type.

Parameters:

url - the url

contentType - the content type

Returns:

true if the given URL is accepted