com.norconex.collector.http.data

Class HttpCrawlData

java.lang.Object

com.norconex.collector.core.data.BaseCrawlData

com.norconex.collector.http.data.HttpCrawlData

All Implemented Interfaces:

ICrawlData, Serializable, Cloneable

public class HttpCrawlData
extends BaseCrawlData

A URL being crawled holding relevant crawl information.

Author:

Pascal Essiembre

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
HttpCrawlData() Constructor.
HttpCrawlData(ICrawlData crawlData) Constructor
HttpCrawlData(String url, int depth) Constructor.

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object other)
int	getDepth() Gets the URL depth.
String	getOriginalReference()
String[]	getRedirectTrail() Gets the trail of URLs that were redirected up to this one.
String[]	getReferencedUrls() Gets URLs referenced by this one.
String	getReferrerLinkTag()
String	getReferrerLinkText()
String	getReferrerLinkTitle()
String	getReferrerReference()
String	getSitemapChangeFreq() Gets the sitemap change frequency.
Long	getSitemapLastMod() Gets the sitemap last modified date in milliseconds (EPOCH date).
Float	getSitemapPriority() Gets the sitemap priority.
String	getUrlRoot() Gets the URL root (protocol + domain, e.g.
int	hashCode()
void	setDepth(int depth) Sets the URL depth.
void	setOriginalReference(String originalReference)
void	setRedirectTrail(String... redirectTrail) Sets the trail of URLs that were redirected up to this one.
void	setReference(String url)
void	setReferencedUrls(String... referencedUrls) Sets URLs referenced by this one.
void	setReferrerLinkTag(String referrerLinkTag)
void	setReferrerLinkText(String referrerLinkText)
void	setReferrerLinkTitle(String referrerLinkTitle)
void	setReferrerReference(String referrerReference)
void	setSitemapChangeFreq(String sitemapChangeFreq) Sets the sitemap change frequency.
void	setSitemapLastMod(Long sitemapLastMod) Sets the sitemap last modified date in milliseconds (EPOCH date).
void	setSitemapPriority(Float sitemapPriority) Sets the sitemap priority.
String	toString()

Methods inherited from class com.norconex.collector.core.data.BaseCrawlData

clone, getContentChecksum, getContentType, getCrawlDate, getMetaChecksum, getParentRootReference, getReference, getState, isRootParentReference, setContentChecksum, setContentType, setCrawlDate, setMetaChecksum, setParentRootReference, setRootParentReference, setState

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

HttpCrawlData

public HttpCrawlData()

Constructor.

HttpCrawlData

public HttpCrawlData(ICrawlData crawlData)

Constructor

Parameters:

crawlData - initialized this instance this data

HttpCrawlData

public HttpCrawlData(String url,
                     int depth)

Constructor.

Parameters:

url - URL being crawled

depth - URL depth

Method Detail

getOriginalReference

public String getOriginalReference()

setOriginalReference

public void setOriginalReference(String originalReference)

getDepth

public int getDepth()

Gets the URL depth.

Returns:

URL depth

getSitemapLastMod

public Long getSitemapLastMod()

Gets the sitemap last modified date in milliseconds (EPOCH date).

Returns:

date as long

setSitemapLastMod

public void setSitemapLastMod(Long sitemapLastMod)

Sets the sitemap last modified date in milliseconds (EPOCH date).

Parameters:

sitemapLastMod - date as long

getSitemapChangeFreq

public String getSitemapChangeFreq()

Gets the sitemap change frequency.

Returns:

sitemap change frequency

setSitemapChangeFreq

public void setSitemapChangeFreq(String sitemapChangeFreq)

Sets the sitemap change frequency.

Parameters:

sitemapChangeFreq - sitemap change frequency

getSitemapPriority

public Float getSitemapPriority()

Gets the sitemap priority.

Returns:

sitemap priority

setSitemapPriority

public void setSitemapPriority(Float sitemapPriority)

Sets the sitemap priority.

Parameters:

sitemapPriority - sitemap priority

setDepth

public final void setDepth(int depth)

Sets the URL depth.

Parameters:

depth - URL depth

getReferrerLinkText

public String getReferrerLinkText()

setReferrerLinkText

public void setReferrerLinkText(String referrerLinkText)

getReferrerReference

public String getReferrerReference()

setReferrerReference

public void setReferrerReference(String referrerReference)

getReferrerLinkTag

public String getReferrerLinkTag()

setReferrerLinkTag

public void setReferrerLinkTag(String referrerLinkTag)

getReferrerLinkTitle

public String getReferrerLinkTitle()

setReferrerLinkTitle

public void setReferrerLinkTitle(String referrerLinkTitle)

setReference

public final void setReference(String url)

Overrides:

setReference in class BaseCrawlData

getUrlRoot

public String getUrlRoot()

Gets the URL root (protocol + domain, e.g. http://www.host.com).

Returns:

URL root

getReferencedUrls

public String[] getReferencedUrls()

Gets URLs referenced by this one.

Returns:

URLs referenced by this one.

Since:

2.6.0

setReferencedUrls

public void setReferencedUrls(String... referencedUrls)

Sets URLs referenced by this one.

Parameters:

referencedUrls - referenced URLs

Since:

2.6.0

getRedirectTrail

public String[] getRedirectTrail()

Gets the trail of URLs that were redirected up to this one.

Returns:

URL redirection trail to this one

Since:

2.8.0

setRedirectTrail

public void setRedirectTrail(String... redirectTrail)

Sets the trail of URLs that were redirected up to this one.

Parameters:

redirectTrail - URL redirection trail to this one

Since:

2.8.0

equals

public boolean equals(Object other)

Overrides:

equals in class BaseCrawlData

hashCode

public int hashCode()

Overrides:

hashCode in class BaseCrawlData

toString

public String toString()

Overrides:

toString in class BaseCrawlData