com.norconex.collector.http.sitemap.impl

Class StandardSitemapResolverFactory

java.lang.Object

com.norconex.collector.http.sitemap.impl.StandardSitemapResolverFactory

All Implemented Interfaces:

ISitemapResolverFactory, IXMLConfigurable

public class StandardSitemapResolverFactory
extends Object
implements ISitemapResolverFactory, IXMLConfigurable

Factory used to created StandardSitemapResolver instances. Refer to StandardSitemapResolver for resolution logic.

XML configuration usage:

  <sitemapResolverFactory
     ignore="[false|true]"
     lenient="[false|true]"
     escalateErrors="[false|true]"
     fromDate="(Optional EPOCH date, in milliseconds)"
     class="com.norconex.collector.http.sitemap.impl.StandardSitemapResolverFactory">
     <tempDir>(where to store temp files)</tempDir>
     <path>
       (Optional path relative to URL root for a sitemap. Use a single empty
        "path" tag to rely instead on any sitemaps specified as start URLs or
        defined in robots.txt, if enabled.  Not specifying any path tags
        falls back to trying to locate sitemaps using default paths.)
     </path>
     (... repeat path tag as needed ...)
  </sitemapResolverFactory>
 

Usage example:

The following ignores sitemap files present on web sites.

  <sitemapResolverFactory ignore="true"/>
 

Author:

Pascal Essiembre

See Also:

StandardSitemapResolver

Constructor Summary

Constructors

Constructor and Description
StandardSitemapResolverFactory()

Method Summary

Modifier and Type	Method and Description
ISitemapResolver	createSitemapResolver(HttpCrawlerConfig config, boolean resume)
boolean	equals(Object other)
long	getFromDate() Gets the minimum EPOCH date (in milliseconds) a sitemap entry should have to be considered.
String[]	getSitemapLocations() Deprecated. Since 2.3.0, use HttpCrawlerConfig.getStartSitemapURLs()
String[]	getSitemapPaths() Gets the URL paths, relative to the URL root, from which to try locate and resolve sitemaps.
File	getTempDir() Gets the directory where sitemap files are temporary stored before they are parsed.
int	hashCode()
boolean	isEscalateErrors() Gets whether errors should be thrown instead of logged.
boolean	isLenient()
void	loadFromXML(Reader in)
void	saveToXML(Writer out)
void	setEscalateErrors(boolean escalateErrors) Sets whether errors should be thrown instead of logged.
void	setFromDate(long fromDate) Sets the minimum EPOCH date (in milliseconds) a sitemap entry should have to be considered.
void	setLenient(boolean lenient)
void	setSitemapLocations(String... sitemapLocations) Deprecated. Since 2.3.0, use HttpCrawlerConfig.setStartSitemapURLs(String[])
void	setSitemapPaths(String... sitemapPaths) Sets the URL paths, relative to the URL root, from which to try locate and resolve sitemaps.
void	setTempDir(File tempDir) Sets the temporary directory where sitemap files are temporary stored before they are parsed.
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

StandardSitemapResolverFactory

public StandardSitemapResolverFactory()

Method Detail

createSitemapResolver

public ISitemapResolver createSitemapResolver(HttpCrawlerConfig config,
                                              boolean resume)

Specified by:

createSitemapResolver in interface ISitemapResolverFactory

getSitemapPaths

public String[] getSitemapPaths()

Gets the URL paths, relative to the URL root, from which to try locate and resolve sitemaps. Default paths are "/sitemap.xml" and "/sitemap-index.xml".

Returns:

sitemap paths.

Since:

2.3.0

setSitemapPaths

public void setSitemapPaths(String... sitemapPaths)

Sets the URL paths, relative to the URL root, from which to try locate and resolve sitemaps.

Parameters:

sitemapPaths - sitemap paths.

Since:

2.3.0

getSitemapLocations

@Deprecated
public String[] getSitemapLocations()

Deprecated. Since 2.3.0, use HttpCrawlerConfig.getStartSitemapURLs()

Get sitemap locations.

Returns:

sitemap locations

setSitemapLocations

@Deprecated
public void setSitemapLocations(String... sitemapLocations)

Deprecated. Since 2.3.0, use HttpCrawlerConfig.setStartSitemapURLs(String[])

Set sitemap locations.

Parameters:

sitemapLocations - sitemap locations

isLenient

public boolean isLenient()

setLenient

public void setLenient(boolean lenient)

getFromDate

public long getFromDate()

Gets the minimum EPOCH date (in milliseconds) a sitemap entry should have to be considered.

Returns:

from date

Since:

2.9.1

setFromDate

public void setFromDate(long fromDate)

Sets the minimum EPOCH date (in milliseconds) a sitemap entry should have to be considered.

Parameters:

fromDate - from date

Since:

2.9.1

isEscalateErrors

public boolean isEscalateErrors()

Gets whether errors should be thrown instead of logged.

Returns:

true if throwing errors

Since:

2.9.1

setEscalateErrors

public void setEscalateErrors(boolean escalateErrors)

Sets whether errors should be thrown instead of logged.

Parameters:

escalateErrors - true if throwing errors

Since:

2.9.1

getTempDir

public File getTempDir()

Gets the directory where sitemap files are temporary stored before they are parsed. When null (default), temporary files are created directly under AbstractCrawlerConfig.getWorkDir(). the crawler working directory is also undefined, it will use the system temporary directory, as returned by FileUtils.getTempDirectory().

Returns:

directory where temporary files are written

Since:

2.3.0

setTempDir

public void setTempDir(File tempDir)

Sets the temporary directory where sitemap files are temporary stored before they are parsed.

Parameters:

tempDir - directory where temporary files are written

Since:

2.3.0

loadFromXML

public void loadFromXML(Reader in)
                 throws IOException

Specified by:

loadFromXML in interface IXMLConfigurable

Throws:

IOException

saveToXML

public void saveToXML(Writer out)
               throws IOException

Specified by:

saveToXML in interface IXMLConfigurable

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

equals

public boolean equals(Object other)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object