com.norconex.collector.http.filter.impl

Class SegmentCountURLFilter

java.lang.Object

com.norconex.collector.http.filter.impl.SegmentCountURLFilter

All Implemented Interfaces:

IDocumentFilter, IMetadataFilter, IReferenceFilter, IXMLConfigurable, IOnMatchFilter

public class SegmentCountURLFilter
extends Object
implements IOnMatchFilter, IReferenceFilter, IDocumentFilter, IMetadataFilter, IXMLConfigurable

Filters URL based based on the number of URL segments. A URL with a number of segments equal or more than the specified count will either be included or excluded, as specified.

By default segments are obtained by breaking the URL text at each forward slashes (/), starting after the host name. You can define different or additional segment separator characters.

When duplicate is true, it will count the maximum number of duplicate segments found.

XML configuration usage:

<filter
    class="com.norconex.collector.http.filter.impl.SegmentCountURLFilter"
    onMatch="[include|exclude]"
    count="(numeric value)"
    duplicate="[false|true]"
    separator="(a regex identifying segment separator)"/>

XML usage example:

<filter
    class="SegmentCountURLFilter"
    onMatch="exclude"
    count="5"/>

The above example will reject URLs with more than 5 forward slashes after the domain.

Since:

1.2

Author:

Pascal Essiembre

See Also:

Pattern

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_SEGMENT_COUNT Default segment count.
static String	DEFAULT_SEGMENT_SEPARATOR_PATTERN Default segment separator pattern.

Constructor Summary

Constructors

Constructor and Description
SegmentCountURLFilter() Constructor.
SegmentCountURLFilter(int count) Constructor.
SegmentCountURLFilter(int count, OnMatch onMatch) Constructor.
SegmentCountURLFilter(int count, OnMatch onMatch, boolean duplicate) Constructor.

Method Summary

Modifier and Type	Method and Description
boolean	acceptDocument(Doc document)
boolean	acceptMetadata(String reference, Properties metadata)
boolean	acceptReference(String url)
boolean	equals(Object other)
int	getCount()
OnMatch	getOnMatch()
String	getSeparator() Gets the segment separator pattern
int	hashCode()
boolean	isDuplicate()
void	loadFromXML(XML xml)
void	saveToXML(XML xml)
void	setCount(int count)
void	setDuplicate(boolean duplicate)
void	setOnMatch(OnMatch onMatch)
void	setSeparator(String separator)
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_SEGMENT_SEPARATOR_PATTERN

public static final String DEFAULT_SEGMENT_SEPARATOR_PATTERN

Default segment separator pattern.

See Also:

Constant Field Values

DEFAULT_SEGMENT_COUNT

public static final int DEFAULT_SEGMENT_COUNT

Default segment count.

See Also:

Constant Field Values

Constructor Detail

SegmentCountURLFilter

public SegmentCountURLFilter()

Constructor.

SegmentCountURLFilter

public SegmentCountURLFilter(int count)

Constructor.

Parameters:

count - how many segment

SegmentCountURLFilter

public SegmentCountURLFilter(int count,
                             OnMatch onMatch)

Constructor.

Parameters:

count - how many segment

onMatch - what to do on match

SegmentCountURLFilter

public SegmentCountURLFilter(int count,
                             OnMatch onMatch,
                             boolean duplicate)

Constructor.

Parameters:

count - how many segment

onMatch - what to do on match

duplicate - whether to handle duplicates

Method Detail

getSeparator

public String getSeparator()

Gets the segment separator pattern

Returns:

the pattern

setSeparator

public final void setSeparator(String separator)

getCount

public int getCount()

setCount

public final void setCount(int count)

isDuplicate

public boolean isDuplicate()

setDuplicate

public final void setDuplicate(boolean duplicate)

getOnMatch

public OnMatch getOnMatch()

Specified by:

getOnMatch in interface IOnMatchFilter

setOnMatch

public void setOnMatch(OnMatch onMatch)

acceptDocument

public boolean acceptDocument(Doc document)

Specified by:

acceptDocument in interface IDocumentFilter

acceptMetadata

public boolean acceptMetadata(String reference,
                              Properties metadata)

Specified by:

acceptMetadata in interface IMetadataFilter

acceptReference

public boolean acceptReference(String url)

Specified by:

acceptReference in interface IReferenceFilter

loadFromXML

public void loadFromXML(XML xml)

Specified by:

loadFromXML in interface IXMLConfigurable

saveToXML

public void saveToXML(XML xml)

Specified by:

saveToXML in interface IXMLConfigurable

equals

public boolean equals(Object other)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object