com.norconex.importer.handler.filter

Class AbstractStringFilter

java.lang.Object

com.norconex.importer.handler.AbstractImporterHandler

com.norconex.importer.handler.filter.AbstractDocumentFilter

com.norconex.importer.handler.filter.AbstractCharStreamFilter

com.norconex.importer.handler.filter.AbstractStringFilter

All Implemented Interfaces:

IXMLConfigurable, IDocumentFilter, IOnMatchFilter, IImporterHandler

Direct Known Subclasses:

RegexContentFilter, ScriptFilter

public abstract class AbstractStringFilter
extends AbstractCharStreamFilter

Base class to facilitate creating filters based on text content, loading text into StringBuilder for memory processing.

Since 2.2.0 this class limits the memory used for content filtering by reading one section of text at a time. Each sections are sent for filtering once they are read until a match is found. No two sections exists in memory at once. Sub-classes should respect this approach. Each section have a maximum number of characters equal to the maximum read size defined using setMaxReadSize(int). When none is set, the default read size is defined by TextReader.DEFAULT_MAX_READ_SIZE.

An attempt is made to break sections nicely after a paragraph, sentence, or word. When not possible, long text will be cut at a size equal to the maximum read size.

Implementors should be conscious about memory when dealing with the string builder.

Subclasses inherit this IXMLConfigurable configuration:

  <!-- parent tag has these attributes: 
      maxReadSize="(max characters to read at once)" 
      sourceCharset="(character encoding)"
      onMatch="[include|exclude]"
    -->
  <restrictTo caseSensitive="[false|true]"
          field="(name of header/metadata field name to match)" >
      (regular expression of value to match)
  </restrictTo>
  <!-- multiple "restrictTo" tags allowed (only one needs to match) -->
 

Author:

Pascal Essiembre

Constructor Summary

Constructors

Constructor and Description
AbstractStringFilter()

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object obj)
int	getMaxReadSize() Gets the maximum number of characters to read for filtering at once.
int	hashCode()
protected abstract boolean	isStringContentMatching(String reference, StringBuilder content, ImporterMetadata metadata, boolean parsed, int sectionIndex)
protected boolean	isTextDocumentMatching(String reference, Reader input, ImporterMetadata metadata, boolean parsed)
protected void	loadCharStreamFilterFromXML(org.apache.commons.configuration.XMLConfiguration xml) Loads configuration settings specific to the implementing class.
protected abstract void	loadStringFilterFromXML(org.apache.commons.configuration.XMLConfiguration xml) Loads configuration settings specific to the implementing class.
protected void	saveCharStreamFilterToXML(EnhancedXMLStreamWriter writer) Saves configuration settings specific to the implementing class.
protected abstract void	saveStringFilterToXML(EnhancedXMLStreamWriter writer) Saves configuration settings specific to the implementing class.
void	setMaxReadSize(int maxReadSize) Sets the maximum number of characters to read for filtering at once.
String	toString()

Methods inherited from class com.norconex.importer.handler.filter.AbstractCharStreamFilter

getSourceCharset, isDocumentMatched, loadFilterFromXML, saveFilterToXML, setSourceCharset

Methods inherited from class com.norconex.importer.handler.filter.AbstractDocumentFilter

acceptDocument, getOnMatch, loadHandlerFromXML, saveHandlerToXML, setOnMatch

Methods inherited from class com.norconex.importer.handler.AbstractImporterHandler

addRestriction, addRestriction, addRestrictions, clearRestrictions, detectCharsetIfBlank, getRestrictions, isApplicable, loadFromXML, removeRestriction, removeRestriction, saveToXML

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

AbstractStringFilter

public AbstractStringFilter()

Method Detail

isTextDocumentMatching

protected final boolean isTextDocumentMatching(String reference,
                                               Reader input,
                                               ImporterMetadata metadata,
                                               boolean parsed)
                                        throws ImporterHandlerException

Specified by:

isTextDocumentMatching in class AbstractCharStreamFilter

Throws:

ImporterHandlerException

getMaxReadSize

public int getMaxReadSize()

Gets the maximum number of characters to read for filtering at once. Default is TextReader.DEFAULT_MAX_READ_SIZE.

Returns:

maximum read size

setMaxReadSize

public void setMaxReadSize(int maxReadSize)

Sets the maximum number of characters to read for filtering at once.

Parameters:

maxReadSize - maximum read size

isStringContentMatching

protected abstract boolean isStringContentMatching(String reference,
                                                   StringBuilder content,
                                                   ImporterMetadata metadata,
                                                   boolean parsed,
                                                   int sectionIndex)
                                            throws ImporterHandlerException

Throws:

ImporterHandlerException

saveCharStreamFilterToXML

protected final void saveCharStreamFilterToXML(EnhancedXMLStreamWriter writer)
                                        throws XMLStreamException

Description copied from class: AbstractCharStreamFilter

Saves configuration settings specific to the implementing class. The parent tag along with the "class" attribute are already written. Implementors must not close the writer.

Specified by:

saveCharStreamFilterToXML in class AbstractCharStreamFilter

Parameters:

writer - the xml writer

Throws:

XMLStreamException - could not save to XML

saveStringFilterToXML

protected abstract void saveStringFilterToXML(EnhancedXMLStreamWriter writer)
                                       throws XMLStreamException

Saves configuration settings specific to the implementing class. The parent tag along with the "class" attribute are already written. Implementors must not close the writer.

Parameters:

writer - the xml writer

Throws:

XMLStreamException - could not save to XML

loadCharStreamFilterFromXML

protected final void loadCharStreamFilterFromXML(org.apache.commons.configuration.XMLConfiguration xml)
                                          throws IOException

Description copied from class: AbstractCharStreamFilter

Loads configuration settings specific to the implementing class.

Specified by:

loadCharStreamFilterFromXML in class AbstractCharStreamFilter

Parameters:

xml - xml configuration

Throws:

IOException - could not load from XML

loadStringFilterFromXML

protected abstract void loadStringFilterFromXML(org.apache.commons.configuration.XMLConfiguration xml)
                                         throws IOException

Loads configuration settings specific to the implementing class.

Parameters:

xml - xml configuration

Throws:

IOException - could not load from XML

equals

public boolean equals(Object obj)

Overrides:

equals in class AbstractCharStreamFilter

hashCode

public int hashCode()

Overrides:

hashCode in class AbstractCharStreamFilter

toString

public String toString()

Overrides:

toString in class AbstractCharStreamFilter