com.norconex.importer.handler.tagger

Class AbstractStringTagger

java.lang.Object

com.norconex.importer.handler.AbstractImporterHandler

com.norconex.importer.handler.tagger.AbstractDocumentTagger

com.norconex.importer.handler.tagger.AbstractCharStreamTagger

com.norconex.importer.handler.tagger.AbstractStringTagger

All Implemented Interfaces:

IXMLConfigurable, IImporterHandler, IDocumentTagger

Direct Known Subclasses:

CountMatchesTagger, LanguageTagger, ScriptTagger, TextBetweenTagger, TextPatternTagger, TitleGeneratorTagger

public abstract class AbstractStringTagger
extends AbstractCharStreamTagger

Base class to facilitate creating taggers based on text content, loading text into StringBuilder for memory processing.

Since 2.2.0 this class limits the memory used for analysing content by reading one section of text at a time. Each sections are sent for tagging once they are read, so that no two sections exists in memory at once. Sub-classes should respect this approach. Each of them have a maximum number of characters equal to the maximum read size defined using setMaxReadSize(int). When none is set, the default read size is defined by TextReader.DEFAULT_MAX_READ_SIZE.

An attempt is made to break sections nicely after a paragraph, sentence, or word. When not possible, long text will be cut at a size equal to the maximum read size.

Implementors should be conscious about memory when dealing with the string builder.

Subclasses inherit this IXMLConfigurable configuration:

  <!-- parent tag has this attribute: 
      maxReadSize="(max characters to read at once)"
      sourceCharset="(character encoding)"
    -->
  <restrictTo caseSensitive="[false|true]"
          field="(name of header/metadata field name to match)" >
      (regular expression of value to match)
  </restrictTo>
  <!-- multiple "restrictTo" tags allowed (only one needs to match) -->
 

Author:

Pascal Essiembre

Constructor Summary

Constructors

Constructor and Description
AbstractStringTagger()

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object obj)
int	getMaxReadSize() Gets the maximum number of characters to read from content for tagging at once.
int	hashCode()
protected void	loadCharStreamTaggerFromXML(org.apache.commons.configuration.XMLConfiguration xml) Loads configuration settings specific to the implementing class.
protected abstract void	loadStringTaggerFromXML(org.apache.commons.configuration.XMLConfiguration xml) Loads configuration settings specific to the implementing class.
protected void	saveCharStreamTaggerToXML(EnhancedXMLStreamWriter writer) Saves configuration settings specific to the implementing class.
protected abstract void	saveStringTaggerToXML(EnhancedXMLStreamWriter writer) Saves configuration settings specific to the implementing class.
void	setMaxReadSize(int maxReadSize) Sets the maximum number of characters to read from content for tagging at once.
protected abstract void	tagStringContent(String reference, StringBuilder content, ImporterMetadata metadata, boolean parsed, int sectionIndex)
protected void	tagTextDocument(String reference, Reader input, ImporterMetadata metadata, boolean parsed)
String	toString()

Methods inherited from class com.norconex.importer.handler.tagger.AbstractCharStreamTagger

getSourceCharset, loadHandlerFromXML, saveHandlerToXML, setSourceCharset, tagApplicableDocument

Methods inherited from class com.norconex.importer.handler.tagger.AbstractDocumentTagger

tagDocument

Methods inherited from class com.norconex.importer.handler.AbstractImporterHandler

addRestriction, addRestriction, addRestrictions, clearRestrictions, detectCharsetIfBlank, getRestrictions, isApplicable, loadFromXML, removeRestriction, removeRestriction, saveToXML

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

AbstractStringTagger

public AbstractStringTagger()

Method Detail

tagTextDocument

protected final void tagTextDocument(String reference,
                                     Reader input,
                                     ImporterMetadata metadata,
                                     boolean parsed)
                              throws ImporterHandlerException

Specified by:

tagTextDocument in class AbstractCharStreamTagger

Throws:

ImporterHandlerException

getMaxReadSize

public int getMaxReadSize()

Gets the maximum number of characters to read from content for tagging at once. Default is TextReader.DEFAULT_MAX_READ_SIZE.

Returns:

maximum read size

setMaxReadSize

public void setMaxReadSize(int maxReadSize)

Sets the maximum number of characters to read from content for tagging at once.

Parameters:

maxReadSize - maximum read size

tagStringContent

protected abstract void tagStringContent(String reference,
                                         StringBuilder content,
                                         ImporterMetadata metadata,
                                         boolean parsed,
                                         int sectionIndex)
                                  throws ImporterHandlerException

Throws:

ImporterHandlerException

saveCharStreamTaggerToXML

protected final void saveCharStreamTaggerToXML(EnhancedXMLStreamWriter writer)
                                        throws XMLStreamException

Description copied from class: AbstractCharStreamTagger

Saves configuration settings specific to the implementing class. The parent tag along with the "class" attribute are already written. Implementors must not close the writer.

Specified by:

saveCharStreamTaggerToXML in class AbstractCharStreamTagger

Parameters:

writer - the xml writer

Throws:

XMLStreamException - could not save to XML

saveStringTaggerToXML

protected abstract void saveStringTaggerToXML(EnhancedXMLStreamWriter writer)
                                       throws XMLStreamException

Saves configuration settings specific to the implementing class. The parent tag along with the "class" attribute are already written. Implementors must not close the writer.

Parameters:

writer - the xml writer

Throws:

XMLStreamException - could not save to XML

loadCharStreamTaggerFromXML

protected final void loadCharStreamTaggerFromXML(org.apache.commons.configuration.XMLConfiguration xml)
                                          throws IOException

Description copied from class: AbstractCharStreamTagger

Loads configuration settings specific to the implementing class.

Specified by:

loadCharStreamTaggerFromXML in class AbstractCharStreamTagger

Parameters:

xml - xml configuration

Throws:

IOException - could not load from XML

loadStringTaggerFromXML

protected abstract void loadStringTaggerFromXML(org.apache.commons.configuration.XMLConfiguration xml)
                                         throws IOException

Loads configuration settings specific to the implementing class.

Parameters:

xml - xml configuration

Throws:

IOException - could not load from XML

equals

public boolean equals(Object obj)

Overrides:

equals in class AbstractCharStreamTagger

hashCode

public int hashCode()

Overrides:

hashCode in class AbstractCharStreamTagger

toString

public String toString()

Overrides:

toString in class AbstractCharStreamTagger