public abstract class AbstractStringTagger extends AbstractCharStreamTagger
Base class to facilitate creating taggers based on text content, loading
text into StringBuilder
for memory processing.
Since 2.2.0 this class limits the memory used for analysing
content by reading one section of text at a time. Each
sections are sent for tagging once they are read,
so that no two sections exists in memory at once. Sub-classes should
respect this approach. Each of them have a maximum number of characters
equal to the maximum read size defined using setMaxReadSize(int)
.
When none is set, the default read size is defined by
TextReader.DEFAULT_MAX_READ_SIZE
.
An attempt is made to break sections nicely after a paragraph, sentence, or word. When not possible, long text will be cut at a size equal to the maximum read size.
Implementors should be conscious about memory when dealing with the string builder.
Subclasses inherit this IXMLConfigurable
configuration:
<!-- parent tag has this attribute: maxReadSize="(max characters to read at once)" sourceCharset="(character encoding)" --> <restrictTo caseSensitive="[false|true]" field="(name of header/metadata field name to match)" > (regular expression of value to match) </restrictTo> <!-- multiple "restrictTo" tags allowed (only one needs to match) -->
Constructor and Description |
---|
AbstractStringTagger() |
Modifier and Type | Method and Description |
---|---|
boolean |
equals(Object obj) |
int |
getMaxReadSize()
Gets the maximum number of characters to read from content for tagging
at once.
|
int |
hashCode() |
protected void |
loadCharStreamTaggerFromXML(org.apache.commons.configuration.XMLConfiguration xml)
Loads configuration settings specific to the implementing class.
|
protected abstract void |
loadStringTaggerFromXML(org.apache.commons.configuration.XMLConfiguration xml)
Loads configuration settings specific to the implementing class.
|
protected void |
saveCharStreamTaggerToXML(EnhancedXMLStreamWriter writer)
Saves configuration settings specific to the implementing class.
|
protected abstract void |
saveStringTaggerToXML(EnhancedXMLStreamWriter writer)
Saves configuration settings specific to the implementing class.
|
void |
setMaxReadSize(int maxReadSize)
Sets the maximum number of characters to read from content for tagging
at once.
|
protected abstract void |
tagStringContent(String reference,
StringBuilder content,
ImporterMetadata metadata,
boolean parsed,
int sectionIndex) |
protected void |
tagTextDocument(String reference,
Reader input,
ImporterMetadata metadata,
boolean parsed) |
String |
toString() |
getSourceCharset, loadHandlerFromXML, saveHandlerToXML, setSourceCharset, tagApplicableDocument
tagDocument
addRestriction, addRestriction, addRestrictions, clearRestrictions, detectCharsetIfBlank, getRestrictions, isApplicable, loadFromXML, removeRestriction, removeRestriction, saveToXML
protected final void tagTextDocument(String reference, Reader input, ImporterMetadata metadata, boolean parsed) throws ImporterHandlerException
tagTextDocument
in class AbstractCharStreamTagger
ImporterHandlerException
public int getMaxReadSize()
TextReader.DEFAULT_MAX_READ_SIZE
.public void setMaxReadSize(int maxReadSize)
maxReadSize
- maximum read sizeprotected abstract void tagStringContent(String reference, StringBuilder content, ImporterMetadata metadata, boolean parsed, int sectionIndex) throws ImporterHandlerException
ImporterHandlerException
protected final void saveCharStreamTaggerToXML(EnhancedXMLStreamWriter writer) throws XMLStreamException
AbstractCharStreamTagger
saveCharStreamTaggerToXML
in class AbstractCharStreamTagger
writer
- the xml writerXMLStreamException
- could not save to XMLprotected abstract void saveStringTaggerToXML(EnhancedXMLStreamWriter writer) throws XMLStreamException
writer
- the xml writerXMLStreamException
- could not save to XMLprotected final void loadCharStreamTaggerFromXML(org.apache.commons.configuration.XMLConfiguration xml) throws IOException
AbstractCharStreamTagger
loadCharStreamTaggerFromXML
in class AbstractCharStreamTagger
xml
- xml configurationIOException
- could not load from XMLprotected abstract void loadStringTaggerFromXML(org.apache.commons.configuration.XMLConfiguration xml) throws IOException
xml
- xml configurationIOException
- could not load from XMLpublic boolean equals(Object obj)
equals
in class AbstractCharStreamTagger
public int hashCode()
hashCode
in class AbstractCharStreamTagger
public String toString()
toString
in class AbstractCharStreamTagger
Copyright © 2009–2021 Norconex Inc.. All rights reserved.