public abstract class AbstractMappedCommitter extends AbstractBatchCommitter implements com.norconex.commons.lang.config.IXMLConfigurable
A base class batching documents and offering mappings of source reference and source content fields to target reference and target content fields. Batched documents are queued on the file system.
By default the document reference from the source document comes from
document reference value passed to the Committer (obtained internally
using IAddOperation.getReference()
or IDeleteOperation.getReference()
).
If you wish to ignore that original document
reference and use a metadata field instead, use the
setSourceReferenceField(String)
method to do so.
The default (or constant) target reference field is for subclasses to define.
When both a source and target reference
fields are defined, the source reference field will be deleted unless the
keepSourceReferenceField
attribute is set to true
.
Content typically only occurs when committing additions.
The default source document content is the actual document content
(obtained internally using IAddOperation.getContentStream()
).
Defining a sourceContentField
will use the matching metadata property instead.
The default (or constant) target content field is for subclasses to define.
When both a source and target content fields are defined, the
source content field will be deleted unless the
keepSourceContentField
attribute is set to
true
.
As of 2.1.0, XML configuration entries expecting millisecond durations
can be provided in human-readable format (English only), as per
DurationParser
(e.g., "5 minutes and 30 seconds" or "5m30s").
Subclasses implementing IXMLConfigurable
should allow this inner
configuration:
<sourceReferenceField keep="[false|true]"> (Optional name of field that contains the document reference, when the default document reference is not used. The reference value will be mapped to the "targetReferenceField" specified or target repository default field if one is defined by the concrete implementation. Once re-mapped, this metadata source field is deleted, unless "keep" is set totrue
.) </sourceReferenceField> <targetReferenceField> (Name of target repository field where to store a document reference. If not specified, behavior is defined by the concrete implementation.) </targetReferenceField> <sourceContentField keep="[false|true]"> (If you wish to use a metadata field to act as the document "content", you can specify that field here. Default does not take a metadata field but rather the document content. Once re-mapped, the metadata source field is deleted, unless "keep" is set totrue
.) </sourceContentField> <targetContentField> (Target repository field name for a document content/body. Default is defined by concrete implementation.) </targetContentField> <commitBatchSize> (max number of documents to send to target repository at once) </commitBatchSize> <queueDir>(optional path where to queue files)</queueDir> <queueSize>(max queue size before committing)</queueSize> <maxRetries>(max retries upon commit failures)</maxRetries> <maxRetryWait>(max delay in milliseconds between retries)</maxRetryWait>
DEFAULT_COMMIT_BATCH_SIZE
DEFAULT_QUEUE_DIR, filesCommitting
DEFAULT_QUEUE_SIZE, queueSize
Constructor and Description |
---|
AbstractMappedCommitter()
Creates a new instance.
|
AbstractMappedCommitter(int commitBatchSize)
Creates a new instance with given commit batch size.
|
Modifier and Type | Method and Description |
---|---|
boolean |
equals(Object obj) |
String |
getSourceContentField()
Gets the source field name holding the document content.
|
String |
getSourceReferenceField()
Gets the source field name holding the unique identifier.
|
String |
getTargetContentField()
Gets the target field where to store the document content.
|
String |
getTargetReferenceField()
Gets the target field name to store the unique identifier.
|
int |
hashCode() |
boolean |
isKeepSourceContentField()
Whether to keep the content source field or not, once mapped.
|
boolean |
isKeepSourceReferenceField()
Whether to keep the reference source field or not, once mapped.
|
void |
loadFromXML(Reader in) |
protected abstract void |
loadFromXml(XMLConfiguration xml)
Allows subclasses to load their config from xml
|
protected void |
prepareCommitAddition(IAddOperation operation)
Optionally performs actions on a document to be added before
actually committing it.
|
void |
saveToXML(Writer out) |
protected abstract void |
saveToXML(XMLStreamWriter writer)
Allows subclasses to write their config to xml
|
void |
setKeepSourceContentField(boolean keepSourceContentField)
Sets whether to keep the content source field or not, once mapped.
|
void |
setKeepSourceReferenceField(boolean keepSourceReferenceField)
Sets whether to keep the ID source field or not, once mapped.
|
void |
setSourceContentField(String sourceContentField)
Sets the source field name holding the document content.
|
void |
setSourceReferenceField(String sourceReferenceField)
sets the source field name holding the unique identifier.
|
void |
setTargetContentField(String targetContentField)
Sets the target field where to store the document content.
|
void |
setTargetReferenceField(String targetReferenceField)
Sets the target field name to store the unique identifier.
|
String |
toString() |
commitAddition, commitBatch, commitComplete, commitDeletion, getCommitBatchSize, getMaxRetries, getMaxRetryWait, setCommitBatchSize, setMaxRetries, setMaxRetryWait
commit, getInitialQueueDocCount, getQueueDir, prepareCommitDeletion, queueAddition, queueRemoval, setQueueDir
add, getQueueSize, remove, setQueueSize
public AbstractMappedCommitter()
public AbstractMappedCommitter(int commitBatchSize)
commitBatchSize
- commit batch sizepublic String getSourceReferenceField()
public void setSourceReferenceField(String sourceReferenceField)
sourceReferenceField
- source field namepublic String getTargetReferenceField()
public void setTargetReferenceField(String targetReferenceField)
targetReferenceField
- target field namepublic String getTargetContentField()
public void setTargetContentField(String targetContentField)
targetContentField
- target field namepublic String getSourceContentField()
public void setSourceContentField(String sourceContentField)
sourceContentField
- source field namepublic boolean isKeepSourceReferenceField()
true
when keeping source reference fieldpublic void setKeepSourceReferenceField(boolean keepSourceReferenceField)
keepSourceReferenceField
- true
when keeping
source reference fieldpublic boolean isKeepSourceContentField()
true
when keeping content source fieldpublic void setKeepSourceContentField(boolean keepSourceContentField)
keepSourceContentField
- true
when keeping content
source fieldprotected void prepareCommitAddition(IAddOperation operation) throws IOException
AbstractFileQueueCommitter
prepareCommitAddition
in class AbstractFileQueueCommitter
operation
- addition to be performedIOException
- problem preparing commit additionpublic void saveToXML(Writer out) throws IOException
saveToXML
in interface com.norconex.commons.lang.config.IXMLConfigurable
IOException
protected abstract void saveToXML(XMLStreamWriter writer) throws XMLStreamException
writer
- the xml being writtenXMLStreamException
- problem saving to XMLpublic void loadFromXML(Reader in)
loadFromXML
in interface com.norconex.commons.lang.config.IXMLConfigurable
protected abstract void loadFromXml(XMLConfiguration xml)
xml
- XML configurationpublic int hashCode()
hashCode
in class AbstractBatchCommitter
public boolean equals(Object obj)
equals
in class AbstractBatchCommitter
public String toString()
toString
in class AbstractBatchCommitter
Copyright © 2009–2020 Norconex Inc.. All rights reserved.