com.norconex.committer.idol

Class IdolCommitter

java.lang.Object

com.norconex.committer.core.AbstractCommitter

com.norconex.committer.core.AbstractFileQueueCommitter

com.norconex.committer.core.AbstractBatchCommitter

com.norconex.committer.core.AbstractMappedCommitter

com.norconex.committer.idol.IdolCommitter

All Implemented Interfaces:

ICommitter, IXMLConfigurable

public class IdolCommitter
extends AbstractMappedCommitter

Commits documents to HP Autonomy IDOL Server/DIH or HP Autonomy Connector Framework Server (CFS). Specifying either the index port or the cfs port determines which of the two will be the documents target.

As of 2.1.1, XML configuration entries expecting millisecond durations can be provided in human-readable format (English only), as per DurationParser (e.g., "5 minutes and 30 seconds" or "5m30s").

XML configuration usage:

   <committer class="com.norconex.committer.idol.IdolCommitter">

      <!-- To commit documents to IDOL or DIH: -->
      <host>(IDOL/DIH host name or IP)</host>
      <indexPort>(IDOL/DIH index port)</indexPort>
      <databaseName>(Optional IDOL Database Name where to store documents)</databaseName>
      <dreAddDataParams>
         <param name="(parameter name)">(parameter value)</param>
      </dreAddDataParams>
      <dreDeleteRefParams>
         <param name="(parameter name)">(parameter value)</param>
      </dreDeleteRefParams>

      <!-- To commit documents to CFS: -->
      <host>(CFS host name or IP)</host>
      <cfsPort>(CFS Server/Ingest port)</cfsPort>

      <!-- Common settings: -->
      <sourceReferenceField keep="[false|true]">
         (Optional name of field that contains the document reference, when
         the default document reference is not used.  The reference value
         will be mapped to the IDOL "DREREFERENCE" field, or the
         "targetReferenceField" specified.
         Once re-mapped, this metadata source field is
         deleted, unless "keep" is set to true.)
      </sourceReferenceField>
      <targetReferenceField>
         (Optional name of IDOL target field where to store the source
         reference. If not specified, default is "DREREFERENCE".)
      </targetReferenceField>
      <sourceContentField keep="[false|true]">
         (If you wish to use a metadata field to act as the document
         "content", you can specify that field here.  Default
         does not take a metadata field but rather the document content.
         Once re-mapped, the metadata source field is deleted,
         unless "keep" is set to true.)
      </sourceContentField>
      <targetContentField>
         (IDOL target field name for a document content/body.
          Default is: DRECONTENT)
      </targetContentField>
      <commitBatchSize>
          (max number of docs to send IDOL at once)
      </commitBatchSize>
      <queueDir>(optional path where to queue files)</queueDir>
      <queueSize>(max queue size before committing)</queueSize>
      <maxRetries>(max retries upon commit failures)</maxRetries>
      <maxRetryWait>(max delay in milliseconds between retries)</maxRetryWait>
   </committer>
 

Author:

Stephen Jacob, Martin Fournier, Pascal Essiembre

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_IDOL_CONTENT_FIELD Default field for content in Autonomy Idol Database.
static String	DEFAULT_IDOL_REFERENCE_FIELD Default key field in Autonomy Idol database.

Fields inherited from class com.norconex.committer.core.AbstractBatchCommitter

DEFAULT_COMMIT_BATCH_SIZE

Fields inherited from class com.norconex.committer.core.AbstractFileQueueCommitter

DEFAULT_QUEUE_DIR, filesCommitting

Fields inherited from class com.norconex.committer.core.AbstractCommitter

DEFAULT_QUEUE_SIZE, queueSize

Constructor Summary

Constructors

Constructor and Description
IdolCommitter() Constructor.

Method Summary

Modifier and Type	Method and Description
void	addDreAddDataParam(String name, String value) Adds a DREADDDATA URL parameter.
void	addDreDeleteRefParam(String name, String value) Adds the DREDELETEREF URL parameter value for the parameter name.
protected void	buildCfsXmlDocument(XMLStreamWriter writer, InputStream is, Properties properties)
protected String	buildIdxDocument(InputStream is, Properties properties) Build an idol document using the idx file format
protected void	commitBatch(List<ICommitOperation> batch)
void	dreAddData(List<IAddOperation> addOperations) Commits the addition operations.
void	dreDeleteRef(List<IDeleteOperation> deleteOperations) Commits the deletion operations
int	getCfsPort() Gets CFS port.
String	getDatabaseName() Gets IDOL database name.
String	getDreAddDataParam(String name) Gets the DREADDDATA URL parameter value for the parameter name.
Set<String>	getDreAddDataParamNames() Gets all DREADDDATA URL parameter names
String	getDreDeleteRefParam(String name) Gets the DREDELETEREF URL parameter value for the parameter name.
Set<String>	getDreDeleteRefParamNames() Gets all DREDELETEREF URL parameter names
String	getHost() Sets IDOL host name.
int	getIndexPort() Gets IDOL index port.
protected void	loadFromXml(XMLConfiguration xml)
protected void	saveToXML(XMLStreamWriter writer)
void	setCfsPort(int cfsPort) Sets CFS port.
void	setDatabaseName(String databaseName) Sets IDOL database name.
void	setHost(String host) Gets IDOL host name.
void	setIndexPort(int indexPort) Sets IDOL index port.
void	sync() Perform a DRESYNC / Commit on the idol Database.

Methods inherited from class com.norconex.committer.core.AbstractMappedCommitter

equals, getSourceContentField, getSourceReferenceField, getTargetContentField, getTargetReferenceField, hashCode, isKeepSourceContentField, isKeepSourceReferenceField, loadFromXML, prepareCommitAddition, saveToXML, setKeepSourceContentField, setKeepSourceReferenceField, setSourceContentField, setSourceReferenceField, setTargetContentField, setTargetReferenceField, toString

Methods inherited from class com.norconex.committer.core.AbstractBatchCommitter

commitAddition, commitComplete, commitDeletion, getCommitBatchSize, getMaxRetries, getMaxRetryWait, setCommitBatchSize, setMaxRetries, setMaxRetryWait

Methods inherited from class com.norconex.committer.core.AbstractFileQueueCommitter

commit, getInitialQueueDocCount, getQueueDir, prepareCommitDeletion, queueAddition, queueRemoval, setQueueDir

Methods inherited from class com.norconex.committer.core.AbstractCommitter

add, getQueueSize, remove, setQueueSize

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_IDOL_REFERENCE_FIELD

public static final String DEFAULT_IDOL_REFERENCE_FIELD

Default key field in Autonomy Idol database.

See Also:

Constant Field Values

DEFAULT_IDOL_CONTENT_FIELD

public static final String DEFAULT_IDOL_CONTENT_FIELD

Default field for content in Autonomy Idol Database.

See Also:

Constant Field Values

Constructor Detail

IdolCommitter

public IdolCommitter()

Constructor.

Method Detail

getCfsPort

public int getCfsPort()

Gets CFS port.

Returns:

CFS port

setCfsPort

public void setCfsPort(int cfsPort)

Sets CFS port.

Parameters:

cfsPort - CFS port

getIndexPort

public int getIndexPort()

Gets IDOL index port.

Returns:

IDOL index port

setIndexPort

public void setIndexPort(int indexPort)

Sets IDOL index port.

Parameters:

indexPort - IDOL index port

getDatabaseName

public String getDatabaseName()

Gets IDOL database name.

Returns:

IDOL database name

setDatabaseName

public void setDatabaseName(String databaseName)

Sets IDOL database name.

Parameters:

databaseName - IDOL database name

getHost

public String getHost()

Sets IDOL host name.

Returns:

IDOL host name

setHost

public void setHost(String host)

Gets IDOL host name.

Parameters:

host - IDOL host name

getDreAddDataParam

public String getDreAddDataParam(String name)

Gets the DREADDDATA URL parameter value for the parameter name.

Parameters:

name - parameter name

Returns:

parameter value

addDreAddDataParam

public void addDreAddDataParam(String name,
                               String value)

Adds a DREADDDATA URL parameter.

Parameters:

name - parameter name

value - parameter value

getDreDeleteRefParam

public String getDreDeleteRefParam(String name)

Gets the DREDELETEREF URL parameter value for the parameter name.

Parameters:

name - parameter name

Returns:

parameter value

addDreDeleteRefParam

public void addDreDeleteRefParam(String name,
                                 String value)

Adds the DREDELETEREF URL parameter value for the parameter name.

Parameters:

name - parameter name

value - parameter value

getDreAddDataParamNames

public Set<String> getDreAddDataParamNames()

Gets all DREADDDATA URL parameter names

Returns:

parameter names

getDreDeleteRefParamNames

public Set<String> getDreDeleteRefParamNames()

Gets all DREDELETEREF URL parameter names

Returns:

parameter names

commitBatch

protected void commitBatch(List<ICommitOperation> batch)

Specified by:

commitBatch in class AbstractBatchCommitter

loadFromXml

protected void loadFromXml(XMLConfiguration xml)

Specified by:

loadFromXml in class AbstractMappedCommitter

saveToXML

protected void saveToXML(XMLStreamWriter writer)
                  throws XMLStreamException

Specified by:

saveToXML in class AbstractMappedCommitter

Throws:

XMLStreamException

buildIdxDocument

protected String buildIdxDocument(InputStream is,
                                  Properties properties)

Build an idol document using the idx file format

Parameters:

is - input stream

properties - properties

Returns:

a string containing a document in the idx format

dreAddData

public void dreAddData(List<IAddOperation> addOperations)

Commits the addition operations.

Parameters:

addOperations - additions

dreDeleteRef

public void dreDeleteRef(List<IDeleteOperation> deleteOperations)

Commits the deletion operations

Parameters:

deleteOperations - deletions

sync

public void sync()

Perform a DRESYNC / Commit on the idol Database.

buildCfsXmlDocument

protected void buildCfsXmlDocument(XMLStreamWriter writer,
                                   InputStream is,
                                   Properties properties)
                            throws XMLStreamException,
                                   IOException

Throws:

XMLStreamException

IOException