com.norconex.collector.core.checksum

Interface IDocumentChecksummer

All Known Implementing Classes:

AbstractDocumentChecksummer, MD5DocumentChecksummer

public interface IDocumentChecksummer

Creates a checksum representing a a document. Checksums are used to quickly filter out documents that have already been processed or that have changed since a previous run.

Two or more ImporterDocument can hold different values, but be deemed logically the same. Such documents do not have to be equal, but they should return the same checksum. An example of this can be two different URLs pointing to the same document, where only a single instance should be kept.

There are no strict rules that define what is equivalent or not.

Classes implementing IXMLConfigurable should offer the following XML configuration usage:

  <documentChecksummer 
      class="(class)">
      keep="[false|true]"
      targetField="(optional metadata field to store the checksum)" />
 

targetField is ignored unless the keep attribute is set to true.

Author:

Pascal Essiembre

See Also:

AbstractDocumentChecksummer

Method Summary

Modifier and Type	Method and Description
String	createDocumentChecksum(ImporterDocument document) Creates a document checksum.

Method Detail

createDocumentChecksum

String createDocumentChecksum(ImporterDocument document)

Creates a document checksum.

Parameters:

document - an HTTP document

Returns:

a checksum value