com.norconex.importer.handler.filter.impl

Class ScriptFilter

java.lang.Object

com.norconex.importer.handler.AbstractImporterHandler

com.norconex.importer.handler.filter.AbstractDocumentFilter

com.norconex.importer.handler.filter.AbstractCharStreamFilter

com.norconex.importer.handler.filter.AbstractStringFilter

com.norconex.importer.handler.filter.impl.ScriptFilter

All Implemented Interfaces:

IXMLConfigurable, IDocumentFilter, IOnMatchFilter, IImporterHandler

public class ScriptFilter
extends AbstractStringFilter

Filter incoming documents using a scripting language. The default script engine is JavaScript.

Refer to ScriptRunner for more information on using a scripting language with Norconex Importer.

How to filter documents with scripting:

The following are variables made available to your script for each document:

• reference: Document unique reference as a string.
• content: Document content, as a string (of maxReadSize length).
• metadata: Document metadata as a ImporterMetadata object.
• parsed: Whether the document was already parsed, as a boolean.
• sectionIndex: Content section index if it had to be split, as an integer.

The expected return value from your script is a boolean indicating whether the document was matched or not.

XML configuration usage:

  <filter class="com.norconex.importer.handler.filter.impl.ScriptFilter"
          engineName="(script engine name)"
          onMatch="[include|exclude]" 
          sourceCharset="(character encoding)"
          maxReadSize="(max content characters to read at once)" >
          
      <restrictTo caseSensitive="[false|true]"
              field="(name of header/metadata field name to match)">
          (regular expression of value to match)
      </restrictTo>
      <!-- multiple "restrictTo" tags allowed (only one needs to match) -->          
          
      <script>(your script)</script>
  </filter>
 

Usage example:

JavaScript:

  <filter class="com.norconex.importer.handler.filter.impl.ScriptFilter">
    <script><![CDATA[
      var isAppleDoc = metadata.getString('fruit') == 'apple'
              || content.indexOf('Apple') > -1;
      /*return*/ isAppleDoc;
    ]]></script>
  </filter>
 

Lua:

  <filter class="com.norconex.importer.handler.filter.impl.ScriptFilter"
      engineName="lua">
    <script><![CDATA[
      local isAppleDoc = metadata:getString('fruit') == 'apple'
              and content:find('Apple') ~= nil;
      return isAppleDoc;
    ]]></script>
  </filter>
 

Since:

2.4.0

Author:

Pascal Essiembre

See Also:

ScriptRunner

Constructor Summary

Constructors

Constructor and Description
ScriptFilter()

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object other)
String	getEngineName()
String	getScript()
int	hashCode()
protected boolean	isStringContentMatching(String reference, StringBuilder content, ImporterMetadata metadata, boolean parsed, int sectionIndex)
protected void	loadStringFilterFromXML(org.apache.commons.configuration.XMLConfiguration xml) Loads configuration settings specific to the implementing class.
protected void	saveStringFilterToXML(EnhancedXMLStreamWriter writer) Saves configuration settings specific to the implementing class.
void	setEngineName(String engineName)
void	setScript(String script)
String	toString()

Methods inherited from class com.norconex.importer.handler.filter.AbstractStringFilter

getMaxReadSize, isTextDocumentMatching, loadCharStreamFilterFromXML, saveCharStreamFilterToXML, setMaxReadSize

Methods inherited from class com.norconex.importer.handler.filter.AbstractCharStreamFilter

getSourceCharset, isDocumentMatched, loadFilterFromXML, saveFilterToXML, setSourceCharset

Methods inherited from class com.norconex.importer.handler.filter.AbstractDocumentFilter

acceptDocument, getOnMatch, loadHandlerFromXML, saveHandlerToXML, setOnMatch

Methods inherited from class com.norconex.importer.handler.AbstractImporterHandler

addRestriction, addRestriction, addRestrictions, clearRestrictions, detectCharsetIfBlank, getRestrictions, isApplicable, loadFromXML, removeRestriction, removeRestriction, saveToXML

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

ScriptFilter

public ScriptFilter()

Method Detail

getEngineName

public String getEngineName()

setEngineName

public void setEngineName(String engineName)

getScript

public String getScript()

setScript

public void setScript(String script)

isStringContentMatching

protected boolean isStringContentMatching(String reference,
                                          StringBuilder content,
                                          ImporterMetadata metadata,
                                          boolean parsed,
                                          int sectionIndex)
                                   throws ImporterHandlerException

Specified by:

isStringContentMatching in class AbstractStringFilter

Throws:

ImporterHandlerException

saveStringFilterToXML

protected void saveStringFilterToXML(EnhancedXMLStreamWriter writer)
                              throws XMLStreamException

Description copied from class: AbstractStringFilter

Saves configuration settings specific to the implementing class. The parent tag along with the "class" attribute are already written. Implementors must not close the writer.

Specified by:

saveStringFilterToXML in class AbstractStringFilter

Parameters:

writer - the xml writer

Throws:

XMLStreamException - could not save to XML

loadStringFilterFromXML

protected void loadStringFilterFromXML(org.apache.commons.configuration.XMLConfiguration xml)
                                throws IOException

Description copied from class: AbstractStringFilter

Loads configuration settings specific to the implementing class.

Specified by:

loadStringFilterFromXML in class AbstractStringFilter

Parameters:

xml - xml configuration

Throws:

IOException - could not load from XML

equals

public boolean equals(Object other)

Overrides:

equals in class AbstractStringFilter

hashCode

public int hashCode()

Overrides:

hashCode in class AbstractStringFilter

toString

public String toString()

Overrides:

toString in class AbstractStringFilter