com.norconex.importer.handler.splitter.impl

Class TranslatorSplitter

java.lang.Object

com.norconex.importer.handler.AbstractImporterHandler

com.norconex.importer.handler.splitter.AbstractDocumentSplitter

com.norconex.importer.handler.splitter.impl.TranslatorSplitter

All Implemented Interfaces:

IXMLConfigurable, IImporterHandler, IDocumentSplitter

public class TranslatorSplitter
extends AbstractDocumentSplitter

Translate documents using one of the supported translation API. The following lists the supported APIs, along with the required authentication properties or settings for each:

• microsoft
  • clientId
  • clientSecret
• google
  • apiKey
• lingo24
  • userKey
• moses
  • smtPath
  • scriptPath
• yandex
  • apiKey

For example, the Microsoft Translation API requires a client ID and a client secret, both obtained on Microsoft Azure Marketplace with your Microsoft account.

Translated documents will have the original document language stored in a field "document.translatedFrom".

This class is not a document "splitter" per se, but like regular splitters, the translation will create children documents for each translation performed. The parent document will always remain the original document, while the children will always be the translations.

XML configuration usage:

  <splitter class="com.norconex.importer.handler.splitter.impl.TranslatorSplitter"
          api="(microsoft|google|lingo24|moses|yandex)" >
      <ignoreContent>(false|true)</ignoreContent>
      <ignoreNonTranslatedFields>(false|true)</ignoreNonTranslatedFields>
      <fieldsToTranslate>(coma-separated list of fields)</fieldsToTranslate>
      <sourceLanguageField>(field containing language)</sourceLanguageField>
      <sourceLanguage>(language when no source language field)</sourceLanguage>
      <targetLanguages>(coma-separated list of languages)</targetLanguages>
      
      <!-- Microsoft -->
      <clientId>...</clientId>
      <clientSecret>...</clientSecret>
      
      <!-- Google -->
      <apiKey>...</apiKey>
      
      <!-- Lingo24 -->
      <userKey>...</userKey>
      
      <!-- Moses -->
      <smtPath>...</smtPath>
      <scriptPath>...</scriptPath>
      
      <!-- Yandex -->
      <apiKey>...</apiKey>
          
      <restrictTo caseSensitive="[false|true]"
              field="(name of header/metadata field name to match)">
          (regular expression of value to match)
      </restrictTo>
      <!-- multiple "restrictTo" tags allowed (only one needs to match) -->
  </splitter>
 

Usage example:

The following uses the Google translation API to translate documents into French, taking the source document language from a field called "langField".

  <splitter class="com.norconex.importer.handler.splitter.impl.TranslatorSplitter"
          api="google" >
      <sourceLanguageField>langField</sourceLanguageField>
      <targetLanguages>fr</targetLanguages>
      <apiKey>...MYKEYHERE...</apiKey>
  </splitter>
 

Since:

2.1.0

Author:

Pascal Essiembre

Field Summary

Fields

Modifier and Type	Field and Description
static String	API_GOOGLE
static String	API_LINGO24
static String	API_MICROSOFT
static String	API_MOSES
static String	API_YANDEX

Constructor Summary

Constructors

Constructor and Description
TranslatorSplitter() Constructor.

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object other)
String	getApi()
String	getApiKey()
String	getClientId()
String	getClientSecret()
String[]	getFieldsToTranslate()
String	getScriptPath()
String	getSmtPath()
String	getSourceLanguage()
String	getSourceLanguageField()
String[]	getTargetLanguages()
String	getUserKey()
int	hashCode()
boolean	isIgnoreContent()
boolean	isIgnoreNonTranslatedFields()
protected void	loadHandlerFromXML(org.apache.commons.configuration.XMLConfiguration xml) Loads configuration settings specific to the implementing class.
static void	main(String[] args)
protected void	saveHandlerToXML(EnhancedXMLStreamWriter writer) Saves configuration settings specific to the implementing class.
void	setApi(String api)
void	setApiKey(String apiKey)
void	setClientId(String clientId)
void	setClientSecret(String clientSecret)
void	setFieldsToTranslate(String... fieldsToTranslate)
void	setIgnoreContent(boolean ignoreContent)
void	setIgnoreNonTranslatedFields(boolean ignoreNonTranslatedFields)
void	setScriptPath(String scriptPath)
void	setSmtPath(String smtPath)
void	setSourceLanguage(String sourceLanguage)
void	setSourceLanguageField(String sourceLanguageField)
void	setTargetLanguages(String... targetLanguages)
void	setUserKey(String userKey)
protected List<ImporterDocument>	splitApplicableDocument(SplittableDocument doc, OutputStream output, CachedStreamFactory streamFactory, boolean parsed)
String	toString()

Methods inherited from class com.norconex.importer.handler.splitter.AbstractDocumentSplitter

splitDocument

Methods inherited from class com.norconex.importer.handler.AbstractImporterHandler

addRestriction, addRestriction, addRestrictions, clearRestrictions, detectCharsetIfBlank, getRestrictions, isApplicable, loadFromXML, removeRestriction, removeRestriction, saveToXML

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

API_MICROSOFT

public static final String API_MICROSOFT

See Also:

Constant Field Values

API_GOOGLE

public static final String API_GOOGLE

See Also:

Constant Field Values

API_LINGO24

public static final String API_LINGO24

See Also:

Constant Field Values

API_MOSES

public static final String API_MOSES

See Also:

Constant Field Values

API_YANDEX

public static final String API_YANDEX

See Also:

Constant Field Values

Constructor Detail

TranslatorSplitter

public TranslatorSplitter()

Constructor.

Method Detail

splitApplicableDocument

protected List<ImporterDocument> splitApplicableDocument(SplittableDocument doc,
                                                         OutputStream output,
                                                         CachedStreamFactory streamFactory,
                                                         boolean parsed)
                                                  throws ImporterHandlerException

Specified by:

splitApplicableDocument in class AbstractDocumentSplitter

Throws:

ImporterHandlerException

isIgnoreContent

public boolean isIgnoreContent()

setIgnoreContent

public void setIgnoreContent(boolean ignoreContent)

getFieldsToTranslate

public String[] getFieldsToTranslate()

setFieldsToTranslate

public void setFieldsToTranslate(String... fieldsToTranslate)

isIgnoreNonTranslatedFields

public boolean isIgnoreNonTranslatedFields()

setIgnoreNonTranslatedFields

public void setIgnoreNonTranslatedFields(boolean ignoreNonTranslatedFields)

getSourceLanguageField

public String getSourceLanguageField()

setSourceLanguageField

public void setSourceLanguageField(String sourceLanguageField)

getSourceLanguage

public String getSourceLanguage()

setSourceLanguage

public void setSourceLanguage(String sourceLanguage)

getTargetLanguages

public String[] getTargetLanguages()

setTargetLanguages

public void setTargetLanguages(String... targetLanguages)

getApiKey

public String getApiKey()

setApiKey

public void setApiKey(String apiKey)

getUserKey

public String getUserKey()

setUserKey

public void setUserKey(String userKey)

getSmtPath

public String getSmtPath()

setSmtPath

public void setSmtPath(String smtPath)

getScriptPath

public String getScriptPath()

setScriptPath

public void setScriptPath(String scriptPath)

getClientId

public String getClientId()

setClientId

public void setClientId(String clientId)

getClientSecret

public String getClientSecret()

setClientSecret

public void setClientSecret(String clientSecret)

getApi

public String getApi()

setApi

public void setApi(String api)

main

public static void main(String[] args)
                 throws ImporterHandlerException

Throws:

ImporterHandlerException

loadHandlerFromXML

protected void loadHandlerFromXML(org.apache.commons.configuration.XMLConfiguration xml)
                           throws IOException

Description copied from class: AbstractImporterHandler

Loads configuration settings specific to the implementing class.

Specified by:

loadHandlerFromXML in class AbstractImporterHandler

Parameters:

xml - xml configuration

Throws:

IOException - could not load from XML

saveHandlerToXML

protected void saveHandlerToXML(EnhancedXMLStreamWriter writer)
                         throws XMLStreamException

Description copied from class: AbstractImporterHandler

Saves configuration settings specific to the implementing class. The parent tag along with the "class" attribute are already written. Implementors must not close the writer.

Specified by:

saveHandlerToXML in class AbstractImporterHandler

Parameters:

writer - the xml writer

Throws:

XMLStreamException - could not save to XML

equals

public boolean equals(Object other)

Overrides:

equals in class AbstractImporterHandler

hashCode

public int hashCode()

Overrides:

hashCode in class AbstractImporterHandler

toString

public String toString()

Overrides:

toString in class AbstractImporterHandler