public class JdbcDataStoreEngine extends Object implements IDataStoreEngine, IXMLConfigurable
Data store engine using a JDBC-compatible database for storing crawl data.
To use this data store engine, you need its JDBC database driver on the classpath.
This JDBC data store engine uses
Hikari as the JDBC
datasource implementation, which provides efficient connection-pooling.
Refer to
Hikari's documentation for all configuration options. The Hikari options
are passed as-is, via datasource
properties as shown below.
This class only use a few data types to store its data in a generic way. It will try to detect what data type to use for your database. If you get errors related to field data types not being supported, you have the option to redefined them.
<dataStoreEngine
class="com.norconex.collector.core.store.impl.jdbc.JdbcDataStoreEngine">
<!-- Hikari datasource configuration properties: -->
<datasource>
<property
name="(property name)">
(property value)
</property>
</datasource>
<tablePrefix>
(Optional prefix used for table creation. Default is the collector
id plus the crawler id, each followed by an underscore character.)
</tablePrefix>
<!--
Optionally overwrite default SQL data type used. You should only
use if you get data type-related errors.
-->
<dataTypes>
<varchar
use="(equivalent data type for your database)"/>
<timestamp
use="(equivalent data type for your database)"/>
<text
use="(equivalent data type for your database)"/>
</dataTypes>
</dataStoreEngine>
<dataStoreEngine
class="JdbcDataStoreEngine">
<datasource>
<property
name="jdbcUrl">
jdbc:mysql://localhost:33060/sample
</property>
<property
name="username">
dbuser
</property>
<property
name="password">
dbpwd
</property>
<property
name="connectionTimeout">
1000
</property>
</datasource>
</dataStoreEngine>
The above example contains basic settings for creating a MySQL data source.
Constructor and Description |
---|
JdbcDataStoreEngine() |
Modifier and Type | Method and Description |
---|---|
boolean |
clean() |
void |
close() |
boolean |
dropStore(String storeName) |
Properties |
getConfigProperties() |
Set<String> |
getStoreNames() |
Optional<Class<?>> |
getStoreType(String storeName) |
String |
getTablePrefix() |
String |
getTextType() |
String |
getTimestapType() |
String |
getVarcharType() |
void |
init(Crawler crawler) |
void |
loadFromXML(XML xml) |
<T> IDataStore<T> |
openStore(String storeName,
Class<? extends T> type) |
boolean |
renameStore(IDataStore<?> dataStore,
String newStoreName) |
void |
saveToXML(XML xml) |
void |
setConfigProperties(Properties configProperties) |
void |
setTablePrefix(String tablePrefix) |
void |
setTextType(String textType) |
void |
setTimestapType(String timestapType) |
void |
setVarcharType(String varcharType) |
public Properties getConfigProperties()
public void setConfigProperties(Properties configProperties)
public String getTablePrefix()
public void setTablePrefix(String tablePrefix)
public String getVarcharType()
public void setVarcharType(String varcharType)
public String getTimestapType()
public void setTimestapType(String timestapType)
public String getTextType()
public void setTextType(String textType)
public void init(Crawler crawler)
init
in interface IDataStoreEngine
public boolean clean()
clean
in interface IDataStoreEngine
public void close()
close
in interface IDataStoreEngine
close
in interface Closeable
close
in interface AutoCloseable
public <T> IDataStore<T> openStore(String storeName, Class<? extends T> type)
openStore
in interface IDataStoreEngine
public boolean dropStore(String storeName)
dropStore
in interface IDataStoreEngine
public boolean renameStore(IDataStore<?> dataStore, String newStoreName)
renameStore
in interface IDataStoreEngine
public Set<String> getStoreNames()
getStoreNames
in interface IDataStoreEngine
public Optional<Class<?>> getStoreType(String storeName)
getStoreType
in interface IDataStoreEngine
public void loadFromXML(XML xml)
loadFromXML
in interface IXMLConfigurable
public void saveToXML(XML xml)
saveToXML
in interface IXMLConfigurable
Copyright © 2014–2023 Norconex Inc.. All rights reserved.