public class BaseMongoSerializer extends Object implements IMongoSerializer
BaseCrawlData
instances.IMongoSerializer.Stage
FIELD_CONTENT_CHECKSUM, FIELD_CONTENT_TYPE, FIELD_CRAWL_DATE, FIELD_CRAWL_STATE, FIELD_DEPTH, FIELD_IS_ROOT_PARENT_REFERENCE, FIELD_IS_VALID, FIELD_META_CHECKSUM, FIELD_PARENT_ROOT_REFERENCE, FIELD_REFERENCE, FIELD_REFERENCE_EXCESSIVE, FIELD_STAGE
Constructor and Description |
---|
BaseMongoSerializer() |
Modifier and Type | Method and Description |
---|---|
void |
createIndices(com.mongodb.client.MongoCollection<org.bson.Document> referenceCollection,
com.mongodb.client.MongoCollection<org.bson.Document> cachedCollection)
Creates Mongo indices for the given collections.
|
protected void |
ensureIndex(com.mongodb.client.MongoCollection<org.bson.Document> coll,
boolean unique,
String... fields) |
ICrawlData |
fromDocument(org.bson.Document doc)
Converts a Mongo
Document to an ICrawlData . |
org.bson.Document |
getNextQueued(com.mongodb.client.MongoCollection<org.bson.Document> collRefs)
Gets the next queued DB document from the given collection.
|
org.bson.Document |
toDocument(IMongoSerializer.Stage stage,
ICrawlData crawlData)
Converts a
ICrawlData to a Mongo Document . |
public org.bson.Document toDocument(IMongoSerializer.Stage stage, ICrawlData crawlData)
IMongoSerializer
ICrawlData
to a Mongo Document
.toDocument
in interface IMongoSerializer
stage
- the Mongo serializer stagecrawlData
- the data to serializepublic ICrawlData fromDocument(org.bson.Document doc)
IMongoSerializer
Document
to an ICrawlData
.fromDocument
in interface IMongoSerializer
doc
- Mongo documentpublic org.bson.Document getNextQueued(com.mongodb.client.MongoCollection<org.bson.Document> collRefs)
IMongoSerializer
getNextQueued
in interface IMongoSerializer
collRefs
- the collection to get the next document frompublic void createIndices(com.mongodb.client.MongoCollection<org.bson.Document> referenceCollection, com.mongodb.client.MongoCollection<org.bson.Document> cachedCollection)
IMongoSerializer
createIndices
in interface IMongoSerializer
referenceCollection
- the collection holding crawl referencescachedCollection
- the collection holding cached crawl referencesprotected final void ensureIndex(com.mongodb.client.MongoCollection<org.bson.Document> coll, boolean unique, String... fields)
Copyright © 2014–2021 Norconex Inc.. All rights reserved.