Modifier and Type | Class and Description |
---|---|
class |
CrawlLogIterator
An implementation of a
is.hi.bok.deduplicator.CrawlDataIterator
capable of iterating over a Heritrix's style crawl.log . |
Modifier and Type | Method and Description |
---|---|
long |
DigestIndexer.writeToIndex(CrawlDataIterator dataIt,
String mimefilter,
boolean blacklist,
String defaultOrigin,
boolean verbose)
Writes the contents of a
CrawlDataIterator to this index. |
long |
DigestIndexer.writeToIndex(CrawlDataIterator dataIt,
String mimefilter,
boolean blacklist,
String defaultOrigin,
boolean verbose,
boolean skipDuplicates,
long minSize)
Writes the contents of a
CrawlDataIterator to this index. |
Copyright © 2014 National and University Library of Iceland. All Rights Reserved.