Sitemap

An index of all important pages on doc.unobtanium.rocks:

Data Structures

• Base Database
• Crawl exit code
• Crawl Log
• Crawl Summary
• Entity Generation
• Link Locality
• Location Signature
• Text Pile
• URL

Concepts

• Duplicate Types
• Entity Data Tree
• Patience and Fluke Events

Algorithms

• Calculating Indexiness
• Crawl Loop
• Databse Batching (Pattern)
• Summarizing

Manual

• About the unobtanium.rocks Crawler
• Configuring SearxNG to use Unobtanium
• Crawler crawl configuration
• Crawler User-Agent and robots.txt
• Terminology
• Unobtanium Overview
• Unobtanium Query syntax
• Useful database queries

Criteria

• Boolean
• Criterium Chain
• Number
• Origin
• String
• Url