Sitemap

An index of all important pages on doc.unobtanium.rocks:

Data Structures

• Base Database
• Crawl exit code
• Crawl Log
• Crawl Summary
• Crawler Database
• Database Data Types
• Database Info
• Entity Generation
• Information Source
• Link destination type
• Link Locality
• Location Signature
• Summary Database
• Text Pile
• URL

Concepts

• Crawl Data Tree
• Duplicate Types
• Entity Data Tree
• Patience and Fluke Events
• Token Index

Algorithms

• Calculating Indexiness
• Crawl Delay between Requests
• Crawl Loop
• Databse Batching (Pattern)
• Summarizing

Manual

• About the unobtanium.rocks Crawler
• Configuring SearxNG to use Unobtanium
• Crawler crawl configuration
• Crawler User-Agent and robots.txt
• Selfhosting Unobtanium
• Terminology
• Unobtanium Overview
• Unobtanium Query syntax
• unobtanium-crawler
• unobtanium-viewer
• Useful database queries

Criteria

• Boolean
• Criterium Chain
• Number
• Origin
• String
• Url