Expand awareness.
Keep the evidence.

Two registries, one research loop

A collection target is a source, feed or document we want to understand. A collection tool is the mechanism that might retrieve it. Rank them separately: a capable crawler does not make a source useful, and a useful source does not make a private API available.

The source registry records stable IDs, canonical metadata URLs, publisher families, format, topic requirements, provenance, editorial estimates and access evidence. The tool registry records implementation status, formats, cost assumptions and reliability basis. History links observations to scoped claims, supporting or contradicting evidence, dates and content digests.

Change the topic, requirement weights or evidence in a local snapshot, then rerun the ranker. It proposes a bounded plan; it does not execute collection. The supplied example contains 11 targets, 3 tools, 4 requirements and 4 scoped synthetic claims.

Permission is a gate, not a ranking bonus

Simulation eligibility requires verified fixture permission, a check no older than 14 days, unexpired evidence, exact canonical URL scope, separate terms and robots decisions, synthetic labels and a compatible implemented offline reader. Revoked, hypothetical, stale, missing or mismatched evidence blocks selection. Real public references remain metadata-only and blocked. There is no collect command, and every export reports collection_enabled: false.

A future connector needs independently verified owner/account authorization, purpose, fields, retention/export rights, API/path scope, terms, applicable robots rules, quotas and immediate pre-dispatch revocation checks. Structured assertions are not proof of permission. Changing a flag in this prototype cannot activate a private feed.

RFC 9309 defines robots processing and explicitly distinguishes robots rules from access authorization. An allowed robots path, public URL, paid account or hypothetical future agreement is not a general collection or republication license. The two real metadata references are Ruby's engineering guide and that public standard; no restricted directory data is included.

Explain every priority

The target score uses relevance (20%), expected coverage gain (30%), reliability (15%), novelty (10%), freshness need (10%), contradiction opportunity (10%) and affordability (5%). It is multiplied by 1 / sqrt(1 + prior observations). These are editable editorial heuristics, not probabilities or validated utility predictions. Blocked sources can remain visible for permission-review planning but cannot enter the simulated collection plan.

Expected gain is the weighted unmet declared requirement times estimated contribution. Exact known content receives zero novelty; prior unique content lowers it. Age relative to a refresh interval sets freshness need. A target that might challenge current support earns a contradiction opportunity. Relative cost units are synthetic, not prices or measured latency.

Every third plan slot explores the least-observed eligible target, then favors a less-represented publisher family. Other slots discount repeated families in the plan. At most six targets fit the budget. This prevents repeated popular sources from monopolizing the sample; it is neither an optimal budget solver nor a calibrated bandit. Actual new evidence triggers reranking; hypothetical gain is not recomputed after every selection.

Coverage remains bounded by the questions

• Access and permission boundaries: 0/2 supporting provenance clusters; 0 current evidence units; no current contradiction recorded. Freshness window: 30 days.
• Source diversity and coverage gaps: 1/2 supporting provenance clusters; 1 current evidence units; no current contradiction recorded. Freshness window: 30 days.
• Evidence quality and contradictions: 1/2 supporting provenance clusters; 1 current evidence units; no current contradiction recorded. Freshness window: 30 days.
• Change detection and freshness: 0/2 supporting provenance clusters; 0 current evidence units; no current contradiction recorded. Freshness window: 30 days.

The source universe is unknown and completeness percentage is null. Evidence progress applies only to the declared checklist. Exact-content duplicates count once; families sharing identical observed material are conservatively linked into provenance clusters, even when a mirror arrives later. This can undercount independence. Semantic deduplication and automatic publisher identity resolution remain future work.

The reproducible dry-run plan

Fixture budget: 10 relative units. Nothing in this list is submitted to a provider or fetched.

1. gap + local-fixture · 2 relative units. exploitation: score discounted for repeated family in plan.
2. counter + local-fixture · 3 relative units. exploitation: score discounted for repeated family in plan.
3. new-family + local-fixture · 1 relative units. exploration: fewest prior events, then family diversity.
4. injection + local-fixture · 1 relative units. exploitation: score discounted for repeated family in plan.
5. same-content + local-fixture · 1 relative units. exploitation: score discounted for repeated family in plan.
6. popular + local-fixture · 2 relative units. exploration: fewest prior events, then family diversity.

Collection targets: priority and blockers

The injection-like title below is intentionally displayed as escaped data. It cannot alter permissions, scores or commands. Inputs are structured JSON, not executable instructions.

Collection tools: a separate ranking

Tool scores use format fit (55%), reliability basis (25%) and affordability (20%). Proposed tools can rank highly for future investigation but remain ineligible. Performance is unbenchmarked outside fixtures.

From collection history to a useful dossier

The JSON includes claim IDs, propositions, scope, qualitative status, confidence rationale, unknowns and supporting/contradicting event references. Apply the supplied counter-evidence event and the reliability claim becomes disputed; it is not averaged away.

• claim-access: unknown. The synthetic overview adequately addresses access and permission boundaries. Synthetic demonstration only; not a factual claim about a real publisher. Support: ; contradiction: .
• claim-coverage: one_sided_fixture_evidence. The synthetic overview adequately addresses source diversity and coverage gaps. Synthetic demonstration only; not a factual claim about a real publisher. Support: event-1; contradiction: .
• claim-reliability: disputed. The synthetic overview adequately addresses evidence quality and contradictions. Synthetic demonstration only; not a factual claim about a real publisher. Support: event-2; contradiction: field-report-1.
• claim-freshness: unknown. The synthetic overview adequately addresses change detection and freshness. Synthetic demonstration only; not a factual claim about a real publisher. Support: ; contradiction: .

A small hand-authored ACH (analysis of competing hypotheses) example asks whether repeated overview collection is sufficient or independent specialist sources are needed. Its matrix stores qualitative assessments and rationale, not numerical likelihoods. New evidence changes the claim summary; it does not silently rewrite the human's hypothesis assessment. Use ACH for a consequential unresolved question, not an empty matrix for every fact.

Bias prompts ask what would disconfirm the preferred explanation, whether sources share an origin, whether popularity displaced relevance, and what scope remains unknown. Here “CBT” means the requested cognitive-bias and assumption checks, not clinical diagnosis. Next research avenues link targets to unmet requirements and exported gain components.

Longer briefing dossiers, rights-reviewed video/transcript analysis and multimodal evidence comparison are proposed extensions. This version neither downloads media nor performs semantic claim extraction. Hourly research handoffs can supply reviewed candidate snapshots; this package installs no schedule and grants no deployment authority.

Run it locally

ruby -I. -e 'Dir["test_*.rb"].sort.each { |f| require_relative f }'
ruby cli.rb rank registry.json 2026-10-16 > ranking.json
ruby cli.rb apply-history registry.json new-history.json updated.json 2026-10-16
ruby cli.rb rank updated.json 2026-10-16 > updated-ranking.json

Run inside the extracted prototype directory with Ruby 3.3 or a compatible Ruby. No gems are required. History import accepts only reviewed synthetic fixtures, creates a new snapshot, refuses overwrite, ignores identical event replay and rejects conflicting event IDs. It records local evidence; it does not assert real-world permission or perform collection.

What has been tested, and what must come next

Regression tests cover revoked/stale/expired permission, terms/robots denial, scope mismatch, blocked real metadata and proposed tools, duplicate targets, later mirrors, weight scaling, deterministic exploration/budgets, injection-like text, contradictory evidence and replay-safe local snapshots. They prove bounded fixture behavior, not research quality on unseen subjects.

1. Establish held-out topics. An independent reviewer defines useful new claims, missing perspectives and expected permission decisions before tuning.
2. Compare simple baselines. Measure this ranking against relevance-only and recency-only selection using useful verified claims per unit cost, independent-source diversity, contradiction discovery and stale-evidence errors.
3. Measure forbidden outcomes. Target zero unauthorized attempts, complete provenance, no duplicate gain from mirrors and no injection-driven action. Record failures rather than weakening tests.
4. Benchmark any future connector. Measure latency, cost, quality, rate limits, error/retry behavior, revocation, robots changes and retention using authorized test sources.
5. Release reviewed handoffs. Keep input/version hashes, scoring changes, evidence and gaps. Publish only after independent review and scoped deployment checks.

Keep the registry small until those measurements justify more machinery. No completeness guarantee, private-feed integration or production crawler is claimed. Follow the collaboration and release protocol and consent engineering when later work crosses into real data.