Models propose · software executes · evidence decides
The software around a model that turns a request into a controlled sequence of work.
A model can propose a patch. A harness lets the system inspect files, apply permitted edits, run tests, examine failures and try again. Reliability depends on both the model and the machinery around it.
This is a broad checklist of responsibilities, not a universal bill of materials. Some implementations combine components; others distribute them. Nothing here asserts that a proposed JayAI system is already installed.
Model: consumes context and produces responses, including proposed tool calls. A tool call is data until an executor acts on it. Harness: prepares requests, connects capabilities, runs the loop and applies controls. Agent: the working system of model, harness, instructions and capabilities pursuing a task.
A framework supplies reusable construction blocks; it need not define the whole application. A runtime executes work and manages its lifecycle; this can mean orchestration, the tool environment or both. Product terminology overlaps, so inspect actual responsibilities.
Claude Code describes its agentic harness in terms of tools and context management, and documents a gather-context, act, verify loop. That provides a concrete starting point; production operation adds responsibilities beyond that loop. Claude Code architecture.
New observations return to context assembly. A denial is also an observation; it does not authorize a workaround. Stop when acceptance criteria are met, a boundary needs human input, the user cancels, or a limit is reached. A timeout is not proof that a remote action did nothing.
Each entry gives its role, an input → output example, and a failure it helps prevent. “Core” means a minimal tool-using agent needs the responsibility, not necessarily a separate service. These examples are our engineering synthesis; source-specific designs are linked below.
Core
Turns a chat, CLI command, API request or authorized event into a scoped job, with a route for clarification, cancellation and corrections.
Input → output: “Fix discount rounding; leave deployment to me” → task T-42, repository reference, expected result and no-deployment boundary.
Failure prevented: An ambiguous goal or task ID being mistaken for authorization.
Core
Tracks instruction provenance and precedence, while enforceable policy determines which actions may execute. Project conventions and retrieved text have different authority.
Input → output: “Run pricing tests” plus an allowed working directory → instruction bundle and action policy. A README cannot grant permission to upload the repository.
Failure prevented: Prompt injection or helpful-sounding text silently expanding permissions.
Core
Selects instructions, messages, relevant files, tool definitions and observations within finite context capacity. Retains full evidence outside the prompt when necessary.
Input → output: A long conversation and 20,000-line log → active constraints, relevant code, failure excerpt and reference to the full log. Mark truncation explicitly.
Failure prevented: Lost constraints, irrelevant context and incomplete output presented as complete.
Conditional
Finds useful information and retains selected knowledge across steps or sessions. Files, SQL, keyword search and embeddings are options; a vector database is optional.
Input → output: “Where is discount rounding?” → matching methods with paths and revisions. A later task may retrieve a verified convention with owner, freshness and deletion rules.
Failure prevented: Stale knowledge or another task’s data contaminating the current job. Distinguish temporary task state from durable knowledge.
Core
Converts messages and tool definitions into provider requests and normalizes responses, usage, errors and stop reasons. Streaming and routing can live here.
Input → output: Messages plus a read_file schema → a structured read request, final answer, refusal or incomplete generation. These are distinct outcomes.
Failure prevented: Provider differences leaking everywhere, or a generation limit being reported as task success.
Core
Describes names, arguments, results and executors. Record whether a capability reads or changes state, its timeout behavior and relevant limits.
Input → output: read_file(path: "app/pricing.rb", start_line: 20) → numbered contents and a revision identifier. Local functions, APIs or MCP can provide the interface.
Failure prevented: Malformed calls and tools whose names hide their actual side effects. A protocol connection is not authorization.
Core
Coordinates model decisions, proposed actions, observations and transitions. State holds the objective, outstanding calls, evidence and pending questions. A detailed plan is optional for small jobs.
Input → output: A failing test → inspect the rounding method → read action → new observation → another model decision. Completion has explicit criteria.
Failure prevented: Endless loops, forgotten work and treating silence as success.
Core for executable tools
Supplies a process, container, VM, remote workspace or browser with the necessary files, dependencies, working directory and cleanup.
Input → output: Repository revision plus a Ruby environment specification → workspace where bundle exec ruby test/pricing_test.rb returns logs and exit status.
Failure prevented: Tests running against the wrong files or dependencies; leftover state affecting later tasks.
Required for real resources
Separates technical reachability, task authorization, approval boundaries and credential delivery. Enforce filesystem and network restrictions outside model instructions.
Input → output: Proposed deployment → permitted execution, a bounded approval request or denial with a reason. Inject scoped credentials only where needed; redact logs.
Failure prevented: Unauthorized effects and exposed secrets. A container alone is not a security guarantee, and approval must cover the actual action.
Required for real resources
Validates arguments before actions and outcomes afterward. Deterministic checks enforce schemas, paths, revisions and test results wherever possible.
Input → output: Patch → expected-revision check, syntax status, regression results and independently reviewed diff against acceptance criteria.
Failure prevented: A plausible answer masking a broken patch or an edit overwriting newer work.
Conditional; essential for consequential retries
Records action intent and observed outcome so interrupted work can resume. Checkpoints preserve state; idempotency keys help a destination recognize repeated operations.
Input → output: API timeout with uncertain result → operation-ID status lookup → resume, safe retry or escalation. Do not blindly repeat a deployment or message.
Failure prevented: Duplicate side effects. A checkpoint cannot undo external changes or by itself guarantee exactly-once execution.
Optional
Assigns bounded work, result contracts, cancellation and resource limits. One coordinator reconciles evidence and owns delivery.
Input → output: Implementation investigation and regression-case design → two reports checked by the coordinator. Use separate worktrees or explicit file ownership for edits.
Failure prevented: Conflicting writes, duplicated actions and unreviewed worker conclusions. Extra workers must justify their coordination cost.
Core
Bounds tokens, money, elapsed time, tool duration, concurrency and repeated failure. Define per-operation and total limits.
Input → output: Budget nearly exhausted → cancel unnecessary work and return an honest partial result, or request an authorized extension.
Failure prevented: Runaway cost and work that never terminates. Completed, failed, cancelled and budget-exhausted differ from paused or awaiting approval.
Production quality layer
Runs controlled tasks against a candidate system. Include regressions, unsafe requests, outages, malformed outputs and ambiguous completion. Repeat stochastic cases and record configuration.
Input → output: Fixed held-out bug-fix tasks + candidate → outcome pass rates, side-effect violations, cost, latency and specific regressions.
Failure prevented: Optimizing persuasive answers while actual outcomes worsen, or tuning repeatedly against the only test set.
Basic record from the start
Connects task IDs, model requests, tool IDs, durations, transitions, approvals, errors, usage and artifacts. Apply access controls, redaction and retention.
Input → output: A twelve-minute task → trace showing nine minutes spent installing dependencies, with action and result records.
Failure prevented: Unexplained failures and misleading aggregate metrics. Observable events are useful without collecting private internal reasoning.
Core
Packages a patch, report, attachment or verified URL the recipient can inspect. Distinguishes prepared work from authorized publication.
Input → output: Changed files and passing tests → diff, exact checks, known limits and accessible artifact location. For this example, report “fixed locally.”
Failure prevented: Claiming deployment without publishing and checking it, or returning an inaccessible artifact.
Production layer
Tracks compatible coordinator, model identifiers, instructions, schemas, policies, dependencies and storage versions. Defines startup, shutdown, recovery and rollback.
Input → output: Candidate release → evaluated version, dated changelog, deployment verification and rollback path. Change one major variable at a time when practical.
Failure prevented: Untraceable regressions. Recorded versions improve comparisons even when repeated model calls yield different outputs.
Start with a task, instructions, model adapter, small tool interface, loop and enough state to feed observations back. Add a stopping rule and deliver the result. When tools affect real resources, enforce permissions and validation and keep a basic event record from the beginning.
Long-term memory, MCP, vector search, graph orchestration, distributed workers and multiple agents are optional. Durability matters when interruption would lose valuable work. A browser matters when the task needs browser interaction. Choose components from the work and its failure modes.
A Ruby-first implementation could use a small Ruby coordinator, SQLite or Postgres for task state, and narrowly defined tools behind provider adapters. That is a design option, not an implementation announcement. There is no requirement to adopt an entire framework.
Illustrative task: a Ruby pricing function rounds a discount incorrectly. The user authorizes a local fix and explicitly excludes deployment.
A crash after an edit should trigger inspection of actual files before replaying it. A tool outage should become an explicit failure or pause, not an invented test result. The harness makes those distinctions visible.
The agent harness performs work. An evaluation harness runs controlled tasks against that system and measures behavior. It can supply fixtures, tools or simulators, task datasets, scorers and reports. Shared components do not make the two purposes identical.
For the pricing example, evaluate correct output, unchanged unrelated files, respected deployment boundaries, interruption recovery, cost and elapsed time. Use held-out tasks; do not weaken a check to turn failure into success. Repeat stochastic runs and compare recorded configurations. A polished final answer is not a passing patch.
Inspect is a concrete evaluation framework to study. Its dataset, solver and scorer concepts help separate the task inputs, model interaction and assessment. Those concepts are useful even if a small project implements its tests directly in Ruby.
Keep: Ruby-first simplicity, provider adapters, minimal diffs and evidence-led delivery. The public Ruby project instructions already require tests first, independent judgment, no weakened evaluations, held-out outcomes and side-effect/cost/latency checks, plus deployment and rollback checks. These are existing standards, not gaps discovered by this article.
Make concrete per task: authorized scope, ownership, tool effects, versioned evidence and completion criteria. Follow the agent collaboration protocol for isolated edits, review and release ownership. Written standards alone do not prove every enforcement mechanism is installed.
Add only when evidence warrants it: checkpointing for costly interruptions, retrieval for repeated knowledge needs, and tracing sufficient to explain failures. Pilot one bounded Ruby maintenance task; success means its acceptance tests pass, prohibited effects stay at zero and measured cost/latency fit a declared budget.
Skip by default: a vector database, elaborate orchestration graph or worker swarm without a demonstrated need. Read the broader AI harness landscape for market context and the Karpathy field guide for the ideas behind agentic engineering and verifiability.
Start with a concrete loop, then inspect interfaces, persistence and evaluation. Product terminology and APIs change; these links were reviewed for this October 16, 2026 guide. The component taxonomy and worked examples above are our synthesis.
Provided as is, for information only, with no warranty and no liability. Not legal or professional advice. Security and radio material is for authorized, lawful use only. Full disclaimer.
Visits are logged on our own server: IP address, browser and device details, and pages viewed. We use them to understand traffic and spot attacks, and never share or sell them.