FarmOpsBench and the Autonomous Enterprise Benchmark

1. Executive decision

A whole farm is a heterogeneous system: humanoids, tractors, treatment UGVs, drones, fixed irrigation, workers, inventory, weather, crop biology, deadlines, energy, transport and regulation. A fully robotic company adds customer demand, production commitments, quality, storage, maintenance, cybersecurity, finance, supplier dependencies and software release management.

The benchmark must reward selecting the correct embodiment—including not using a humanoid when a specialized machine is safer, faster or cheaper. Humanoids are strongest around human infrastructure, irregular manipulation, tools, repairs and exceptions. Broadacre coverage, high-throughput transport and repetitive processing generally belong to specialized machines.

2. Cutting-edge position in August 2026

Current real-world maturity

3. FarmOpsBench: whole-farm benchmark architecture

Participant belief versus benchmark truth

The operator console must not expose truth during a competitive run. Evaluator and replay modes may reveal it after completion.

Hybrid multi-fidelity farm twin

Farm scales

Benchmark hierarchy

Realistic humanoid roles

4. Hierarchical agent and MCP contract

MCP resource surface

MCP tools

Structured mission contract

{
  "goal": {
    "kind": "repair_irrigation_valve",
    "target": "irrigation/valve-17"
  },
  "assignedAsset": "humanoid/h2",
  "operatingZone": "yard/irrigation-east",
  "window": { "earliest": "06:40", "deadline": "08:00" },
  "preconditions": ["pump-3-isolated", "zone-cleared"],
  "resources": ["tool/valve-key-20mm", "part/seal-17b"],
  "stopConditions": [
    "human-distance-under-3m",
    "unexpected-pressure",
    "localization-confidence-under-0.95"
  ],
  "requiredEvidence": ["before-image", "pressure-test", "after-image"],
  "approvalRef": "approval/maintenance-291",
  "idempotencyKey": "run42-valve17-repair"
}

Natural language may express intent, but the released mission must compile to a typed contract. Scope tools by farm, zone, asset and action class. Validate every input and structured output. Treat tool descriptions and annotations as untrusted. Log every approval, denial and invocation.

5. Benchmark and operations console

6. Scenarios, qualification and scoring

Representative whole-farm scenario: Stormfront Harvest

A strong agent gathers information before committing high-cost assets, detects inconsistent evidence, assigns coverage and repair to appropriate embodiments, builds contingency branches, stops safely around people, replans after failure, preserves the most valuable harvest and produces auditable proof.

Perturbation families

Hard qualification gates

Scorecard

G is the safety qualification gate. Each category is normalized to [0,1]. H is a harmonic mean, preventing one strong category from hiding a zero. The raw metric vector and Pareto frontier remain authoritative.

Report mean utility and lower-tail/CVaR behavior. Normalize against a deterministic reactive baseline and an offline constrained upper bound.

Evaluation protocol

• Public development factors and sealed evaluation factors.
• Paired seeds and held-out combinations, not only held-out random numbers.
• Bootstrap confidence intervals and separate within-site versus between-site variation.
• Active factor selection for expensive real trials.
• Worst-group, failure-region and sim-to-real reporting.
• Frozen model endpoint, policy hash, tool catalog and scenario version.
• Upgrade and schema-migration tests for every release.

Required baselines

1. FIFO or reactive finite-state dispatcher.
2. Rule-based agronomic scheduler.
3. OR-Tools CP-SAT resource-aware planner.
4. Limited-information hierarchical agent.
5. Clairvoyant constrained upper bound used only for normalization.
6. Human dispatcher where practical.

7. Autonomous Enterprise Benchmark

Control layers

Robots as capabilities, not hard-coded devices

Work requests should describe required outcomes and evidence. The orchestration layer selects an eligible robot, machine or human by safety qualification, ODD, availability, expected quality, cost, energy and deadline.

{
  "capabilityId": "equipment.repair.valve",
  "outputs": ["valve-functional", "pressure-test-evidence"],
  "eligibleEmbodiments": ["humanoid", "mobile-manipulator", "human-technician"],
  "preconditions": ["energy-isolated", "replacement-part-available"],
  "operationalDomain": {
    "zones": ["farm-yard", "greenhouse"],
    "temperatureC": [-5, 45],
    "maximumWindMps": 15,
    "humanSeparationM": 3
  },
  "resources": ["20mm-key", "seal-kit"],
  "qualityRequirements": { "maximumLeakKpaPerMinute": 0.2 },
  "evidence": ["before-image", "torque-record", "pressure-test"],
  "failureModes": ["part-mismatch", "unexpected-pressure", "manipulation-failure"],
  "recovery": ["make-safe", "request-technician"],
  "safetyCaseRef": "sc/valve-maintenance/4.2"
}

Farm company reference profile

End-to-end work graph

Customer order
  → production and harvest forecast
  → field inspection
  → harvest schedule
  → robot and attachment allocation
  → collection and transport
  → grading and quality checks
  → packing
  → cold storage
  → outbound loading
  → delivery evidence
  → settlement and compliance record

The episode ends when the correct product reaches the contracted destination at acceptable quality—or when the company safely determines that fulfillment is impossible and executes the appropriate commercial response.

Company benchmark hierarchy

Company scenario: coordinated harvest, quality and release crisis

The system must reforecast capacity, decide which commitments remain feasible, allocate shared machines, roll back the model, isolate inventory, preserve traceability, continue at the disconnected site, manage workers entering the packhouse and calculate customer and financial impact.

Protocol architecture

8. Professional assurance model

Machine-readable operational design domain

Every robot, capability and workflow declares its validated site, zone, terrain, slope, weather, temperature, lighting, visibility, human proximity, crop/product type, payload, network state, sensors, tools, software/model version and required safety infrastructure.

Safety-case hierarchy

A benchmark result is not certification. It should produce reproducible evidence suitable for engineering, risk assessment and a jurisdiction-specific compliance process.

Runtime assurance envelope

Change assurance

Robot firmware, VLA/reasoning models, prompts, MCP schemas, fleet planners, maps, safety configurations, attachments and sensor calibrations all require change-impact evaluation.

Cyber-physical security tests

Use IEC 62443-style zones and conduits separating safety controllers, robot control, site operations, enterprise IT, remote support, model providers and external partners. Test stolen credentials, malicious MCP resources, replayed missions, compromised sensors, GNSS spoofing, network partitions, remote-support takeover, supply-chain compromise and model endpoint substitution.

9. Real-world qualification ladder

Enterprise metrics

Report human attention separately. Otherwise remote operator and teleoperation labor disappears from the business model.

Robotic Enterprise Operations Center

10. Technology and implementation sequence

Technology choices

FarmOpsBench build order

1. Authoritative domain and evaluator: truth/belief separation, event model, deterministic clock, scenario compiler, assertions, signed result and replay.
2. Complete fast FarmDay benchmark: multiple scales, crop/weather dynamics, discrete-event fleet, logistics and deterministic baselines.
3. MCP gateway and console: resources, tools, scoped authorization, durable tasks, operator/evaluator/replay modes and human-attention accounting.
4. Physics-backed humanoid cells: workshop, irrigation, greenhouse and loading tasks linked to the farm simulator through standardized outcomes.
5. Paired real evaluation: instrumented resets, evidence capture, sim-to-real calibration and active selection of trials.
6. Season and cooperative tracks: delayed agronomy, shared fleets, cross-site communication and upgrade governance.

First complete enterprise deployment

1. Canonical work, capability, event, ODD and evidence schemas.
2. Deterministic company twin and benchmark authority.
3. ERP/FMIS, maintenance, inventory and quality adapters.
4. MCP agent gateway with scoped capabilities.
5. ROS 2, ISOBUS, VDA 5050 and OPC UA fleet/system adapters.
6. Operations center and exact replay.
7. Shadow-mode evaluation against actual operations.
8. Bounded robot control with rollback.
9. Matched simulation and physical work cells.
10. Multi-site and lights-out ODD qualification.

11. Primary source index

Sources are grouped by the role they play in the architecture. Access and product claims remain subject to the linked source’s own terms and evidence.