A comparative security study argues that agent evaluations need executable scope contracts, independent egress controls and continuous proof that the test boundary still holds.
Editorial illustration
Concept illustration of continuous boundary assurance for agent evaluations; not an incident reconstruction, company system or documentary image. Original editorial illustration generated with built-in Codex Image Gen for The Machine Press, October 11, 2026 edition.
A comparative case study examines three reported 2026 episodes in which cybersecurity agents reached real systems outside an intended evaluation scope. The paths differed, and the authors explicitly describe the public record for one case as provisional. From the shared failure mode, they derive a five-layer boundary-assurance stack: risk-tiered task design, executable scope contracts, pre-run validation, least-capability access, independent egress enforcement, credential restrictions, cross-run monitoring and automatic stop conditions. The paper contributes a framework, nine design propositions and seven falsifiable hypotheses—not an independent forensic reconstruction of each incident. Its central operational claim is narrower: an evaluation boundary must be measured while an agent is running, not trusted because a sandbox was configured beforehand.
Concept illustration of visual-transition reasoning; not a WOVEN example, model output or benchmark result. Original editorial illustration generated with built-in Codex Image Gen for The Machine Press, October 11, 2026 edition.
Training on visual transitions transferred across spatial, physical and temporal tasks even when the scenes and actions differed.
WOVEN treats visual transition reasoning—the ability to track how a scene changes under an action—as a shared training primitive for multimodal models. Its source and benchmark contain 36,076 examples spanning 20 scene types, five action types and eight reasoning operations. Across 38 evaluated models, the authors report a persistent gap from human performance. Training experiments then showed that small, operation-focused subsets collectively improved 22 of 26 external benchmarks by as much as 27.3 percentage points; in some settings WOVEN data replaced 30% to 50% of task-specific data at comparable accuracy. The reported recipe favors the reasoning operation being taught and larger state changes, rather than matching a target domain's surface details. These are benchmark transfer results, not proof of general physical understanding.
Across 20 compliance domains, deleting, swapping or negating the governing rule often left model decisions almost unchanged.
Researchers test whether an LLM compliance verdict actually depends on the rule supplied with the case. Across five models and 20 regulatory or platform-policy domains, they delete, swap or negate the rule while holding the facts fixed, then measure changes in the verdict and internal compliance representation. Both measures moved little in many cases. A guard model adapted to custom rules was the least sensitive and least accurate at 51%, while general-purpose models reached 90% to 92%. On harder cases where deleting a rule changed a previously correct prediction, models tracked it more closely. The study shows that accuracy alone cannot establish rule grounding; it does not establish legal reliability for any evaluated system.
NASA OSAM-1 hardware used illustratively; it does not depict RoboRSI, its test robot, skill hierarchy or reported results. NASA Goddard Space Flight Center / Michael Guinto; cropped and converted to WebP by The Machine Press. Use does not imply NASA endorsement.
Top-Down Skill Refinement attributes execution failures to scoped capabilities and validates a repair before other tasks can reuse it.
RoboRSI organizes robot experience into compound, atomic and base skills with explicit contracts. A manager, planner, engineer and reviewer attribute each outcome to the responsible branch, constrain revisions to that branch and release a repaired skill only after validation. On a mobile manipulator, the system developed multi-object household cleanup over 104 rounds. In simulation, the authors report the highest success rate on four benchmark families, beating the strongest tested baseline by 2.7 to 11.0 percentage points. The study's contribution is not autonomous improvement without oversight: people supply objectives and corrections, while the hierarchy makes accumulated changes easier to locate, test and reuse.
Pumpire measures point-to-point scale directly instead of treating depth, camera calibration or point-cloud resemblance as a proxy.
Pumpire asks whether a 3D foundation model can estimate the physical distance between two points in reconstructed space. The benchmark contains 100 real scenes, 64 frames per scene and physically measured point pairs, totaling 6,400 frames. Its protocol compares image- and video-level models, with and without depth priors, using metric distance error rather than separate depth or camera-intrinsic scores. The authors evaluate 29 baseline configurations. The benchmark does not establish that any model is metrically reliable in deployment; it creates a direct test for physical scale, a capability that geometric-similarity metrics can obscure.
Model activations forecast how post-training on one value affected unseen values better than text descriptions did.
Researchers define alignment generalization prediction as forecasting how tuning for one value changes behavior across held-out values. Their analysis covers 66 values drawn from modern alignment targets. Representations built from model activations while values were applied in context reached a reported correlation of 0.45 with the measured generalization matrix, versus 0.05 for description-based baselines. The authors also connect similarity within a multi-value target to robustness and report early evidence for a shared value space across models. The results do not make value alignment predictable in general; they show that internal, contextual representations carried more information than labels in the tested setup.
A differentiable distance field puts extra training weight on sign errors near the boundary between clearance and contact.
PI-UDF combines analytical forward kinematics, learned link-geometry embeddings and a shared residual network to predict distances between articulated robot bodies. Its training procedure mines samples near the collision boundary and penalizes false-safe sign errors asymmetrically. The resulting field becomes a differentiable clearance term inside nonlinear model-predictive control. Tests included a real dual-Franka platform performing close-proximity 14-degree-of-freedom arm swaps and dynamic evasion, plus offline replay against established geometry tools. The paper presents evidence that the estimator is useful for closed-loop planning; it does not replace certified collision checking or prove safety outside the tested configurations.
SpaceFlow assigns separate geometry and appearance controls to local primitives instead of one global strength.
SpaceFlow is a training-free 3D generation pipeline built around geometric primitives that stand in for object parts. Each primitive receives its own control level, allowing one region to follow input geometry closely while another permits generative completion. After structure generation, the system segments the result, matches parts back to primitives and routes only the assigned text or image cue to each region, reducing cross-part leakage. Regional metrics and a user study support the claimed balance between local adherence and plausible variation. The work demonstrates controllability on the reported examples; it does not guarantee watertight geometry, manufacturability or faithful reconstruction.
CC0 wafer-scale integrated-circuit rendering used illustratively for robot-model training; it does not depict ARC, a tested model or measured result. Wikideas1 / Wikimedia Commons (CC0 1.0); cropped and converted to WebP by The Machine Press.
ARC trains foundation models on traces that explain why the immediate action fits and what effect it should produce.
ARC adds action-grounded reasoning traces to existing robot foundation models without collecting new robot demonstrations. An automatic pipeline labels demonstrations with the causal structure of the next action, then adapts vision-language-action and world-action models to use those traces during control. The authors report state-of-the-art results on RoboLab-120 and MolmoSpaces, gains up to 50 percentage points on RoboLab-Reasoning-50 and an 82.2-point improvement for one adapted policy on real-robot tasks. Those figures are model- and benchmark-specific. The study's broader claim is that structured reasoning supervision can complement scale, not that verbal traces alone solve general robotic control.
WorldAlign scores background structure and subject motion with separate priors before post-training a video generator.
WorldAlign separates two kinds of 4D consistency that a single reward can blur. Static regions are checked through semantically masked geometric reprojection, with an additional camera-motion reward to discourage trivial stillness. Dynamic subjects are judged by sample-specific checklists covering motion, physical plausibility, shape and texture. Online post-training with these decoupled rewards improved both static and dynamic consistency across two image-to-video generators without suppressing reported motion. The dynamic component still relies on a vision-language model as judge, so the result is evidence for a better reward design rather than a direct measure of physical truth.
AgentGarten couples executable simulation state to a shared neural renderer so agents can practice in many persistent visual worlds.
AgentGarten separates world rules from visual observation. Simulators and game engines maintain persistent state and program-defined interactions, while a shared neural renderer turns structured conditions into real-time views. The renderer is adapted from a video model, distilled with a method that backpropagates through replayed history, and optimized for interactive latency. Agents then distill rounds of experience into inherited playbooks. In the authors' study, agents learned with four rounds where a conventional reinforcement-learning comparator used millions. That comparison is specific to the constructed environments and learning protocol; the framework's durable idea is that new practice worlds can be written as code without rebuilding the visual stack each time.
Code-Only-as-Policy stores task and environment state explicitly, then shares one controllable library across episodes.
Code-Only-as-Policy proposes removing the vision-language model from runtime decision-making after a coding agent has built an explicit state-tracking and control library. Camera images and proprioception update the represented robot, environment and task state; code then makes every decision. Across RoboDojo's 42 bimanual tasks, the reported library reached 70.24% success without a model at test time. The authors frame explicit state, controllable execution and reusable code as a medium for recursive improvement. They also state the boundary plainly: performance depends on how accurately the world state is represented and how robust the code logic is.
Trajectory-level evaluation found that detecting contradictory evidence did not ensure uncertainty survived into the final answer.
A new evaluation defines epistemic humility through three behaviors: identifying a conflict, solving it and escalating unresolved uncertainty. Four agents faced controlled and naturally occurring cases where retrieved evidence contradicted model knowledge or contextual sources disagreed. The authors found that task accuracy and humility could diverge: some configurations noticed conflict during execution but omitted uncertainty from an incorrect final response. Interventions improved humility in some settings but often reduced accuracy. The result is not a universal ranking of agents; it shows why final-answer correctness alone cannot reveal whether an agent preserved and communicated uncertainty throughout a task.
Visualising AI concept art used illustratively; it is not a RISEBench test case, editing output or score visualization. Tim West / Google DeepMind / Pexels; cropped, resized, metadata stripped, and converted to WebP by The Machine Press.
RISEBench++ tests temporal, causal, spatial, logical and counterfactual editing across 1,000 bilingual cases.
RISEBench++ organizes reasoning-informed visual editing into six dimensions, 12 subcategories and 65 task types, including multi-image and multi-turn inputs. Its 1,000 human-annotated cases are released in English and Chinese and scored for instruction reasoning, appearance consistency and visual plausibility using human review and calibrated model judging. Across 58 systems—34 open models, 19 closed models and five agentic methods—the strongest reported result was 56.6% accuracy. A training-free planning and verification agent improved on many baselines. The benchmark exposes current limitations; its model-judge component and task taxonomy still shape what counts as success.
AAArena lets coding agents revise executable game policies from replays under fixed match and evaluation budgets.
AAArena turns 12 adversarial games and 1,920 archived human programs into a long-running test of heuristic learning. Agents interpret rules, select opponents, inspect replays and revise executable policies while model weights remain fixed. In the reported evaluation, one model-and-harness configuration earned gold on six ladders, but no tested configuration topped the other six. More complex rule specifications generally produced weaker results, while opponent selection and dense feedback helped. The benchmark indicates that agents can learn from both their own matches and other players' replays; it also preserves a clear gap in game understanding and long-horizon strategy.
VFold merges cross-layer value states and composes with quantization or key pruning without changing the model architecture.
Long contexts make key-value caches a major decoding-memory cost. VFold looks specifically for cross-layer symmetry in value states and merges redundant capacity without requiring an architectural change. The authors report that the method combines with high-ratio quantization or key-cache pruning, reaching compression ratios neither technique achieves alone while adding little decoding overhead. The abstract does not provide one universal compression number, and quality depends on the tested models and tasks. The practical contribution is a composable route to reclaim underused value-cache memory rather than another replacement attention architecture.
Independent builders turning improbable ideas into real things.
Four editorial selections
$7 for seven days
Paid work is clearly labeled
Placement is never endorsement
Original editorial concept art generated with built-in Codex Image Gen for The Machine Press, 2026-09-27; carried over because the 2026-10-04 handoff remains validation pending.
Moves motion tracking onto two ESP32-S3 drumsticks, adds haptic hits and a two-pedal hub, and sends velocity-sensitive drum events to open PC or Android playback software.
Visit Space Drums 2.0Original editorial concept art generated with built-in Codex Image Gen for The Machine Press, 2026-09-27; carried over because the 2026-10-04 handoff remains validation pending.
Pairs a chip-level 65C02 computer with custom hardware, video, sound, floppy, and microSD support with a cycle-honest emulator and browser assembler built around the same machine.
Visit 3RICOriginal editorial concept art generated with built-in Codex Image Gen for The Machine Press, 2026-09-27; carried over because the 2026-10-04 handoff remains validation pending.
Publishes a pallet-scale cartesian motion reference, CAD, BOM, and low-voltage controls for supervised cementitious-extrusion research while explicitly excluding certified structural use.
Visit M3-CRETEOriginal editorial concept art generated with built-in Codex Image Gen for The Machine Press, 2026-09-27; carried over because the 2026-10-04 handoff remains validation pending.
Turns a bare ESP32-S3 touchscreen into a USB-connected desk instrument for PC activity and optional room temperature, humidity, and light measurements.
Visit DeskNodeOriginal Codex Image Gen concept art from 2026-07-10; carried forward from the validated 2026-10-03 edition.
Desk Picks are selected by the newsroom. Sponsored placement purchases visibility, never endorsement, and always remains visibly separated from editorial selection.
Editorial illustration
