Results

Unified comparison under matched training budget and checkpoint rule

This comparison matches the training budget of ATLA-PPO, ATLA-SA-PPO, RAMBO and RORL to that of the standard RL references and reports RFQI at its last checkpoint, as every other method. Everything else is unchanged.

Nominal score and score under shift on the library-wide grid, original → unified.

MethodNominalScore under shift
ATLA-PPO72.4 → 78.653.4 → 55.5
ATLA-SA-PPO77.6 → 72.459.6 → 55.5
RFQI79.3 → 62.343.6 → 42.0
RORL79.3 → 63.845.3 → 39.4
RAMBO95.5 → 85.753.7 → 46.7
Pooled standard and robust scores across the severity quartiles and the gains of ten robust methods, in the unified comparison.
Figure 1. Left: pooled Standard and Robust scores across the severity quartiles Q1–Q4, with the gap at Q1 and Q4. Right: the gain of each method over its own base algorithm (left group of each block) and over the mean of the standard RL references of the same regime (right group), online and offline.
Retention across the severity quartiles in the unified comparison, for standard against robust RL, online against offline, and the five mechanism families.
Figure 2. Retention: the score under shift divided by the method's nominal score. Left: Standard RL against Robust RL. Middle: online against offline; solid lines are robust methods and dashed lines standard methods. Right: the five mechanism families.
Isolated shift evaluation of the online methods in the unified comparison, with the leading method of each shift source marked. Isolated shift evaluation of the offline methods in the unified comparison, with the leading method of each shift source marked.
Figure 3. Isolated shift evaluation, online (top) and offline (bottom), of 14 robust and 2 standard RL methods across 50 configurations. The star and the printed value mark the leading method of each shift source, and the dashed line is the mean nominal score of the row. Semantic binding is evaluated only online, on the goal-conditioned causal door task.
Score of each robust method minus the score of its base algorithm per shift source, in the unified comparison. Compound shifts separated by shift source for offline FWM on Hopper and online ATLA-SA-PPO on Pusher.
Figure 4. Left: score of each robust method minus the score of its base algorithm, per shift source of the isolated evaluation. Right: compound shifts separated by shift source, for offline FWM on Hopper and online ATLA-SA-PPO on Pusher; the bars show each shift source in isolation and all of them in composition, and the dashed line marks the method's nominal score.

What changes

The conclusions of the original setup hold in this comparison.

Full results

Normalized score of every robust method on the library-wide grid in the unified comparison, by task family, regime and mechanism family.
Figure 5. The library-wide grid. Every coloured bar is one of the fourteen robust methods, keyed by mechanism family and regime.
Online results per configuration of the library-wide grid in the unified comparison, first part. Online results per configuration of the library-wide grid in the unified comparison, second part.
Figure 6. Online results per configuration of the library-wide grid.
Offline results per configuration of the library-wide grid in the unified comparison, first part. Offline results per configuration of the library-wide grid in the unified comparison, second part.
Figure 7. Offline results per configuration of the library-wide grid.
Online results per condition of the isolated shifts in the unified comparison.
Figure 8. Online results per condition of the isolated shifts.
Offline results per condition of the isolated shifts in the unified comparison.
Figure 9. Offline results per condition of the isolated shifts.
Compound shift results of every method in the unified comparison.
Figure 10. Compound shift results of every method.