Results

Isolated shifts on 50 configurations

Every method is evaluated on the same 50 isolated configurations: five shift sources (observation, action, dynamics, reward and execution latency) with five conditions each, online and offline. Every shift is applied to a frozen policy. Under the reward shift the policy acts in the nominal environment and the reward recorded for its episode is corrupted, by Gaussian noise, uniform noise or a constant shift.

Isolated shift evaluation of the online methods on six shift sources, with the leading method of each source marked. Isolated shift evaluation of the offline methods on five shift sources, with the leading method of each source marked.
Figure 1. Isolated shift evaluation, online (top) and offline (bottom), of 14 robust and 2 standard RL methods across 50 configurations. The star and the printed value mark the leading method of each shift source, and the dashed line is the mean nominal score of the row. Semantic binding is evaluated only online, on the goal-conditioned causal door task.

Claimed channels do not fully predict empirical strengths. Figure 1 shows DR-SAC leads both dynamics and execution latency, while ATLA-SA-PPO leads observation, action and reward; offline leaders vary across shift channels. A method's intended target therefore does not uniquely determine where it performs best. The measured ranking is also conditional on the task, mode and severity used to instantiate each shift.

Score of each robust method minus the score of its base algorithm on five shift sources, online and offline.
Figure 2. Paired gain over the base algorithm: the score of each robust method minus the score of its base algorithm, per shift source.

Online methods gain over their base algorithm; offline methods stay close to it. Every online method improves over its base algorithm on most shift sources. Offline, robust methods stay close to their base algorithm.

Results per condition

Online results per condition of the isolated shifts.
Figure 3. Online results per condition of the isolated shifts.
Offline results per condition of the isolated shifts.
Figure 4. Offline results per condition of the isolated shifts.

The 25 conditions

Each condition is evaluated online and offline, which gives the 50 configurations. All of them are applied to the frozen policy on Hopper; a control step is 8 ms.

Shift sourceConditionModeSetting
Observationobs_rel_0.10stochastic, per stepGaussian at 0.10 σi, σi the dimension's own std
Observationobs_uni_0.10stochastic, per stepU[−0.10, 0.10]
Observationobs_gauss_0.10stochastic, per stepσ = 0.10
Observationmad_0.02adversarial, per stepten-step PGD in an ℓ∞ ball of radius 0.02 maximising the change of the action
Observationmad_0.05adversarial, per stepas above, radius 0.05
Actionact_uni_0.10stochastic, per stepU[−0.10, 0.10] on the clipped action
Actionact_gauss_0.10stochastic, per stepσ = 0.10 on the clipped action
Actionact_offset_0.05static bias+0.05 on every actuator
Actionact_oppose_0.10adversarial, per stepε = 0.10 against the action just chosen
Actioncmd_oppose_0.20adversarial, per stepε = 0.20, applied to the command before clipping
Dynamicsgear_static_0.8static parametricactuator gear × 0.8 for the whole episode
Dynamicsgear_ramp_0.8non-stationarygear 1.0 → 0.8, linear over 1000 steps
Dynamicsgear_sine_p500non-stationarygear cycling in [0.8, 1.2], period 500 steps
Dynamicspush_60event-driven60 N on the torso, 12 steps every 250, sign random
Dynamicstrans_t1stochastic kernelstate noise σq = 10−3, σv = 10−2 per step
Rewardrew_gauss_0.5stochastic, per steprecorded reward + N(0, 0.52)
Rewardrew_gauss_1.0stochastic, per steprecorded reward + N(0, 1.02)
Rewardrew_uni_0.866stochastic, per steprecorded reward + U[−0.866, 0.866] (std 0.5)
Rewardrew_shift_-0.5parametricrecorded reward −0.5 at every step
Rewardrew_shift_-1.0parametricrecorded reward −1.0 at every step
Latencyobs_delay_4mslatency, observationdelay drawn from [2, 4] ms, interpolated
Latencyobs_delay_8mslatency, observationdelay drawn from [4, 8] ms
Latencyobs_delay_16mslatency, observationdelay drawn from [8, 16] ms
Latencyact_delay_8mslatency, actionfixed buffer of 1 control step (8 ms)
Latencyact_delay_rand1latency, non-stationarybuffer redrawn every step, {0, 1} steps

The same figures under the matched training budget and checkpoint rule are in the unified comparison.