RobustRLlib: A Unified Library and Benchmark for
Robust Reinforcement Learning Algorithms

01 Two barriers, one platform

Robust RL has produced many mechanisms, but each is developed, trained and reported inside its own setting. RobustRLlib fixes the unit of comparison at the algorithm rather than adding another task suite.

What RobustRLlib contributes

03 The disruptor toolbox

A deployment shift rarely changes an MDP wholesale; it enters at an identifiable point of the interaction loop — the sensor, the actuator, the physics, the timing, the reward, or the task itself. RobustRLlib describes a shifted environment by one parameter vector

whose components intervene on observation, actuation, physical dynamics, timing, reward, cost and task semantics, while θ = θ₀ recovers the nominal problem exactly. Because each shift has its own component, the toolbox can tell failure modes apart instead of folding them into a single robustness score: a method that tolerates θ^o and breaks under θ^p is reported as exactly that.

Six shift sources

Five modes, and every specification is composable

Graded sim-to-real scenarios

Single-axis sweeps isolate a mechanism, but deployments combine uncertainty sources. Three levels therefore nest progressively richer conditions — L₁ ⊂ L₂ ⊂ L₃ as sets of active components — so a method can be located by the first level at which it breaks.

05 What the capability map shows

Under one protocol, robust methods as a group outperform standard references — but the picture is not uniform, and the shift a method claims is a poor predictor of the shift it actually defends.

1. Robust RL helps at every severity, but everyone degrades as shifts intensify. Robust methods achieve a higher pooled normalised score than standard RL in all four severity quartiles, but their advantage narrows from +10.3 points in Q1 to +5.1 in Q4 while both groups' scores decline. Robustness raises the whole degradation curve without moving the point at which it collapses.
2. Dynamics-aware mechanisms are stronger under dynamics shifts. Environment-centric online methods and generative offline methods show some of the largest gains on physical shifts, consistent with their exposure to varied dynamics or explicit modelling of trajectories. Relative to their own base algorithms, online robustness mechanisms yield larger incremental gains, whereas offline methods benefit more from the robustness of their base learners.
3. Claimed channels do not fully predict empirical strengths. On isolated sweeps, DR-SAC leads both dynamics and execution latency, while ATLA-SA-PPO leads observation and action; the offline leaders vary across channels. The measured ranking is also conditional on the task, the mode and the severity used to instantiate each shift.
4. Dynamics is the common core shift; the second most damaging shift is task-specific. Dynamics alone halves the score on manipulation and is the worst single source in both tested tasks. Locomotion then loses about three quarters of its score to latency — a delayed action on a balancing gait is a fall, not a slower trajectory — while the same delay leaves manipulation essentially untouched, where instead semantic shift costs a quarter. Compound conditions are worse than any of their parts, dropping the two most robust algorithms below half of nominal on manipulation and below a fifth on locomotion.
5. Robustness transfers return but not constraint satisfaction. Constraint satisfaction is what makes a transfer deployable, and return alone does not imply it: on the Franka drawer, DR-PPO matches SPiDR's 99.6% success with 0% of episodes inside the joint-limit cost budget against SPiDR's 84%.
6. Shift channels differ sharply in how much they hurt a VLA policy. At the largest measured severity a fine-tuned π_0.5 policy retains 0.95 under gravity but only 0.33 under image blur, with execution delay in between. Blur at its largest severity moves only 13% of the image yet drops the policy to a third of nominal, while brightness moves the same frames through the same channel at seven times that dose and costs almost nothing — semantic shift is a core factor for action chunks.

Taken together: evaluating dexterous, high-dimensional control needs more than one task or one configuration. Shift channel, mode, severity, dataset quality, task family and base learner each expose different capabilities and limits of current methods, which makes a shared library and a common protocol the precondition for knowing what a robust RL algorithm actually buys.

07 Using the library

One interface for training, one for declaring a shift, and one for evaluation. The snippets below sketch the workflow; the released code, configuration files and evaluation records follow the same structure.

Declaring isolated shifts

# A deployment condition is one parameter vector, declared channel by channel.
from robustrllib import make_shifted_env, ShiftSpec

specs = [
    # theta^p — a randomised physical parameter, drawn once per reset
    ShiftSpec(channel="dynamics",  mode="loguniform", param="mass",    severity=[0.7, 1.4], resample="per_reset"),
    # theta^o — an adversarial observation perturbation at an L-infinity radius
    ShiftSpec(channel="observation", mode="adversarial",                severity=0.10,      resample="per_step"),
    # theta^tau — an interpolated observation delay of 4 control steps
    ShiftSpec(channel="latency",    mode="interp", param="obs_delay",       severity=4,         resample="per_episode"),
]

env = make_shifted_env("Hopper-v5", specs, seed=0)

Graded sim-to-real conditions and evaluation

# L1 / L2 / L3 nest progressively: L1 is theta^p only, L3 adds the timing channel
# together with a time-varying schedule g(t; phi).
from robustrllib import scenario, evaluate

env = scenario("L3", "Unitree-G1", schedule="drift")

report = evaluate(
    policy, env,
    nominal=True,                       # include the unshifted condition
    severities="quartiles",              # report Q1-Q4 of this ladder's own length
    episodes=20, seeds=5,             # 5 training seeds x 20 paired episodes
    checkpoint="exact_last",           # predeclared rule, logged in the run manifest
)
print(report.normalized_score)          # S(R) = 100 (R - R_min) / (R_max - R_min)
print(report.paired_gain_over_base)    # gain over the method's own backbone

Code, trained policies and the full evaluation records are released with the benchmark; the API names shown here mirror the released package layout.