Results

Compound shifts on four tasks

Methods of the library under the compound shift of four tasks, in the setting of the unified comparison. On Hopper, Walker2d and HalfCheetah the compound shift combines action, latency and dynamics shifts; on Pusher it combines latency, observation, semantic and dynamics shifts. Retention is the score under the compound shift divided by the nominal score.

Composition is hard for every method. On Hopper, Walker2d and Pusher no method keeps half of its nominal score. On HalfCheetah, nine of the twelve offline methods keep 32% to 47%.

A high retention can come from a low nominal score. RORL on HalfCheetah (nominal 26.5) and RFQI on Hopper (nominal 43.6) lead their tables on retention, while several methods score higher under the compound shift.

nominal scorescore under the compound shift

Order rows by

Hopper, offline

MethodNominalCompoundRetention
RFQI43.613.932.0%
MOPO standard33.86.820.1%
FWM101.116.916.7%
TD3+BC standard110.218.116.5%
RORL111.118.016.2%
ROMB99.115.916.0%
RAMBO93.214.515.5%
PLR-PVL106.716.115.1%
ATLA-IQL107.215.614.6%
RSC-IQL100.813.813.7%
SynthER standard110.513.912.6%
IQL standard106.813.412.5%
TD3+BC standard110.218.116.5%
RORL111.118.016.2%
FWM101.116.916.7%
PLR-PVL106.716.115.1%
ROMB99.115.916.0%
ATLA-IQL107.215.614.6%
RAMBO93.214.515.5%
SynthER standard110.513.912.6%
RFQI43.613.932.0%
RSC-IQL100.813.813.7%
IQL standard106.813.412.5%
MOPO standard33.86.820.1%
RORL111.118.016.2%
SynthER standard110.513.912.6%
TD3+BC standard110.218.116.5%
ATLA-IQL107.215.614.6%
IQL standard106.813.412.5%
PLR-PVL106.716.115.1%
FWM101.116.916.7%
RSC-IQL100.813.813.7%
ROMB99.115.916.0%
RAMBO93.214.515.5%
RFQI43.613.932.0%
MOPO standard33.86.820.1%

Pusher, online

MethodNominalCompoundRetention
PPO standard100.045.345.3%
RARL-PPO86.437.443.3%
ATLA-SA-PPO99.929.229.3%
ATLA-PPO79.622.428.1%
DR-SAC96.521.522.3%
SAC standard97.918.819.2%
PPO standard100.045.345.3%
RARL-PPO86.437.443.3%
ATLA-SA-PPO99.929.229.3%
ATLA-PPO79.622.428.1%
DR-SAC96.521.522.3%
SAC standard97.918.819.2%
PPO standard100.045.345.3%
ATLA-SA-PPO99.929.229.3%
SAC standard97.918.819.2%
DR-SAC96.521.522.3%
RARL-PPO86.437.443.3%
ATLA-PPO79.622.428.1%

Hopper, online

MethodNominalCompoundRetention
SAC standard75.014.519.3%
DR-SAC103.319.418.7%
ATLA-PPO68.511.616.9%
ATLA-SA-PPO80.112.115.1%
RSC-SAC78.311.214.4%
RARL-PPO75.87.810.3%
RARL-TRPO78.57.89.9%
PPO standard82.68.09.7%
DR-SAC103.319.418.7%
SAC standard75.014.519.3%
ATLA-SA-PPO80.112.115.1%
ATLA-PPO68.511.616.9%
RSC-SAC78.311.214.4%
PPO standard82.68.09.7%
RARL-PPO75.87.810.3%
RARL-TRPO78.57.89.9%
DR-SAC103.319.418.7%
PPO standard82.68.09.7%
ATLA-SA-PPO80.112.115.1%
RARL-TRPO78.57.89.9%
RSC-SAC78.311.214.4%
RARL-PPO75.87.810.3%
SAC standard75.014.519.3%
ATLA-PPO68.511.616.9%

Walker2d, online

MethodNominalCompoundRetention
DR-SAC83.533.339.9%
SAC standard104.436.935.4%
RSC-SAC89.227.731.1%
ATLA-SA-PPO63.813.421.0%
PPO standard81.77.38.9%
RARL-TRPO105.19.18.7%
ATLA-PPO81.25.87.2%
RARL-PPO115.47.26.2%
SAC standard104.436.935.4%
DR-SAC83.533.339.9%
RSC-SAC89.227.731.1%
ATLA-SA-PPO63.813.421.0%
RARL-TRPO105.19.18.7%
PPO standard81.77.38.9%
RARL-PPO115.47.26.2%
ATLA-PPO81.25.87.2%
RARL-PPO115.47.26.2%
RARL-TRPO105.19.18.7%
SAC standard104.436.935.4%
RSC-SAC89.227.731.1%
DR-SAC83.533.339.9%
PPO standard81.77.38.9%
ATLA-PPO81.25.87.2%
ATLA-SA-PPO63.813.421.0%

HalfCheetah, offline

MethodNominalCompoundRetention
RORL26.522.986.4%
TD3+BC standard96.951.653.3%
RFQI61.632.052.0%
RSC-IQL99.946.846.9%
FWM123.752.642.5%
ROMB120.449.941.4%
IQL standard119.649.541.3%
ATLA-IQL112.346.040.9%
PLR-PVL117.947.740.4%
SynthER standard109.639.235.7%
RAMBO115.538.733.5%
MOPO standard68.922.232.3%
FWM123.752.642.5%
TD3+BC standard96.951.653.3%
ROMB120.449.941.4%
IQL standard119.649.541.3%
PLR-PVL117.947.740.4%
RSC-IQL99.946.846.9%
ATLA-IQL112.346.040.9%
SynthER standard109.639.235.7%
RAMBO115.538.733.5%
RFQI61.632.052.0%
RORL26.522.986.4%
MOPO standard68.922.232.3%
FWM123.752.642.5%
ROMB120.449.941.4%
IQL standard119.649.541.3%
PLR-PVL117.947.740.4%
RAMBO115.538.733.5%
ATLA-IQL112.346.040.9%
SynthER standard109.639.235.7%
RSC-IQL99.946.846.9%
TD3+BC standard96.951.653.3%
MOPO standard68.922.232.3%
RFQI61.632.052.0%
RORL26.522.986.4%