Methods of the library under the compound shift of four tasks, in the setting of the unified comparison. On Hopper, Walker2d and HalfCheetah the compound shift combines action, latency and dynamics shifts; on Pusher it combines latency, observation, semantic and dynamics shifts. Retention is the score under the compound shift divided by the nominal score.
Composition is hard for every method. On Hopper, Walker2d and Pusher no method keeps half of its nominal score. On HalfCheetah, nine of the twelve offline methods keep 32% to 47%.
A high retention can come from a low nominal score. RORL on HalfCheetah (nominal 26.5) and RFQI on Hopper (nominal 43.6) lead their tables on retention, while several methods score higher under the compound shift.
nominal scorescore under the compound shift
Order rows by
Hopper, offline
| Method | Nominal | Compound | Retention | |
|---|---|---|---|---|
| RFQI | 43.6 | 13.9 | 32.0% | |
| MOPO standard | 33.8 | 6.8 | 20.1% | |
| FWM | 101.1 | 16.9 | 16.7% | |
| TD3+BC standard | 110.2 | 18.1 | 16.5% | |
| RORL | 111.1 | 18.0 | 16.2% | |
| ROMB | 99.1 | 15.9 | 16.0% | |
| RAMBO | 93.2 | 14.5 | 15.5% | |
| PLR-PVL | 106.7 | 16.1 | 15.1% | |
| ATLA-IQL | 107.2 | 15.6 | 14.6% | |
| RSC-IQL | 100.8 | 13.8 | 13.7% | |
| SynthER standard | 110.5 | 13.9 | 12.6% | |
| IQL standard | 106.8 | 13.4 | 12.5% | |
| TD3+BC standard | 110.2 | 18.1 | 16.5% | |
| RORL | 111.1 | 18.0 | 16.2% | |
| FWM | 101.1 | 16.9 | 16.7% | |
| PLR-PVL | 106.7 | 16.1 | 15.1% | |
| ROMB | 99.1 | 15.9 | 16.0% | |
| ATLA-IQL | 107.2 | 15.6 | 14.6% | |
| RAMBO | 93.2 | 14.5 | 15.5% | |
| SynthER standard | 110.5 | 13.9 | 12.6% | |
| RFQI | 43.6 | 13.9 | 32.0% | |
| RSC-IQL | 100.8 | 13.8 | 13.7% | |
| IQL standard | 106.8 | 13.4 | 12.5% | |
| MOPO standard | 33.8 | 6.8 | 20.1% | |
| RORL | 111.1 | 18.0 | 16.2% | |
| SynthER standard | 110.5 | 13.9 | 12.6% | |
| TD3+BC standard | 110.2 | 18.1 | 16.5% | |
| ATLA-IQL | 107.2 | 15.6 | 14.6% | |
| IQL standard | 106.8 | 13.4 | 12.5% | |
| PLR-PVL | 106.7 | 16.1 | 15.1% | |
| FWM | 101.1 | 16.9 | 16.7% | |
| RSC-IQL | 100.8 | 13.8 | 13.7% | |
| ROMB | 99.1 | 15.9 | 16.0% | |
| RAMBO | 93.2 | 14.5 | 15.5% | |
| RFQI | 43.6 | 13.9 | 32.0% | |
| MOPO standard | 33.8 | 6.8 | 20.1% |
Pusher, online
| Method | Nominal | Compound | Retention | |
|---|---|---|---|---|
| PPO standard | 100.0 | 45.3 | 45.3% | |
| RARL-PPO | 86.4 | 37.4 | 43.3% | |
| ATLA-SA-PPO | 99.9 | 29.2 | 29.3% | |
| ATLA-PPO | 79.6 | 22.4 | 28.1% | |
| DR-SAC | 96.5 | 21.5 | 22.3% | |
| SAC standard | 97.9 | 18.8 | 19.2% | |
| PPO standard | 100.0 | 45.3 | 45.3% | |
| RARL-PPO | 86.4 | 37.4 | 43.3% | |
| ATLA-SA-PPO | 99.9 | 29.2 | 29.3% | |
| ATLA-PPO | 79.6 | 22.4 | 28.1% | |
| DR-SAC | 96.5 | 21.5 | 22.3% | |
| SAC standard | 97.9 | 18.8 | 19.2% | |
| PPO standard | 100.0 | 45.3 | 45.3% | |
| ATLA-SA-PPO | 99.9 | 29.2 | 29.3% | |
| SAC standard | 97.9 | 18.8 | 19.2% | |
| DR-SAC | 96.5 | 21.5 | 22.3% | |
| RARL-PPO | 86.4 | 37.4 | 43.3% | |
| ATLA-PPO | 79.6 | 22.4 | 28.1% |
Hopper, online
| Method | Nominal | Compound | Retention | |
|---|---|---|---|---|
| SAC standard | 75.0 | 14.5 | 19.3% | |
| DR-SAC | 103.3 | 19.4 | 18.7% | |
| ATLA-PPO | 68.5 | 11.6 | 16.9% | |
| ATLA-SA-PPO | 80.1 | 12.1 | 15.1% | |
| RSC-SAC | 78.3 | 11.2 | 14.4% | |
| RARL-PPO | 75.8 | 7.8 | 10.3% | |
| RARL-TRPO | 78.5 | 7.8 | 9.9% | |
| PPO standard | 82.6 | 8.0 | 9.7% | |
| DR-SAC | 103.3 | 19.4 | 18.7% | |
| SAC standard | 75.0 | 14.5 | 19.3% | |
| ATLA-SA-PPO | 80.1 | 12.1 | 15.1% | |
| ATLA-PPO | 68.5 | 11.6 | 16.9% | |
| RSC-SAC | 78.3 | 11.2 | 14.4% | |
| PPO standard | 82.6 | 8.0 | 9.7% | |
| RARL-PPO | 75.8 | 7.8 | 10.3% | |
| RARL-TRPO | 78.5 | 7.8 | 9.9% | |
| DR-SAC | 103.3 | 19.4 | 18.7% | |
| PPO standard | 82.6 | 8.0 | 9.7% | |
| ATLA-SA-PPO | 80.1 | 12.1 | 15.1% | |
| RARL-TRPO | 78.5 | 7.8 | 9.9% | |
| RSC-SAC | 78.3 | 11.2 | 14.4% | |
| RARL-PPO | 75.8 | 7.8 | 10.3% | |
| SAC standard | 75.0 | 14.5 | 19.3% | |
| ATLA-PPO | 68.5 | 11.6 | 16.9% |
Walker2d, online
| Method | Nominal | Compound | Retention | |
|---|---|---|---|---|
| DR-SAC | 83.5 | 33.3 | 39.9% | |
| SAC standard | 104.4 | 36.9 | 35.4% | |
| RSC-SAC | 89.2 | 27.7 | 31.1% | |
| ATLA-SA-PPO | 63.8 | 13.4 | 21.0% | |
| PPO standard | 81.7 | 7.3 | 8.9% | |
| RARL-TRPO | 105.1 | 9.1 | 8.7% | |
| ATLA-PPO | 81.2 | 5.8 | 7.2% | |
| RARL-PPO | 115.4 | 7.2 | 6.2% | |
| SAC standard | 104.4 | 36.9 | 35.4% | |
| DR-SAC | 83.5 | 33.3 | 39.9% | |
| RSC-SAC | 89.2 | 27.7 | 31.1% | |
| ATLA-SA-PPO | 63.8 | 13.4 | 21.0% | |
| RARL-TRPO | 105.1 | 9.1 | 8.7% | |
| PPO standard | 81.7 | 7.3 | 8.9% | |
| RARL-PPO | 115.4 | 7.2 | 6.2% | |
| ATLA-PPO | 81.2 | 5.8 | 7.2% | |
| RARL-PPO | 115.4 | 7.2 | 6.2% | |
| RARL-TRPO | 105.1 | 9.1 | 8.7% | |
| SAC standard | 104.4 | 36.9 | 35.4% | |
| RSC-SAC | 89.2 | 27.7 | 31.1% | |
| DR-SAC | 83.5 | 33.3 | 39.9% | |
| PPO standard | 81.7 | 7.3 | 8.9% | |
| ATLA-PPO | 81.2 | 5.8 | 7.2% | |
| ATLA-SA-PPO | 63.8 | 13.4 | 21.0% |
HalfCheetah, offline
| Method | Nominal | Compound | Retention | |
|---|---|---|---|---|
| RORL | 26.5 | 22.9 | 86.4% | |
| TD3+BC standard | 96.9 | 51.6 | 53.3% | |
| RFQI | 61.6 | 32.0 | 52.0% | |
| RSC-IQL | 99.9 | 46.8 | 46.9% | |
| FWM | 123.7 | 52.6 | 42.5% | |
| ROMB | 120.4 | 49.9 | 41.4% | |
| IQL standard | 119.6 | 49.5 | 41.3% | |
| ATLA-IQL | 112.3 | 46.0 | 40.9% | |
| PLR-PVL | 117.9 | 47.7 | 40.4% | |
| SynthER standard | 109.6 | 39.2 | 35.7% | |
| RAMBO | 115.5 | 38.7 | 33.5% | |
| MOPO standard | 68.9 | 22.2 | 32.3% | |
| FWM | 123.7 | 52.6 | 42.5% | |
| TD3+BC standard | 96.9 | 51.6 | 53.3% | |
| ROMB | 120.4 | 49.9 | 41.4% | |
| IQL standard | 119.6 | 49.5 | 41.3% | |
| PLR-PVL | 117.9 | 47.7 | 40.4% | |
| RSC-IQL | 99.9 | 46.8 | 46.9% | |
| ATLA-IQL | 112.3 | 46.0 | 40.9% | |
| SynthER standard | 109.6 | 39.2 | 35.7% | |
| RAMBO | 115.5 | 38.7 | 33.5% | |
| RFQI | 61.6 | 32.0 | 52.0% | |
| RORL | 26.5 | 22.9 | 86.4% | |
| MOPO standard | 68.9 | 22.2 | 32.3% | |
| FWM | 123.7 | 52.6 | 42.5% | |
| ROMB | 120.4 | 49.9 | 41.4% | |
| IQL standard | 119.6 | 49.5 | 41.3% | |
| PLR-PVL | 117.9 | 47.7 | 40.4% | |
| RAMBO | 115.5 | 38.7 | 33.5% | |
| ATLA-IQL | 112.3 | 46.0 | 40.9% | |
| SynthER standard | 109.6 | 39.2 | 35.7% | |
| RSC-IQL | 99.9 | 46.8 | 46.9% | |
| TD3+BC standard | 96.9 | 51.6 | 53.3% | |
| MOPO standard | 68.9 | 22.2 | 32.3% | |
| RFQI | 61.6 | 32.0 | 52.0% | |
| RORL | 26.5 | 22.9 | 86.4% |