badbb87

Add support for Unity MLAgents environments.

hyerra May 29, 2023

092e26c

Make Unity Environment support heterogeneous observations and multipl…

hyerra Jun 24, 2023

fcc9fb9

WIP

hyerra Jun 30, 2023

416b02c

WIP

hyerra Jul 8, 2023

eaaabaa

Include an environment property to store agent_id to behavior_name ma…

hyerra Jul 10, 2023

46fb97f

Fix some bugs with action conversion

hyerra Jul 17, 2023

329f911

Store observations in the format obs_{agent_id}_{obs_number} to preve…

hyerra Jul 18, 2023

a54bba0

Remove Envpool from CCI (#1390)

osalpekar Jul 18, 2023

48d9e8b

[Doc] Fix doctest examples (#1393)

degensean Jul 18, 2023

99da166

[BugFix] Fix bug in ppo example config (#1396)

degensean Jul 19, 2023

2fa358a

[BugFix] Fix ppo training NaN occurences (#1403)

Jul 21, 2023

4149579

[Example] Multiagent examples: MAPPO-IPPO-MADDPG-IDDPG-IQL-QMIX-VDN (…

matteobettini Jul 25, 2023

970f65e

[BugFix] Reward sum custom key (#1413)

matteobettini Jul 26, 2023

1b7980c

[Refactor] Use masking in collectors (#1412)

Jul 26, 2023

2f24e4f

[BugFix] Pickable buffer (#1410)

albertbou92 Jul 26, 2023

98b2cca

[Tutorial] Multiagent PPO tutorial (#1385)

matteobettini Jul 26, 2023

a16492b

[Refactor] Remove loss device (#1395)

matteobettini Jul 26, 2023

77ccec4

[BugFix] Fix LSTM use with padded/masked segments (#1399)

smorad Jul 26, 2023

5157824

[CI] Fix tuto deps (#1416)

matteobettini Jul 27, 2023

d6f4129

[BugFix] unlock in tensordictmodules tests (#1417)

Jul 27, 2023

ad7c2d6

[Feature] VMAS obs dict (#1419)

matteobettini Jul 28, 2023

a95dc11

[BugFix] Fix `LazyStackedCompositeSpec` and introducing `consolidate_…

matteobettini Jul 31, 2023

7729312

[BugFix] Fix KLTransform execution with LSTM (#1426)

Jul 31, 2023

09d7702

[Versioning] Python 3.11 (#1433)

Aug 1, 2023

2d6f949

python 3.11 in README (#1434)

Aug 1, 2023

8084835

[BugFix] Fix QValueModule multi_one_hot (#1439)

smorad Aug 1, 2023

7d8a8ba

[Feature] Optional mapping of "state" in gym specs (#1431)

matteobettini Aug 4, 2023

a85a2a7

[BugFix] Fix split_trajs (#1444)

Aug 4, 2023

9cf41c4

[Test] Test split trajs (#1445)

Aug 4, 2023

a5335f3

[Feature] Heterogeneous Environments compatibility (#1411)

matteobettini Aug 4, 2023

99268ae

[Tests] Collector compatibility for heterogeneous environments (#1414)

matteobettini Aug 4, 2023

4217ef6

Fix MacOS Mujoco Failure (#1450)

osalpekar Aug 8, 2023

e89f21f

[BugFix] Fix Cython for D4RL (#1429)

Aug 8, 2023

710fd2b

[Feature] IsaacGymEnvs integration (#1443)

Aug 9, 2023

648c3bc

[Benchmark] Benchmark number GPU vectorised environments in VMAS (Tor…

matteobettini Aug 9, 2023

a0a0d20

[Feature] Auto-DoubleToFloat (#1442)

Aug 11, 2023

fbfaa81

[Refactor] Change objectives parameter/buffer/target logic (#1424)

Aug 11, 2023

93bc5de

[BugFix] CompositeSpec.unsqueeze (#1464)

btx0424 Aug 18, 2023

e4c7315

[BugFix] Discrete SAC rewrite (#1461)

matteobettini Aug 30, 2023

b67b1cf

[Feature] Parallel environments lazy heterogenous data compatibility …

matteobettini Aug 30, 2023

d0f8bcb

[Algorithm] Online Decision transformer (#1149)

BY571 Aug 30, 2023

e9663b6

[CI] Fix Mujoco version (#1475)

Aug 30, 2023

919dcf1

[Feature] Allow multiple (nested) action, reward, done keys in `env`,…

matteobettini Aug 30, 2023

3366f93

[CI] Linux Olddeps tests to Nova (#1289)

osalpekar Aug 30, 2023

3c63a58

[Feature] Device transform (#1472)

Aug 30, 2023

29a0565

[BugFix] Fix reward sum within parallel envs (#1454)

Aug 30, 2023

4c5d937

Move to More Efficient Windows Runner (#1476)

osalpekar Aug 30, 2023

7254f83

[Refactor] Faster envs (2) (#1457)

Sep 1, 2023

632078e

[BugFix] Fix RewardSum spec transform to mimic reward spec (#1478)

matteobettini Sep 1, 2023

77cb35e

[BugFix] Fix NoopResetEnv behavior when trials exceeded. (#1477)

skandermoalla Sep 1, 2023

160324c

[BugFix] Fix LSTM - VecEnv compatibility (#1427)

Sep 2, 2023

e96e9de

[BugFix] Fix Gym Categorical/One-hot issues (#1482)

Sep 3, 2023

7444433

[Feature] Masking actions (#1421)

Sep 3, 2023

66063ae

[Doc] Update coding_ppo.py (#1483)

kushaangupta Sep 3, 2023

7152175

[BugFix] Nested key in replay buffer (#1485)

matteobettini Sep 4, 2023

78cd576

[Feature] VMAS: choose between categorical or one-hot actions (#1484)

matteobettini Sep 4, 2023

d70aff6

[CI] Fix examples CI (#1489)

matteobettini Sep 5, 2023

4710106

[Feature] Refactor categorical dists: Masked one-hot and pass-through…

Sep 5, 2023

4709216

[Minor] Missing commit from #1488 (#1490)

Sep 5, 2023

ca38650

[BugFix] Fix action mask test (#1492)

Sep 5, 2023

aa13a67

[BugFix] Fix typo in CatFrames Transform error message. (#1491)

skandermoalla Sep 6, 2023

12ebcce

[Setup] Update setup.py python versions (#1496)

Sep 6, 2023

5608852

[Minor] Capture error in CatFrame edit (#1498)

Sep 6, 2023

d5ddd3d

[Feature] RoboHive integration (#1119)

Sep 6, 2023

a45aaba

[BugFix] Empty-spec tolerance (#1501)

Sep 6, 2023

d8a0bc8

[Doc] correct typos in pendulum tutorial (#1502)

kushaangupta Sep 7, 2023

7ee8f13

[Feature] Making action masks compatible with q value modules and e-g…

matteobettini Sep 7, 2023

c0144fe

[BugFix] Fix DDPG squeezing (#1487)

matteobettini Sep 7, 2023

e28ef5c

[Feature] ClipTransform + rename `min/maximum` -> `low/high` (#1500)

Sep 7, 2023

3cc870a

[BugFix] Multiagent "auto" entropy fix in SAC (#1494)

matteobettini Sep 7, 2023

97d19f3

[Minor] More efficient SAC v1 (#1507)

Sep 7, 2023

e539346

[BugFix] Fix ClipTransform device (#1508)

Sep 7, 2023

3e1d382

[CI] Move linux stable to GHA (#1503)

Sep 8, 2023

0e23485

[BugFix] Add `torch.no_grad()` for rendering in multiagent PPO tutori…

matteobettini Sep 8, 2023

ea16820

[CI] Remove CCI Config (#1456)

osalpekar Sep 8, 2023

074ae61

[Performance] Miscellaneous efficiency improvements (#1513)

Sep 8, 2023

3b30929

[Environment] Petting zoo (#1471)

matteobettini Sep 14, 2023

e08255f

[CI] Fix CI PettingZoo (#1528)

matteobettini Sep 14, 2023

3dbc024

[BugFix] Fix envpool (#1530)

Sep 14, 2023

9b862d3

[Environment, Docs] SMACv2 and docs on action masking (#1466)

matteobettini Sep 15, 2023

549e041

[CI] Fix CI (#1529)

Sep 15, 2023

aadf2e3

[BugFix] Small patches to SMAC (#1533)

matteobettini Sep 15, 2023

a3bf509

[Refactor] Expose all wrappers in torchrl.envs (#1532)

Sep 15, 2023

1873882

[Test] Skip tests in python 3.11 (#1535)

Sep 15, 2023

4000ab4

[BugFIx] Fix entropy signature in truncated normal (#1536)

Sep 15, 2023

776ed06

[CI] Fix habitat CI (#1537)

Sep 15, 2023

4c2a7f9

[Test] Test state_dict and loss modules (#1527)

Sep 15, 2023

7385cd2

[BugFix] Fix SMAC-v2 (#1538)

Sep 15, 2023

297a047

[Feature] Log hparams from python dict (#1517)

matteobettini Sep 15, 2023

3430168

[Features] Conv3dNet and PermuteTransform (#1398)

xmaples Sep 16, 2023

b6929b8

[Feature] Gym 'vectorized' envs compatibility (#1519)

Sep 17, 2023

e28f1e1

[CI] Fix RoboHive CI (#1541)

Sep 18, 2023

ef4e9e0

[Refactor] Fix imports (#1551)

Sep 20, 2023

7070ffd

[Feature] Pettingzoo: add multiagent dimension to single agent groups…

matteobettini Sep 20, 2023

cab3324

[Test] Remove import of test class (#1549)

matteobettini Sep 20, 2023

16bb91c

[Quality] Filter out warnings in subprocs (#1552)

Sep 20, 2023

6aa1444

[BugFix] Sparse info in SMACv2 (#1546)

matteobettini Sep 20, 2023

b63faba

[BugFix] Instantiate 2 losses with different keys (#1553)

matteobettini Sep 20, 2023

5c318cf

[Feature] Unbind specs (#1555)

Sep 20, 2023

4470fc0

[Minor] Missing lint (#1556)

Sep 20, 2023

6f12451

[BugFix] PettingZoo seeding (#1554)

matteobettini Sep 20, 2023

1418bd7

[BugFix] Fix Dreamer test error (#1558)

Sep 21, 2023

ae370e9

[Algorithm] Update PPO examples (#1495)

albertbou92 Sep 21, 2023

e451926

[Feature] make csv logger `exist_ok` on logging folder (#1561)

matteobettini Sep 21, 2023

7714c8f

[BugFix] Load collector frames and iter (#1557)

matteobettini Sep 21, 2023

288c458

[Refactor] Use `wait` instead of `is_set` to get results in ParallelE…

Sep 21, 2023

a7a7163

[Algorithm] Update A2C examples (#1521)

albertbou92 Sep 22, 2023

76903a1

[CI] Add ninja for MacOS builts (#1564)

Sep 22, 2023

bf6db3b

[Feature] Threaded collection and parallel envs (#1559)

Sep 22, 2023

6d442a6

[BugFix] DDPG select also critic input for actor loss (#1563)

matteobettini Sep 22, 2023

0d8d486

[CI] Fix jumanji (#1566)

Sep 22, 2023

610328b

[Test] Skip threading tests in OSX (#1571)

Sep 24, 2023

f5b886e

[BugFix] Fix run_type_checks (#1570)

Sep 24, 2023

d115036

[Tests] DDPG extra critic input tests (#1568)

matteobettini Sep 24, 2023

3d424b9

[Minor] Remove ya gymnasium deprecation warning in vectorized envs (#…

Sep 27, 2023

2d5720e

[Versioning] Write version all cases in setup.py (#1579)

Sep 27, 2023

0c29fcb

[Versioning] Use python 3.8 for GPU tests (#1577)

Sep 27, 2023

f405d76

[Feature] Gym compatibility: Terminal and truncated (#1539)

Sep 29, 2023

78a9961

[BugFix] Improve collector buffer initialisation when policy spec is …

matteobettini Oct 1, 2023

bf8e9c3

[Lint] Add TorchFix linter (#1580)

kit1980 Oct 1, 2023

e89788d

[Feature] Fix DType casting lazy init (#1589)

Oct 2, 2023

256162c

[Performance] Reduce key accessing in transforms (#1590)

matteobettini Oct 2, 2023

c8a29db

[Feature] Make advantages compatible with Terminated, Truncated, Done…

Oct 2, 2023

02b9a75

[BugFix] `RewardSum` transform for multiple reward keys (#1544)

matteobettini Oct 2, 2023

a8f3642

[BugFix] Minor fixes PPO / A2C examples (#1591)

albertbou92 Oct 2, 2023

95476f2

[BugFix] Make VecNorm Transform pickable (#1596)

albertbou92 Oct 3, 2023

b750097

[Algorithm] Update TD3 Example (#1523)

BY571 Oct 3, 2023

f847e69

[Algorithm] Update DDPG Example (#1525)

BY571 Oct 3, 2023

086a7cd

[Algorithm] Update SAC Example (#1524)

BY571 Oct 3, 2023

059763f

[BugFix] Vectorized priority update in replay buffers (#1598)

matteobettini Oct 4, 2023

72e891b

[Feature] CNN version of MultiAgentMLP (#1479)

MarkHaoxiang Oct 4, 2023

e8a1606

[Doc] Fix advantage examples (#1600)

Oct 4, 2023

84f1d8c

[Docs] Fix multi-agent tutorial (#1599)

matteobettini Oct 4, 2023

b64fa55

[BugFix] Fix RLHF tests - transformers v4.34 (#1601)

Oct 4, 2023

acefedf

[Feature] D4rl direct download (#1430)

MateuszGuzek Oct 4, 2023

82fa8f6

[Benchmark] Benchmark Gym vs TorchRL (#1602)

Oct 4, 2023

148e8aa

[Refactor] Refactor DDPG loss in standalone methods (#1603)

Oct 5, 2023

ebf16aa

[Algorithm] Update DT (#1560)

BY571 Oct 5, 2023

f93556e

[Feature] Support for GRU (#1586)

Oct 5, 2023

f9c6e6b

[Feature] End-of-life transform (#1605)

Oct 5, 2023

eea6d36

[BugFix] Patch SAC to allow state_dict manipulation before exec (#1607)

Oct 5, 2023

05e85f1

[CI] Add macOS M1 binaries Wheels (#1504)

DanilBaibak Oct 5, 2023

8148a36

[Algorithm] RLHF end-to-end, clean (#1597)

Oct 5, 2023

11febd9

v0.2.0 branch (#1609)

Oct 5, 2023

a8eac29

[Feature] Warning for `init_random_frames` rounding in collectors (#1…

matteobettini Oct 9, 2023

1016107

[Feature] Add support of non-pickable gym env (#1615)

duburcqa Oct 9, 2023

90ad21c

[BugFix] Add keys to GAE in PPO/A2C (#1618)

Oct 9, 2023

502a2e6

[BugFix] Fix gym benchmark (#1619)

Oct 9, 2023