You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: fast EntropyBottleneck aux_loss minimization via bisection search
This method completes in <1 second and reduces aux_loss to <0.01.
This makes the aux_loss optimization during training unnecessary.
Another alternative would be to run the following post-training:
```python
while aux_loss > 0.1:
aux_loss = model.aux_loss()
aux_loss.backward()
aux_optimizer.step()
aux_optimizer.zero_grad()
```
...but since we do not manage aux_loss learning rates,
the bisection search method might converge better.
0 commit comments