Improve manual_optimization

[tchaton]

🚀 Feature

Motivation

The current manual optimization is not flexible enough.
From discussion with @justusschock and @awaelchli, here is the proposal:

• manual_backward shouldn't take any optimizers as they aren't used internally expect in Horovod. To solve this, in Horovod, call synchronise on all optimizers.
• In LightningOptimizer, optimizer shoudn't follow accumulated_grad_batches.

Pitch

Alternatives

Additional context