Possibility to rename logged metrics postfix 'dataloader_idx_X' to the name of the respective dataset for multiple validation datasets

[Vozf]

🚀 Feature

It would be nice to have a possibility to rename currently automatically set metrics postfix dataloader_idx_X to the custom name for better metrics visualization.
Say I have an imagenet and cifar datasets and accuracy metrics. Now I got accuracy_dataloader_idx_0 and
accuracy_dataloader_idx_1 metrics postfix but it would be much more convinient to have the metrics name like accuracy_imagenet and accuracy_cifar

[Vozf]

This could at least be possible with user manual metrics name override if the auto postfix feature could be disabled via flag.
Now _dataloader_idx_0 is a mandatory postfix for multiple val datasets as far as I know

[tchaton]

Dear @Vozf,

It is in the feature roadmap :)
It shouldn't be too hard.
Would you like to give it a try and open a PR. We will help you if you get blocked.

Best,
T.C

[Vozf]

What should be the api for that?

[SkafteNicki]

hi @Vozf, I have open PR #6274 which should give you the option for custom naming.
sorry if you already started working with on this, I was trying to solve #6091 which is basically the same issue as this

[Vozf]

Hi, I've not really started as I've had no good api ideas. So no progress wasted. Thanks for the pr though