Skip to content
This repository was archived by the owner on Nov 1, 2024. It is now read-only.
This repository was archived by the owner on Nov 1, 2024. It is now read-only.

[AnyPrecision optimizer] - needs unit tests #57

@lessw2020

Description

@lessw2020

This is just an issue to track that we do not have unit tests added yet for the BF16 optimizer.

Main tests would be:
a - verify AdamW and BF16 set to FP32 dtypes produce matching results. (BF16 degrades to AdamW w/o Kahan and dtypes to FP32).
b - simple checks for confirming internal dtypes match expected (i.e. set variance to BF16 but momentum to FP32, verify both are in proper dtype).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions