Skip to content

Repeated HVPs With Sub-Sampling #171

@RS-Coop

Description

@RS-Coop

This question is not necessarily specific to this package, but it seemed like the best place to ask given your exisiting resources for computing HVPs. Apologies if I should be directing this elsewhere, but thanks in advance for the help!

My goal is as follows: given a minibatch of inputs I want to compute the gradients of a function with respect to some parameters at these inputs, and I want to build an HVP operator from a sub-sample of the minibatch inputs -- which I can use multiple times. Ideally I would like to compute the gradients first, and then use a sub-sample of these gradients to build the HVP operator.

It seems that existing approaches (like what I have seen here) recompute the gradients each time the HVP operator is applied. It is also not clear how to accomplish the sub-sampling.

It seems what I am trying to do should be possible, but I am just getting a little lost in the implementation details.

Thanks again!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions