Exception in autograd profiler total_average() calculation

[dkossnick-figma]

(Directed to post here from https://discuss.pytorch.org/t/autograd-profiler-total-average-calculation-exception/56214.)

I had written a layer on top of the autograd profiler to collect more custom ranges of times, and with the upgrade to Pytorch 1.2 my usage is no longer working.

When a designated timing range begins, I have the following snippet (among a longer block in my codebase):

records = torch.autograd._disable_profiler()
events_average = EventList(parse_cpu_trace(records)).total_average()
cpu_time = events_average.cpu_time_total / 1000
cuda_time = events_average.cuda_time_total / 1000

When it runs, the .total_average() call in pytorch, at this line (total_stat += evt) crashes with the Exception: TypeError: unsupported operand type(s) for +=: 'FunctionEventAvg' and 'FunctionEvent'. I noticed FunctionEventAvg doesn’t have a add override, but used to. Seems like it changed with this commit.

I modified the source to change += to .add and it seems to work now. Should FunctionEventAvg have an __iadd__ method like it used to? Or should the += be changed to .add? Am I using it wrong?

Thanks!

cc @ezyang @ssnl @albanD @zou3519 @gqchen

[ezyang]

Unfortunately Alexander Sidorov doesn't work on PyTorch anymore. I think there is not a good reason to not have the __iadd__ overload, and would be happy to review a patch that added it back (implemented by calling add, I suppose.)

[dkossnick-figma]

PR opened at #27498. Thanks!

[albanD]

Associated PR has been merged.