Implement Deformable Convolutions

[fmassa]

Discussion in pytorch/pytorch#15864

The operator should have both CPU and CUDA implementations.

[Ridhwanluthra]

I am familiar with the DCN implementation in mmdetection and I would be happy to include that in torchvision. Please let me know if no one else is working on it.

[fmassa]

@Ridhwanluthra no one else is currently working on it, so contributions are more than welcome!

Note that it should now ideally follow the approaches that we have in torchvision for custom operators, which allow them to be part of torchscript graph, see #1348 for an example

[pedrofreire]

@Ridhwanluthra
Are you working on it? I was considering having a look at it otherwise :)

[pedrofreire]

I am working on this :)

[zhiqwang]

Hi,

https://github.com/facebookresearch/detectron2/blob/b54486e7cee607a968c974de4e0e95c39ffc834e/detectron2/layers/csrc/deformable/deform_conv_cuda.cu#L20

I saw there have DCN implementation in detectron2, is there any plan to merge this implementation here?

[pedrofreire]

Cool, thanks for letting us know! Yeah, so it seems the detectron2 incorporated the code from mmdetection, which is the implementation I'm also basing myself on, so in some sense I'm already merging it! I'm also planning to add testing, a CPU version, and improve code readability a bit :)

[Ridhwanluthra]

@pedrofreire Thanks so much for picking this up.

[fmassa]

Fixed by @pedrofreire in #1586