Transformer tutorial error

[dhpollack]

https://github.com/pytorch/tutorials/blob/master/beginner_source/transformer_tutorial.py#L89

Since you are using the CrossEntropyLoss, you shouldn't do the log_softmax in the above line. You can either switch to the NLLLoss or remove the log_softmax to fix this.

[zhangguanheng66]

@dhpollack Thanks for bringing us the issue. Could you submit a PR and fix it? You could assign @zhangguanheng66 and @SethHWeidman for reviewing.

[dhpollack]

@zhangguanheng66 I think that I'll have time for a PR. Do you mind if I change other things? Personally, I found the batching a bit awkward, why not use pytorch's built in Dataset and DataLoader? Also, you could keep the targets in their original dimensions and transpose the outputs and use the multidimensional functionality of the loss function (easier to see in code then explained in words)... there were a couple small things like that which I saw but didn't think were worth github issues, but if I were writing the tutorial, I would have changed.

[zhangguanheng66]

Yes. This part is very true. The current wiki-text dataset is not compatible with DataLoader. My plan is to fix the wikitext dataset in pytorch/text first and and we could fix the tutorial. Again, the wikitext dataset is based on a wired pattern and I plan to fix it first.

It would be great if you could submit a quick fix for the CrossEntropyLoss part and I will let you know once the wikitext dataset is updated.

[dhpollack]

@zhangguanheng66 alright, here's the quick fix. I don't have the proper rights to assign you as a reviewer.

[soumith]

fixed via #695