UpsampleNetwork #724

jimchen90 · 2020-06-17T13:46:29Z

This Upsampling block is part of WaveRNN model. Now the test is to validate the output dimensions of this block. Other tests will be added after other blocks are combined.
Related to #446

Stack:

~~Add MelResNet Block #705 #751~~
~~Add Upsampling Block #724~~
~~Add WaveRNN Model #735~~
Add example pipeline with WaveRNN #749

codecov · 2020-06-17T13:56:19Z

Codecov Report

Merging #724 into master will increase coverage by 0.13%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #724      +/-   ##
==========================================
+ Coverage   89.21%   89.35%   +0.13%     
==========================================
  Files          32       32              
  Lines        2513     2546      +33     
==========================================
+ Hits         2242     2275      +33     
  Misses        271      271

Impacted Files	Coverage Δ
torchaudio/models/_wavernn.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 878d3da...44bff04. Read the comment docs.

vincentqb · 2020-06-17T19:24:52Z

torchaudio/models/_wavernn.py

+            x: the input sequence to the _Stretch2d layer (required).
+
+        Shape:
+            - x: :math:`(N, C, S, T)`.


nit: remove period at end

vincentqb · 2020-06-17T19:24:56Z

torchaudio/models/_wavernn.py

+
+        Shape:
+            - x: :math:`(N, C, S, T)`.
+            - output: :math:`(N, C, S * y_scale, T * x_scale)`.


nit: remove period at end

torchaudio/models/_wavernn.py

vincentqb · 2020-06-17T19:27:21Z

torchaudio/models/_wavernn.py

+        T is the length of input sequence.
+        """
+
+        n, c, s, t = x.size()


Expecting a four tuple is a little rigid, isn't it?

e.g. some functions support ... to mean an arbitrary number of dimensions, see functionals in torchaudio.

I have updated it.

vincentqb · 2020-06-17T19:43:37Z

torchaudio/models/_wavernn.py

+            x: the input sequence to the _UpsampleNetwork layer (required).
+
+        Shape:
+            - x: :math:`(N, S, T)`.


See notation in readme for output of spectrogram

Spectrogram: (channel, time) -> (channel, freq, time)

Variable names have been updated.

vincentqb · 2020-06-17T19:45:19Z

torchaudio/models/_wavernn.py

+
+        Shape:
+            - x: :math:`(N, S, T)`.
+            - output: :math:`(N, (T - 2 * pad) * Total_Scale, S)`, `(N, (T - 2 * pad) * total_scale, P)`.


nit: lower/upper case in total_scale and Total_Scale. I like the first better :)

Yes, total_scale looks better. Fixed.

vincentqb · 2020-06-17T19:47:20Z

torchaudio/models/_wavernn.py

+            - x: :math:`(N, S, T)`.
+            - output: :math:`(N, (T - 2 * pad) * Total_Scale, S)`, `(N, (T - 2 * pad) * total_scale, P)`.
+        where N is the batch size, S is the number of input sequence, T is the length of input sequence.
+        P is the number of output sequence. Total_Scale is the product of all elements in upsample_scales.


nit: P = output_dims ? just do that, or specify that P = output_dims.

This name has been updated. I use n_output here to match other places. No single letter is used in docstring now.

vincentqb · 2020-06-17T19:51:05Z

torchaudio/models/_wavernn.py

+        resnet_output = self.resnet_stretch(resnet_output)
+        resnet_output = resnet_output.squeeze(1)
+
+        upsampling_output = self.upsample_layers(x.unsqueeze(1))


nit: add a line x = x.unsqueeze(1)

This line has been added.

torchaudio/models/_wavernn.py

test/test_models.py

vincentqb · 2020-06-25T16:14:32Z

Is there a doc available? Can you attach the link?

EDIT: internal

vincentqb · 2020-06-25T16:32:23Z

torchaudio/models/_wavernn.py

+
+        total_scale = 1
+        for upsample_scale in upsample_scales:
+            total_scale *= upsample_scale


Please add an assert or error message checking that total_scale == hop_length, and document this requirement (e.g. "product of upsample_scale must equal hop_length") in docstring.

Because hop_length is a variable only in WaveRNN. I added an error message (line) and document this requirement (line) of WaveRNN #735 .

torchaudio/models/_wavernn.py

test/test_models.py

vincentqb

LGTM :)

jimchen90 requested a review from vincentqb June 17, 2020 13:46

jimchen90 marked this pull request as draft June 17, 2020 13:49

jimchen90 mentioned this pull request Jun 17, 2020

Naming conventions in torchaudio models #721

Closed

vincentqb reviewed Jun 17, 2020

View reviewed changes

jimchen90 force-pushed the upsampling branch 2 times, most recently from f4b4c76 to c98e289 Compare June 18, 2020 19:36

vincentqb reviewed Jun 18, 2020

View reviewed changes

torchaudio/models/_wavernn.py Outdated Show resolved Hide resolved

jimchen90 force-pushed the upsampling branch from ebf7afb to a211c77 Compare June 21, 2020 16:10

This was referenced Jun 22, 2020

Add WaveRNN Model #735

Merged

Add wavernn example pipeline #749

Merged

vincentqb reviewed Jun 25, 2020

View reviewed changes

test/test_models.py Show resolved Hide resolved

vincentqb reviewed Jun 25, 2020

View reviewed changes

test/test_models.py Outdated Show resolved Hide resolved

vincentqb reviewed Jun 25, 2020

View reviewed changes

jimchen90 mentioned this pull request Jun 25, 2020

Add MelResNet Block #705

Merged

vincentqb reviewed Jun 25, 2020

View reviewed changes

torchaudio/models/_wavernn.py Outdated Show resolved Hide resolved

jimchen90 force-pushed the upsampling branch from bb93759 to 54d8c1e Compare June 26, 2020 13:07

jimchen90 mentioned this pull request Jun 26, 2020

Update MelResNet #751

Merged

jimchen90 force-pushed the upsampling branch from 82a918b to c54c772 Compare June 29, 2020 13:40

jimchen90 marked this pull request as ready for review June 29, 2020 15:42

jimchen90 force-pushed the upsampling branch from 0a90c4f to a3f8f82 Compare June 29, 2020 22:10

Ji Chen added 5 commits June 29, 2020 19:01

upsamplenetwork

1ddd3bb

update name

62cc7d5

update name and docstring

1310586

update format

5289545

rebase

a8d1450

jimchen90 force-pushed the upsampling branch from f1205cd to a8d1450 Compare June 30, 2020 13:08

vincentqb reviewed Jun 30, 2020

View reviewed changes

torchaudio/models/_wavernn.py Outdated Show resolved Hide resolved

Ji Chen added 2 commits June 30, 2020 08:42

update docstring

0d56fb2

update docstring

b31fbb2

vincentqb reviewed Jun 30, 2020

View reviewed changes

torchaudio/models/_wavernn.py Outdated Show resolved Hide resolved

vincentqb reviewed Jun 30, 2020

View reviewed changes

torchaudio/models/_wavernn.py Show resolved Hide resolved

vincentqb reviewed Jun 30, 2020

View reviewed changes

torchaudio/models/_wavernn.py Outdated Show resolved Hide resolved

vincentqb reviewed Jul 1, 2020

View reviewed changes

torchaudio/models/_wavernn.py Show resolved Hide resolved

vincentqb reviewed Jul 1, 2020

View reviewed changes

test/test_models.py Show resolved Hide resolved

remove transpose and update docstring

44bff04

vincentqb approved these changes Jul 1, 2020

View reviewed changes

jimchen90 merged commit 6b15905 into pytorch:master Jul 1, 2020

This was referenced Jul 20, 2020

Fix output type of upsampling #801

Merged

Update form of default value in docstring #802

Merged

Remove underscore of wavernn model #810

Merged

UpsampleNetwork #724

UpsampleNetwork #724

Uh oh!

Conversation

jimchen90 commented Jun 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jun 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vincentqb Jun 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vincentqb commented Jun 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentqb Jun 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jimchen90 Jun 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vincentqb left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jimchen90 commented Jun 17, 2020 •

edited

Loading

codecov bot commented Jun 17, 2020 •

edited

Loading

vincentqb Jun 17, 2020 •

edited

Loading

vincentqb commented Jun 25, 2020 •

edited

Loading

vincentqb Jun 25, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading