Can't seem to get torchtext.experimental functions.

[ankitvad]

❓ Questions and Help

Description
I am on Ubuntu using pip in a venv.
Current Version:

Python: 3.6.9
CUDA: 10.2
torch==1.7.0
torchsummary==1.5.1
torchtext==0.8.0
torchvision==0.8.1

I tried updating my previous version of torchtext from 0.5.0 to 0.8.0 as I wanted to use the pretrained SP (sentencepiece) functions added to the torchtext.experimental.functional library.

Before the update I was able to load the experimental.dataset feature.
from torchtext.experimental.datasets import IMDB successfully on torch=1.5 and torchtext=0.5
However, transform had an issue: ModuleNotFoundError: No module named 'torchtext.experimental.transforms'.

So, I upgraded torchtext to 0.8. and got the following traceback:

>>> from torchtext.experimental.transforms import pretrained_sp_model
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/<USER>/Code/pytorch_lightning/DLGPU/lib/python3.6/site-packages/torchtext/__init__.py", line 40, in <module>
    _init_extension()
  File "/home/<USER>/Code/pytorch_lightning/DLGPU/lib/python3.6/site-packages/torchtext/__init__.py", line 36, in _init_extension
    torch.ops.load_library(ext_specs.origin)
  File "/home/<USER>/Code/pytorch_lightning/DLGPU/lib/python3.6/site-packages/torch/_ops.py", line 105, in load_library
    ctypes.CDLL(path)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/<DLGPU>/Code/pytorch_lightning/DLGPU/lib/python3.6/site-packages/torchtext/_torchtext.so: undefined symbol: _ZN3c1023_fastEqualsForContainerERKNS_6IValueES2_

So, I thought there was an issue with my old torch version with the new torchtext and hence I upgraded torch from 1.5.1 to 1.7:

pip install --upgrade torch
Collecting torch
  Downloading torch-1.7.0-cp36-cp36m-manylinux1_x86_64.whl (776.7 MB)
Installing collected packages: typing-extensions, torch
  Attempting uninstall: torch
    Found existing installation: torch 1.5.1
    Uninstalling torch-1.5.1:
      Successfully uninstalled torch-1.5.1

After this step, I obtained some inconsistency with allennlp library.

torchvision 0.4.2 requires torch==1.3.1, but you'll have torch 1.7.0 which is incompatible.
allennlp 1.0.0 requires torch<1.6.0,>=1.5.0, but you'll have torch 1.7.0 which is incompatible.

So, finally removed all torch and torch dependednt libraries and reinstalled them. Current version:

pytorch-lightning==1.0.5
pytorch-nlp==0.5.0
texar-pytorch==0.1.1
torch==1.7.0
torchsummary==1.5.1
torchtext==0.8.0
torchvision==0.8.1
allennlp==1.2.1

And, finally the issue that is persisting:

>>> from torchtext.experimental.functional import sequential_transforms, vocab_func, totensor
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torchtext.experimental'

Am I doing something wrong? Or, on the wrong version of any library?

PS: Mentioning @zhangguanheng66 JIC I can get some specific idea about the Sentencepiece function?

[zhangguanheng66]

Yup. That's expected. According to PyTorch feature classification changes, those building blocks and datasets in the experimental folder are nightly release, which go to the nightly release. Here is the instruction to install the nightly release:

pip install --pre torch torchtext -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html

[zhangguanheng66]

To you second question, since torchvision and torchtext have cpp extension, they are only compatible with a fixed version of torch. For the latest one, you will have

torch==1.7.0
torchtext==0.8.0
torchvision==0.8.1

[schlabrendorff]

Hi!

From my understanding the new API / experiments were not published with 8.0 yet, but will be in 8.1 (judging from the release/8.1 branch), as well as the deprecated functions not moved to legacy (still warning). Correct?

When do you plan to release 8.1?

(I don't want to start a new project with the soon to be retired functions, but would prefer not to compile torchtext myself)

[zhangguanheng66]

8.1 release will be a minor release by the end of this year to follow the compatibility with pytorch 1.7.1. Those experimental stuffs together with the retirement of the legacy code will happen early next year, hopefully Q1.

[schlabrendorff]

OK! Thank you!! Therefore to be future-proof the best way would be to develop with pytorch-nightly images and torchtext-0.9.0dev? Or do you anticipate still significant API changes? Or would you rather recommend to use a different library/framework, if one needs to develop a project now?

[zhangguanheng66]

I would choose the nightly release to get access to those building blocks.

[ankitvad]

Hi @zhangguanheng66 thanks for the replies. I've been trying to update the library using the URL and have been checking this issue: #609
and, I have been facing an issue for using the encode/decode feature.

I opened an issue here: mmcux/de-nds-translation#3

When I do:

SRC = Field(use_vocab = False, tokenize = sp_de.encode,
AttributeError: __torch__.torch.classes.torchtext.SentencePiece does not have a field with name 'encode'

where sp_de is obtained from load_sp_model and the model is trained using spm.SentencePieceTrainer.train

It seems that the encode/decode function is not present. I tried changing it to something else to check further code coverage and I get: sp_de.bos_id(), eos_id and pad_id error.

Am I on a wrong version still or something? How to get that tokenizer+numericalizer especially in Field-tokenize function?

[zhangguanheng66]

You should use EncodeAsIds. We didn't support all the SentencePiece APIs and here is a list of the supported APIs. And BTW, we will retire the Field class soon. You don't need it to build the data processing pipeline.

[ankitvad]

Oh! So what should a future compatible way look like to create the data processing pipeline?

I do have the TabularDataset and BucketIterator calls in the pipeline. Which would probably require a Field Value right? Especially if the data had different types and features all of which couldn’t be covered with sentencepiece!?

Any idea on this?

What if I use the use_vocab=True and in the tokenize function use something like: sentencepiece_tokenizer is that the same?

[zhangguanheng66]

We are working on several tutorials to show how to build the pipeline without the legacy code. For now, you can take a look at text/examples folder.

[bentrevett]

For what it's worth, and it's probably already mentioned somewhere in this thread, to use the experimental features you have to download PyTorch 1.7 - which does not work with Python 3.9, a fact that had me stumped for a while - and then install torchtext using the instructions in the README, namely:

git clone https://github.com/pytorch/text torchtext
cd torchtext
git submodule update --init --recursive
python setup.py clean install

This worked for me on Ubuntu 20.04.

[ankitvad]

I just went back to legacy code, for now, to makes sure certain features work. Will have to see how to move away from using Field class and the other classes depending on Field. Thanks!

[woshicqy]

Hi,

When I try to use the from torchtext.experimental.datasets import IMDB in my code, it didn't work and shows the feedback mentioned above. I used the torchtext == 0.8.1 Could you please give me some suggestions to fix this issue?