CelebA download is broken

[pmeier]

The download of all CelebA files except identity_CelebA.txt is broken. For example, the URL to download img_align_celeba.zip resolves to https://drive.google.com/uc?id=0B7EVK8r0v71pZjFTYXZWM3FlRnM&export=download. This link is publicly accessible, but you have to be logged into Google. Otherwise you'll see a 404 page.

I'll have a look if it is possible to get a general download link from the ID.

cc @pmeier @YosuaMichael

[pmeier]

It seems the author has intentionally restricted the visibility for these files for some reason. If that is true, I think there is no way for us to provide an automatic download.

[pmeier]

It seems something on the GDrive side has changed. The link above now gives a 403 (forbidden) instead a 404 page when you are logged. When you are not logged in you will be prompted to do so and afterwards see the same 403 page.

Exporting the link manually gives https://drive.google.com/file/d/0B7EVK8r0v71pZjFTYXZWM3FlRnM/view. While logged in you get transferred to the download page. If you are not logged in, you get the login prompt but land on page stating that access need to be requested.

For other datasets, e.g. Caltech101, both link variants are equivalent:

• https://drive.google.com/uc?id=137RyRjvTBkBiIfeYBNZBtViDHQ6_Ewsp&export=download
• https://drive.google.com/file/d/137RyRjvTBkBiIfeYBNZBtViDHQ6_Ewsp/view

We prefer the first, because it has less redirects and checks.

In any case it seems that Celeba is no longer automatically downloadable unless you are logged in. Thus, I propose we disable the download functionality preferably before the upcoming release. @NicolasHug Thoughts?

[NicolasHug]

Thanks for reminding me and for your investigations @pmeier !

What do you mean by "being logged in"? Is it logged in from a browser?

[pmeier]

What do you mean by "being logged in"? Is it logged in from a browser?

Yes. In particular, we need the session cookies. As possible way is to ask users to export the cookies to a file from the browser that we can read in. But this far from an "automated approach".

AFAIK, there is no way to login from the command line or through env vars or the like.

[NicolasHug]

Thanks for the deets. I agree we should deactivate it.

[pmeier]

Closed in #6052.

[NicolasHug]

Should we keep it open? Ultimately we'll want to put back the download feature, if the Gdrive becomes available again?

[datumbox]

That's reasonable. Should we also ping the owner (send an email or something) to let them know and ask them if they can open-up or rehost the dataset?

[pmeier]

Ultimately we'll want to put back the download feature, if the Gdrive becomes available again?

I don't think it will come back. This is not a limitation by GDrive, but as explained in #5705 (comment) a conscious decision by the author to limit access. I've contacted them twice and asked to revert it, but got no response.

If we want to keep it open, we should have some kind scheduled test or the like if the download is publicly accessible again. Otherwise we'll just forget about this and will have a stale issue. At least I will forget to regularly check the dataset if the author changed permissions.

[datumbox]

Thanks for clarifying Philip. Let me close again then.