HEAD requests fail to https://dot.net/v1/dotnet-install.ps1

[jonathanpeppers]

We noticed HEAD requests now give a 403:

> Invoke-WebRequest -Uri https://dot.net/v1/dotnet-install.ps1 -Method HEAD
Invoke-WebRequest : The remote server returned an error: (403) Forbidden.
At line:1 char:1
+ Invoke-WebRequest -Uri https://dot.net/v1/dotnet-install.ps1 -Method  ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Our CI started failing, because we have some generic download code that does an initial HEAD request, for showing progress of downloads.

See: dotnet/android#8311

[YuliiaKovalova]

Hi @jonathanpeppers,

Is it still relevant? I can see the attached PR is already green.
I suspect you experienced some network-related issues.

[jonathanpeppers]

This powershell command still fails for me: Invoke-WebRequest -Uri https://dot.net/v1/dotnet-install.ps1 -Method HEAD

The PR linked above is an example of how we worked around it. We'd still be down otherwise, I think.

[YuliiaKovalova]

Hi @mairaw ,

Could you advise on it? I can repro the issue locally, but I am not sure why it returns 403 code on specifying -Method HEAD param.

[mairaw]

@danzhu54 this might be related to thew new WAF policies. Can you check?

[danzhu54]

Yes, I added HEAD requests to our blocked list but it makes sense to for this request to be whitelisted. Can you try again?

[jonathanpeppers]

The powershell is working now, thanks!

> Invoke-WebRequest -Uri https://dot.net/v1/dotnet-install.ps1 -Method HEAD


StatusCode        : 200
StatusDescription : OK
Content           : {}
RawContent        : HTTP/1.1 200 OK
                    Accept-Ranges: bytes
                    Content-Length: 70180
                    Cache-Control: public, max-age=300
                    Content-Type: application/octet-stream
                    ETag: "1d9dc5aaeafc324"
                    Last-Modified: Thu, 31 Aug 2023 22:30...
Headers           : {[Accept-Ranges, bytes], [Content-Length, 70180], [Cache-Control, public, max-age=300], [Content-Type, application/octet-stream]...}
RawContentLength  : 0