Add Trusted Hosts Configuration

[JSanford42]

Several months ago, pub started throwing certificate errors. I have searched every way I can think of to get this issue resolved and have had no luck. I have discovered that it is an issue with the way the proxy is configured.

I would like the ability to specify trusted package sources in the pubspec.yaml file. The trusted hosts would not get stopped by certificate errors.

trusted_hosts:
  - pub.dartlang.org
  - pub.dev

The reason behind this is that I am on a network where the proxy behaves in an unusual manner. The proxy has its own certificate store. When it encounters a site for which it does not have the certificates, it uses an internal certificate chain for the connection between itself and my workstation. When it does have the certificates, the host certificates are "passed through" for the proxy to workstation connection.

This behavior has made it impossible for me to install or update packages. I have tried adding the certificates using the DART_VM_OPTIONS route with no success. I have verified that the certificates that I am adding are the same ones that the proxy has.

[jonasfj]

More context in dart-lang/sdk#45939

When it does have the certificates, the host certificates are "passed through" for the proxy to workstation connection.

Okay, it sounds as if the proxy allows TLS traffic through for some allow-listed domains, and that for the rest you the proxy terminates TLS and forwards traffic using a self-signed certificate.

In this case, the solution seems to be:

• Install self-signed certificate used the proxy on the system certificate store,
• Make Dart / pub load certificates from the system certificate store.

Am I understanding this wrong?

---

@JSanford42 Is it possible that you already have the self-signed certificate installed on the machine, but not a certificate store loaded by Dart SDK, see: dart-lang/sdk#45909

I'm guessing there will be a dev release available for testing next week -> https://dart.dev/tools/sdk/archive (2.14.0-85.0.dev as of writing doesn't look to have the possible fix)

[jonasfj]

To clarify: I'm not convinced we should add an option to ignore invalid HTTPS certificates.
If you really want that, I would suggest running a http <-> https proxy on localhost and using PUB_HOSTED_URL to direct dart pub to talk to the proxy.

Disabling HTTPS is dangerous, providing an easy options for doing so is likely undesirable.
Adding an option to remember and trust unknown certificate, in a trust-on-first-use kind of way, might be more viable :)

But if we could make installing into system certificate store work, that seems like a much better option.

[JSanford42]

Okay, it sounds as if the proxy allows TLS traffic through for some allow-listed domains, and that for the rest you the proxy terminates TLS and forwards traffic using a self-signed certificate.

That appears to be exactly what is happening. I am not privy to the network configuration but that is the behavior that I have seen with my testing.

In this case, the solution seems to be:

• Install self-signed certificate used the proxy on the system certificate store,
• Make Dart / pub load certificates from the system certificate store.

Am I understanding this wrong?

I have verified that the self-signed certificates used by the proxy are in the workstation certificate store but I am unable to determine if it is the system or machine store. I do not have administrator privileges so I can only see the certificates available to my user account.

I understand not wanting to ignore invalid HTTP certificates. I have seen so many posts in various places that have provided fruitless solutions to being behind a proxy that I am trying to get any changes made that will allow pub to work.

[JSanford42]

I'm guessing there will be a dev release available for testing next week -> https://dart.dev/tools/sdk/archive (2.14.0-85.0.dev as of writing doesn't look to have the possible fix)

I will pull down build 93 when it is available and test it out. I'll report my findings here.

[jonasfj]

I will pull down build 93 when it is available and test it out. I'll report my findings here.

Please do.. If we figure out a good solution for this we should document it here:
https://dart.dev/tools/pub/troubleshoot#pub-get-fails-from-behind-a-corporate-firewall

You're not the only one with peculiar network conditions.

[jonasfj]

@JSanford42
Can you tryout 2.14.0-90.0.dev from https://dart.dev/tools/sdk/archive#dev-channel

[JSanford42]

@jonasfj I pulled down build 90 this morning and it still does not work. Progress has been made though. It seems to be able to get the local issuer certificate now but it thinks it is expired. I went to https://pub.dartlang.org/api/packages/pedantic and checked all certificates in the browser and none of the ones in use are expired.

Previous Error

CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate

New Error

CERTIFICATE_VERIFY_FAILED: certificate has expired

I looked in the certificate store that I am able to see from the browser. I have several intermediate certificates in there that are expired but none of them are being used by the browser when I connect to the URL above.

[jonasfj]

@JSanford42, well, that's certainly progress.

I'm guessing we need to figure how to reproduce this. Can you create a custom HTTPS server on localhost using dart:io (or python) which uses a self-signed certificate that installed on the machine and works in the browser, but not in dart:io?

If we can do something like that, maybe we can ask some of the network people to look into it. Or maybe I can figure out how to build Dart SDK for windows and tweak: https://dart.googlesource.com/sdk//+/acf842c56f0c959588241d592dbcab18f1830322/runtime/bin/secure_socket_utils.h#22

---

Just guessing here.. maybe the certificate exists in multiple stores on the system, but one of them is expired, or maybe there is something else wrong. I think we need to reproduce on a clean Windows VM in the cloud to make much progress.

This could be certificate loading not working correctly, or it could be misconfiguration of the system.

[JSanford42]

I did some more digging now that all of the certificate stores are being used. I made a bare bones HTTP client for the pedantic package URL that prints the certificate information in the bad certificate callback. It shows that one of the intermediate certificates is expired.

I looked in the certificate store and found 3 with the same name (e.x. Signing CA 6). Two are expired and one is not.

Just saw your comment after this post. I think the certificate is loading correctly but the first one found is being used. I think, in this case, the first one found is expired.

[JSanford42]

I'm guessing we need to figure how to reproduce this. Can you create a custom HTTPS server on localhost using dart:io (or python) which uses a self-signed certificate that installed on the machine and works in the browser, but not in dart:io?

I created a new cert and added it to the current user certificate store as trusted. I created a small HTTP server in Dart and used the self signed cert and key.

As expected, I got the issuer certificate error when running under 2.12.4. I did not receive any certificate errors when running under 2.14.0-90.0.dev and saw the response text the server returns.

[jonasfj]

I think the certificate is loading correctly but the first one found is being used. I think, in this case, the first one found is expired.

Could the order in which they are inserted into the store be sufficient to reproduce this?
If so, that would be a great bug report for the Dart SDK.

I'm not sure if this is a problem in BoringSSL, how Dart does TLS, or maybe just because we're loading expired certificates when non-expired certificates are available.

/cc @aam

[aam]

When it comes to using Windows certificate stores dart runtime simply loads all certificates from current user and local machine certificate stores into boringssl(https://github.com/dart-lang/sdk/blob/master/runtime/bin/security_context_win.cc#L114) and relies on boringssl to do end-point certificate verification. Not sure why boringssl would be confused by presence of expired certificate if non-expired is present.
I would imagine though that end-point certificate specifies exact issuer certificate which effectively repeats the certificate verification process for the issuer certificate, all the way up the chain until it reaches a certificate that is present in trusted root store (in Windows store, in boringssl).

We also recently added an dart vm option --bypass_trusting_system_roots which bypasses use of Windows system certificate store altogether and relies on dart vm builtin list of certificates. 2.14.0-90.0.dev should have that change too.

[aam]

as @jonasfj pointed out bypassing system store might not be an option for you. In that case there is another command line option available: --root-certs-file=<filename> where filename is something that is given to boringssl via https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_load_verify_locations.html

See #1882 (comment)

[jonasfj]

Hmm, I was able to use --root-certs-file= as follows:

openssl s_client -showcerts -partial_chain -connect pub.dev:443 < /dev/null >> trusted-certificates.txt
openssl s_client -showcerts -partial_chain -connect storage.googleapis.com:443 < /dev/null > trusted-certificates.txt

dart --root-certs-file=trusted-certificates.txt pub upgrade --verbose

This is probably not the best of ideas, as I didn't isolate the root certificate I needed, I just dumped all the certificates into a trusted-certificates.txt file 🙈

---

But now that we have dart pub get, it's obvious that the old hack we had with DART_VM_OPTIONS no longer work (not that we really advertised it widely).

But perhaps it's time we highlight how to use --root-certs-file= on https://dart.dev/tools/pub/troubleshoot