OpenSSL 3 | Providers | Support broader range of URI schemes for loading keys

[Maks027]

We've been using the OpenSSL::Engine to load keys from an HSM thorough a PKCS#11 interface (with the corresponding engine installed).

The Engine API was deprecated in OpenSSL 3 and there seems to be no alternatives for it at the moment using Provider API. The providers can only be loaded, but there seems to be no way to load keys using an uri (for ex. pkcs11 uri scheme)

In the migration guide, the proposed alternative is the usage of OSSL_STORE. As far as I can tell, the PKey interface only supports BIO inputs.
At the moment, I ended up writing a crude C extension that seems to do the job, but it's definitely not a production ready solution.

Are there any plans to bring back the functionality similar to Engines? Or maybe I've missed other available options?

[rhenium]

Could you tell us which engine or provider you are using?

In the migration guide, the proposed alternative is the usage of OSSL_STORE. As far as I can tell, the PKey interface only supports BIO inputs.

Correct, ruby/openssl doesn't provider a wrapper for the OSSL_STORE API yet.

[Maks027]

Hello,
We are trying to use the pkcs11-provider to interact with AWS CloudHSM.

For now, I've created a small separate gem that can load an RSA private key from OSSL_STORE (the functionality is limited by our use case)
The extracted EVP_PKEY is then converted to an instance of OpenSSL::PKey::RSA similar to this: https://github.com/ruby/openssl/blob/master/ext/openssl/ossl_engine.c#L375
The conversion seems like a bodge, but I haven't yet found any better options.

The implementation (error handling was omitted here):

VALUE ossl_provider_load_privkey(VALUE self, VALUE uri)
{
    EVP_PKEY *privkey = NULL;
    OSSL_STORE_INFO *info = NULL;
    OSSL_STORE_CTX *store;

    char *s_uri;

    s_uri = NIL_P(uri) ? NULL : StringValueCStr(uri);

    store = OSSL_STORE_open(s_uri, NULL, NULL, NULL, NULL);

    for (info = OSSL_STORE_load(store); info != NULL; info = OSSL_STORE_load(store)) {
        if (OSSL_STORE_INFO_get_type(info) == OSSL_STORE_INFO_PKEY) {
            privkey = OSSL_STORE_INFO_get1_PKEY(info);
        }

        OSSL_STORE_INFO_free(info);
        info = NULL;
    }

    OSSL_STORE_close(store);

    return ossl_pkey_new(privkey);
}

It seems to do the job, but there are quite a few limitations. For example, calling to_pem (or other methods that try to export a key) cause a segfault. It seems to be caused by the fact that some deprecated functions are still being used, like RSA_get0_key (from a surface level debugging)

Sign/verify operations seem to work fine (even for mTLS). Encrypt/decrypt also works, but there with some quirks.

I could try to implement this and open a PR, but I'm not really sure what would be the best way to integrate it. The simplest and safest way would be to create a separate module, similar to engines, but it kinda goes against the idea of providers. On the other hand, the STORE API can be used universally, but I guess this will require to a lot of significant changes.

[junaruga]

Just for your information: A Fedora Linux's OpenSSL RPM package owner is trying to drop the engine support because they said that these were not FIPS compatible and the API has been deprecated since OpenSSL 3.0. The owner Dmitry Belyavskiy is one of the maintainers in the OpenSSL project as far as I know. And importantly he said, "The engine functionality we are aware of (PKCS#11, TPM) is either covered by providers or will be covered soon.".

https://fedoraproject.org/wiki/Changes/OpensslNoEngine

We are going to build OpenSSL without engine support. Engines are not FIPS compatible and corresponding API is deprecated since OpenSSL 3.0. The engine functionality we are aware of (PKCS#11, TPM) is either covered by providers or will be covered soon.

Below is a discussion about the change proposal in the Fedora project.
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/SEH75XSFCVS44VKDYOP2ZECLTH4YETNC/