Add encryption and decryption support to the SDK

[Lance-Drane]

We want to have a mechanism for users to easily be able to encyrpt or decrypt data sent over INTERSECT communications channels.

API Requirements

The SDK needs to be able to support multiple encryption schemes (we currently only need to support the AES/RSA scheme, but we should not assert that this is the only encryption scheme we must support), and should also be able to handle the "no encryption" (default) workflow.

There needs to be an endpoint to get the public key of a Service. This endpoint should be easily predictable by Clients and should be automatically configured by the SDK.

Services should be able to utilize a declarative format on their @intersect_message and @intersect_event decorators regarding required/supported encryption schemes, and should otherwise not have to explicitly handle the implementation details of encrypting/decrypting the data.

There should be a mechanism for Services to load in their own public/private keys, and for Clients to load in the private keys of other Services.

It should be easy for Clients to specify that they want to get the public key of a specific Service, utilizing the existing mechanism Clients already use.

Implementation suggestions

• Create a core Capability, automatically injected into every Service, to handle encryption/decryption endpoints
• Include message encryption mechanism in the message header

Supplementary issues - REQUIRED

• Rework messaging handling to directly use protocol headers, support multiple content-types #8 - we do NOT want to encrypt the metadata in the headers (which will also include the encryption scheme!), ONLY the actual payload. The representation of metadata must be adjusted to the data format specified by this issue in order for encryption/decryption to work

• Create a "Default INTERSECT Capability" which handles universal INTERSECT functions (i.e. encryption/decryption) #29 - specifying a single "core capability" for INTERSECT-SDK services helps standardize the API for Clients and makes configuration for Services far simpler

Supplementary issues - stretch

• handle inputs/outputs which are too large to exist in memory #28 - we need to figure out how we should handle encrypted/decrypted data which is too large to exist in memory at once. This does not necessarily have to be solved in the first cut of implementing this issue, but we need to make sure an encryption scheme exists which would work with buffered I/O .

An example of the "buffered I/O" workflow (note that this all happens by the SDK):

1. Recipient of message receives large encrypted data too large to read into memory (so it is written to File A (which is temporary))
2. We do a buffered read of the encrypted data file A, unencrypt chunks, and do a buffered write of the unencrypted data into File B
3. Domain scientist code can now access unencrypted File B, they will be responsible for the buffered read of this data as they will be handling domain logic during each read and will have to specify how they even want to buffer it into memory (most often this is done by newlines i.e. CSV, TSV, but not necessarily).