At the moment there is support for s3 connections. We would like to extend this to Azure data lake storage and Google cloud storage.
Modifications that have to happen:
flowfile_core/flowfile_core/flowfile/flow_data_engine/cloud_storage_reader.py: The parsing of configs is handled here
flowfile_core/flowfile_core/flowfile/flow_data_engine/flow_data_engine.py: Depending on if the current set-up handles also scenario's for adls and gcs there might be some refactoring needed to it.
- In the CI test template add Azurite and GCS emulator for testing purposes.