Skip to content

Metadata management flow #36

@nitisht

Description

@nitisht

Current STREAM_INFO map may have an issue with parallel writes to the map for the same stream.

  1. We need to implement a locking mechanism that allows exclusive writes to the map.
  2. Additionally Metadata map get and get methods should handle both local and S3. This way we don’t duplicate all the code to fallback on object store everywhere.
    a. Get Flow: Fetch metadata from local and fallback to object store if not found.
    b. Set Flow: Set to object store and then try setting it locally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions