Use Arrow Schema instead of custom schema type.

[trueleo]

Description

Replace custom schema type with arrow schema wrapped in Option. If metadata was able to load a schema then it will be of Some variant, in case there is no schema ( new stream ) then the None variant will be used. This reduced number of times serialization to schema happens and this schema will be directly passed on to future event processing and queries.

---

This PR has:

• been tested to ensure log ingestion and log query works.
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added documentation for new or modified features or behaviors.