Introduce an optional background dispatching strategy #75
Description
Currently the extension flushes all received data after having received the invocation end signal (from logs API or from the agents directly). Though this unblocks the response of the lambda request itself (response time of the lambda function), it still blocks the lambda function from receiving the next request until the extension has flushed all the data. This has a negative effect on the throughput of the function.
For lambda functions that have a steadily frequent load pattern the extension could delay sending the data to the APM server to the next lambda request and do the sending in parallel to the processing of that next request. This potentially would improve both the lambda function response time and its throughput.
The downside of this strategy is the risk that some APM data might be delayed or even get lost in case that a lambda function has an infrequent load pattern. Therefore, we cannot simply apply the above proposed strategy as the single available and the default one.
This issue is about introducing a config option for the dispatching strategy, that would support:
- flushing strategy: current dispatching strategy - this should also be default strategy if the config option is not specified explicitly
- background strategy: the strategy proposed above
This would allow users to choose between those strategies to optimize for their use case.
Further potential improvements for the future (post GA) are describe in #7.