[Doc] - Guide for sending logs to Kinesis (#4)

* AWS' SDK concerns for cached credentials should not be handled by the logger as they're handled by the aws sdk

* [Doc] - added a guide for configuring AwsKinesisLogger backend.

* [Doc] - Updated README based on reviews

Author: ddragosd

README.md

@@ -94,7 +94,108 @@ Backend systems
 
 AWS Kinesis backend
 ------------------
-Sends logs to AWS Kinesis.
+Backend implementation for sending logs to AWS Kinesis. This guide walks you through a few steps for configuring it.
+ This is not the only setup but it's performant and it's a great way to start with.
+
+In NGINX conf define the following variables:
+
+ * `aws_region` - the AWS region where the kinesis stream is created
+ * `kinesis_stream_name` - the name of the kinesis stream
+
+Make sure to define 2 shared dictionaries:
+
+* `lua_shared_dict stats_kinesis 16m;` - dictionary used to buffer the logs in memory 
+* `lua_shared_dict aws_credentials 1m;` - dictionary used to cache any IAM and STS credentials 
+
+Then in the `log_by_lua` configure the logger to send the information:
+```lua
+local cjson = require "cjson"
+local logger_factory = require "api-gateway.logger.factory"
+
+local function get_logger_configuration()
+    local logger_module = "api-gateway.logger.BufferedAsyncLogger"
+    local logger_opts = {
+        flush_length = 500,          -- http://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecords.html - 500 is max
+        flush_interval = 5,          -- interval in seconds to flush regardless if the buffer is full or not
+        flush_concurrency = 16,      -- max parallel threads used for sending logs
+        flush_throughput = 10000,     -- max logs / SECOND that can be sent to the Kinesis backend
+        sharedDict = "stats_kinesis", -- dict for caching the logs
+        backend = "api-gateway.logger.backend.AwsKinesisLogger",
+        backend_opts = {
+            aws_region = ngx.var.aws_region or "us-east-1",
+            kinesis_stream_name = ngx.var.kinesis_stream_name or "api-gateway-stream",
+            aws_credentials = {
+                provider = "api-gateway.aws.AWSIAMCredentials",
+                shared_cache_dict = "aws_credentials"  -- dict for caching STS and IAM credentials
+            }
+        },
+        callback = function(status)
+            -- capture or log information about each flush
+            -- status.logs_sent    - how many logs have been flushed successfully
+            -- status.logs_failed  - how many logs failed to be sent
+            -- status.backend_response_code - HTTP Status code returned by Kinesis
+            -- status.threads_running - how many parallel threads are active
+            -- status.threads_pending - how many threads are waiting to be executed
+        end
+    }
+    return logger_module, logger_opts
+end    
+
+local function get_logger(name)
+    -- try to reuse an existing logger instance for each worker process
+    if (logger_factory:hasLogger(name)) then
+        return logger_factory:getLogger(name)
+    end
+    
+    -- create a new logger instance
+    local logger_module , logger_opts = get_logger_configuration()
+    return logger_factory:getLogger(name, logger_module, logger_opts)
+end
+    
+local kinesis_logger = get_logger("kinesis-logger")
+
+local partition_key = ngx.utctime() .."-".. math.random(ngx.now() * 1000)
+local kinesis_data = {}
+
+-- add any information you want to capture
+kinesis_data["http_referer"] = ngx.var.http_referer
+kinesis_data["user_agent"] = ngx.var.http_user_agent
+kinesis_data["hostname"] = ngx.var.hostname
+kinesis_data["http_host"] = ngx.var.host
+
+-- at the end log the message
+kinesis_logger:logMetrics( partition_key, cjson.encode(kinesis_data))
+```
+
+If you want to use STS Credentials instead of IAM Credentials with the Kinesis Logger then configure the `backend_opts.aws_credentials` as follows:
+```lua
+aws_credentials = {
+    provider = "api-gateway.aws.AWSSTSCredentials",
+    role_ARN = "arn:aws:iam::" .. ngx.var.kinesis_aws_account .. ":role/" .. ngx.var.kinesis_iam_role,
+    role_session_name = "kinesis-logger-session",
+    shared_cache_dict = "aws_credentials"  -- dict for caching STS and IAM credentials
+}
+```
+
+Make sure to also configure the NGINX variables:
+
+* `kinesis_aws_account` - the AWS Account where the kinesis stream is configured
+* `kinesis_iam_role` - the role to be assumed in order to send the logs to kinesis
+
+>INFO: If you send the logs into the same account where NGINX runs you don't need to configure any STS Credentials, but you can use IAM Credentials. 
+
+For more information about `AWSSTSCredentials` configuration see [the documentation](https://github.com/adobe-apiplatform/api-gateway-aws#sts-credentials).
+
+If you can't use IAM Credentials nor STS Credentials, then you can still send logs to Kinesis by configuring AWS with static `access_key` and `secret`.
+ These are not as secure as IAM/STS but are working for non-AWS deployments:
+
+```lua
+aws_credentials = {
+    provider = "api-gateway.aws.AWSBasicCredentials",
+    access_key = "replace-me",
+    secret_key = "replace-me"
+}
+```
 
 HttpLogger backend
 ------------------