Improves monitoring Neo4j standalone container docs

modules/ROOT/pages/addition/docker/container.adoc

@@ -1,56 +1,105 @@
-= Monitor instance in standalone Docker container   
+= Monitor a Neo4j instance in standalone Docker container
 
-Running additional processes that are not tied to the lifecycle of the container process is not a best practice for containerized applications.
-Nevertheless, it is possible to transparently monitor a containerized Neo4j instance in following ways:
+A Neo4j instance running in a Docker container can be monitored by a NOM agent in two ways:
+
+- The NOM agent can be run as an additional (daemon) process in the Neo4j container
+- The NOM agent can be run on the Docker host to externally monitor the Neo4j container
 
 [NOTE]
 ====
-Both methods below use a bundled NOM agent inside the Neo4j image instead of downloaded agent packages.
-Replacing `products/neo4j-ops-manager-agent-*-linux-amd64.tar.gz` with path to NOM agent package downloaded should give the same results.
+Running additional processes that are not tied to the lifecycle of the container process is not a best practice for containerized applications.
 ====
 
-. Monitoring a Neo4j instance Running inside a Docker container:
+== In-container Monitoring
+The following steps describe running a NOM agent in Neo4j container as an additional (daemon) process to monitor the instance:
+
+* Run the Neo4j docker container.
+    ** For more information, see link:https://neo4j.com/docs/operations-manual/current/docker/[Operations Manual -> Docker].
+    ** A custom entrypoint script can be mounted to the Neo4j container to run the NOM agent at DBMS startup which is described <<entrypoint, further on>>.
+
+* Install the NOM agent to make the `agent` command available to run.
+    ** Bundled
+        *** NOM agent binaries are bundled in the Neo4j package and reside in the `products` folder of the Neo4j installation directory. 
+        *** It can be extracted from the package and installed into a system binaries directory for access to the `agent` command.
 +
-[source, shell, role=noheader]
+[source, shell]
 ----
-docker exec -dit <neo4j container name> /bin/sh 'tar -xzf products/neo4j-ops-manager-agent-*-linux-amd64.tar.gz --strip-components 1 \
-&& mv bin/agent /bin/agent \
-&& CONFIG_AGENT_NAME="test_local" \
-CONFIG_AGENT_DESCRIPTION="local self-registered agent" \
-CONFIG_SERVER_GRPC_ADDRESS="server:9090" \
-CONFIG_SERVER_HTTP_ADDRESS="https://server:8080" \
-CONFIG_LOG_LEVEL="debug" \
-CONFIG_INSTANCE_1_NAME="test_local" \
-CONFIG_INSTANCE_1_BOLT_URI="bolt://localhost:7687" \
-CONFIG_INSTANCE_1_BOLT_USERNAME="neo4j" \
-CONFIG_INSTANCE_1_BOLT_PASSWORD="passw0rd" \
-CONFIG_INSTANCE_1_QUERY_LOG_PORT="9500" \
-CONFIG_INSTANCE_1_LOG_CONFIG_PATH="/var/lib/neo4j/conf/server-logs.xml" \
-CONFIG_INSTANCE_1_QUERY_LOG_MIN_DURATION="100" \
-NEO4J_ACCEPT_LICENSE_AGREEMENT="yes" \
-NEO4J_AUTH=neo4j/passw0rd \
-NEO4J_EDITION="enterprise" \
-NEO4J_server_metrics_prometheus_enabled="true" \
-NEO4J_server_metrics_prometheus_endpoint="localhost:2004" \
-NEO4J_server_metrics_filter="*" \
-agent console -s'
+tar -xzf $NEO4J_HOME/products/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz --strip-components 1 && mv bin/agent-<NOM_VERSION>-linux-amd64 /bin/agent
+----
+
+    ** Mounted
+        *** NOM agent binaries can be mounted into the container as a volume mount and installed into a system binaries directory for access to `agent` command.
++
+[source, shell]
+----
+docker run -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/agent/binary>:/agent/bin neo4j/neo4j:latest --name <neo4j container name>
+----
++
+Wait for the container to be ready and then run :
++
+[source, shell]
+----
+docker exec -it <neo4j container name> -- /bin/sh -c "mv /agent/bin/agent-*-linux-amd64 /bin/agent"
+----
+
+    ** Downloaded
+        *** NOM agent package can be directly downloaded on to the Neo4j container and then extracted and installed.
++
+[source, shell]
+----
+wget -O /tmp/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz https://dist.neo4j.org/ops-manager/<NOM_VERSION>/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz
+tar -xzf /tmp/neo4j-ops-manager-agent-*-linux-amd64.tar.gz --strip-components 1 && mv bin/agent-*-linux-amd64 /bin/agent
 ----
 
-. Start a Neo4j instance Docker container with NOM agent using a custom entry point:
+* Run the NOM agent, which involves executing the agent binary with appropriate agent configuration.
+    ** Agent configuration can be provided as environment variables set during docker `exec` command or as part of container startup command.
+    ** Run agent in console mode
 +
+[source, shell]
+----
+docker exec -dit <neo4j container name> -- /bin/sh 'CONFIG_AGENT_NAME="test_local" \
+    CONFIG_AGENT_DESCRIPTION="local self-registered agent" \
+    CONFIG_SERVER_GRPC_ADDRESS="server:9090" \
+    CONFIG_SERVER_HTTP_ADDRESS="https://server:8080" \
+    CONFIG_TOKEN_URL=https://server:8080/api/login \
+    CONFIG_TOKEN_CLIENT_ID=rj34hg6456ghrhjthet45hg6 \
+    CONFIG_TOKEN_CLIENT_SECRET=5h346jhrjkthkjthjerhtkrt \
+    CONFIG_TLS_TRUSTED_CERTS=/path/to/trusted/certs/pem/file \
+    CONFIG_LOG_FILE=/path/to/nom-agent/log.txt \
+    CONFIG_LOG_LEVEL="debug" \
+    CONFIG_INSTANCE_1_NAME="test_local" \
+    CONFIG_INSTANCE_1_BOLT_URI="bolt://localhost:7687" \
+    CONFIG_INSTANCE_1_BOLT_USERNAME="neo4j" \
+    CONFIG_INSTANCE_1_BOLT_PASSWORD="passw0rd" \
+    CONFIG_INSTANCE_1_QUERY_LOG_PORT="9500" \
+    CONFIG_INSTANCE_1_LOG_CONFIG_PATH="/var/lib/neo4j/conf/server-logs.xml" \
+    CONFIG_INSTANCE_1_QUERY_LOG_MIN_DURATION="100" \
+    agent console -s'
+----
+
 [[entrypoint]]
+== Start a Neo4j container with NOM agent using a custom entry point
+
+Start the Neo4j container by running the entrypoint script `/startup/docker-entrypoint.sh` with the command `neo4j`. 
+To run the NOM agent on the container startup, this entrypoint needs to be overridden with a custom entrypoint script as shown:
+
+* In console mode:
++
 .entrypoint.sh
 [source, shell]
 ----
 #!/bin/bash -eu
 
 /startup/docker-entrypoint.sh neo4j &
-tar -xzf products/neo4j-ops-manager-agent-*-linux-amd64.tar.gz --strip-components 1 \
-&& mv bin/agent /bin/agent \
-&& CONFIG_AGENT_NAME="test_local" \
+CONFIG_AGENT_NAME="test_local" \
 CONFIG_AGENT_DESCRIPTION="local self-registered agent" \
 CONFIG_SERVER_GRPC_ADDRESS="server:9090" \
 CONFIG_SERVER_HTTP_ADDRESS="https://server:8080" \
+CONFIG_TOKEN_URL=https://server:8080/api/login \
+CONFIG_TOKEN_CLIENT_ID=rj34hg6456ghrhjthet45hg6 \
+CONFIG_TOKEN_CLIENT_SECRET=5h346jhrjkthkjthjerhtkrt \
+CONFIG_TLS_TRUSTED_CERTS=/path/to/trusted/certs/pem/file \
+CONFIG_LOG_FILE=/path/to/nom-agent/log.txt \
 CONFIG_LOG_LEVEL="debug" \
 CONFIG_INSTANCE_1_NAME="test_local" \
 CONFIG_INSTANCE_1_BOLT_URI="bolt://localhost:7687" \
@@ -59,16 +108,63 @@ CONFIG_INSTANCE_1_BOLT_PASSWORD="passw0rd" \
 CONFIG_INSTANCE_1_QUERY_LOG_PORT="9500" \
 CONFIG_INSTANCE_1_LOG_CONFIG_PATH="/var/lib/neo4j/conf/server-logs.xml" \
 CONFIG_INSTANCE_1_QUERY_LOG_MIN_DURATION="100" \
-NEO4J_ACCEPT_LICENSE_AGREEMENT="yes" \
-NEO4J_AUTH=neo4j/passw0rd \
-NEO4J_EDITION="enterprise" \
-NEO4J_server_metrics_prometheus_enabled="true" \
-NEO4J_server_metrics_prometheus_endpoint="localhost:2004" \
-NEO4J_server_metrics_filter="*" \
 agent console -s
 ----
 +
 [source, shell, role=noheader]
 ----
  docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/custom/entrypoint>:/custom $NEO4J_IMAGE
 ----
+
+* In service mode:
++
+.entrypoint.sh
+[source, shell]
+----
+#!/bin/bash -eu
+
+/startup/docker-entrypoint.sh neo4j &
+agent service install
+cat > neo4j-ops-manager-agent.service <<EOF
+[Service]
+Environment="CONFIG_SERVER_GRPC_ADDRESS=<server grpc address>"
+Environment="CONFIG_SERVER_HTTP_ADDRESS=<server http address>"
+Environment="CONFIG_TOKEN_URL=<server http login url>"
+Environment="CONFIG_TOKEN_CLIENT_ID=<client id>"
+Environment="CONFIG_TOKEN_CLIENT_SECRET=<client secret>"
+Environment="CONFIG_TLS_TRUSTED_CERTS=</path/to/trusted/certs/pem/file>"
+Environment="CONFIG_LOG_FILE=</path/to/nom-agent/log.txt>"
+Environment="CONFIG_INSTANCE_1_NAME=<instance name>"
+Environment="CONFIG_INSTANCE_1_BOLT_URI=<bolt uri of the local instance>"
+Environment="CONFIG_INSTANCE_1_BOLT_USERNAME=<local instance user name>"
+Environment="CONFIG_INSTANCE_1_BOLT_PASSWORD=<local instance password>"
+Environment="CONFIG_INSTANCE_1_QUERY_LOG_PORT=<an available port>"
+Environment="CONFIG_INSTANCE_1_LOG_CONFIG_PATH=<path to server-logs.xml>"
+EOF
+systemctl start neo4j-ops-manager-agent.service
+----
++
+[source, shell, role=noheader]
+----
+ docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/custom/entrypoint>:/custom $NEO4J_IMAGE
+----
+
+== External monitoring
+
+A NOM agent can be run in either the console or service mode on the Docker host.
+It can be configured to have access to Neo4j container resources.
+Apply the following additional configurations to the Neo4j container run config to enable an external NOM agent to monitor the instance correctly:
+
+[source, shell, role=nocopy]
+----
+docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes \
+           -v <path/to/custom/entrypoint>:/custom \
+           -v </path/to/local/neo4j/home>:/var/lib/neo4j \ <1>
+           -p "8884:2004" \ <2>
+           -p "9500:9500" \ <3>
+       neo4j/neo4j:latest
+----
+
+<1> Neo4j home directory needs to be mounted back on the Docker host to enable access to agent.
+<2> Neo4j prometheus endpoint port (2004) needs to be exposed via port mapping.
+<3> Query log port (9500) needs to be mapped for log appender to forward query logs.