Skip to content

Commit 2b349b6

Browse files
ravikiran-sulikeriAlexicaWrightalexnblidiazuin
authored
Improves monitoring Neo4j standalone container docs (#175)
* Improves monitoring Neo4j standalone container docs * minor edits * Update modules/ROOT/pages/addition/docker/container.adoc Co-authored-by: Jessica Wright <[email protected]> * Update modules/ROOT/pages/addition/docker/container.adoc Co-authored-by: Jessica Wright <[email protected]> * Apply suggestions from code review Co-authored-by: Jessica Wright <[email protected]> * correct the doc formmating * formatted shell commands * combined 3 shell blocks into one with callouts * fix agent env config scripts * Apply suggestions from code review Co-authored-by: Lidia Zuin <[email protected]> * Update modules/ROOT/pages/addition/docker/container.adoc Co-authored-by: Alexander Bouriakov <[email protected]> * Apply suggestions from code review Co-authored-by: Jessica Wright <[email protected]> Co-authored-by: Lidia Zuin <[email protected]> --------- Co-authored-by: Jessica Wright <[email protected]> Co-authored-by: Alexander Bouriakov <[email protected]> Co-authored-by: Lidia Zuin <[email protected]>
1 parent a5208a0 commit 2b349b6

File tree

1 file changed

+134
-38
lines changed

1 file changed

+134
-38
lines changed

‎modules/ROOT/pages/addition/docker/container.adoc‎

Lines changed: 134 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,105 @@
1-
= Monitor instance in standalone Docker container
1+
= Monitor a Neo4j instance in standalone Docker container
22

3-
Running additional processes that are not tied to the lifecycle of the container process is not a best practice for containerized applications.
4-
Nevertheless, it is possible to transparently monitor a containerized Neo4j instance in following ways:
3+
A Neo4j instance running in a Docker container can be monitored by a NOM agent in two ways:
4+
5+
- The NOM agent can be run as an additional (daemon) process in the Neo4j container
6+
- The NOM agent can be run on the Docker host to externally monitor the Neo4j container
57
68
[NOTE]
79
====
8-
Both methods below use a bundled NOM agent inside the Neo4j image instead of downloaded agent packages.
9-
Replacing `products/neo4j-ops-manager-agent-*-linux-amd64.tar.gz` with path to NOM agent package downloaded should give the same results.
10+
Running additional processes that are not tied to the lifecycle of the container process is not a best practice for containerized applications.
1011
====
1112

12-
. Monitoring a Neo4j instance Running inside a Docker container:
13+
== In-container Monitoring
14+
The following steps describe running a NOM agent in Neo4j container as an additional (daemon) process to monitor the instance:
15+
16+
* Run the Neo4j docker container.
17+
** For more information, see link:https://neo4j.com/docs/operations-manual/current/docker/[Operations Manual -> Docker].
18+
** A custom entrypoint script can be mounted to the Neo4j container to run the NOM agent at DBMS startup which is described <<entrypoint, further on>>.
19+
20+
* Install the NOM agent to make the `agent` command available to run.
21+
** Bundled
22+
*** NOM agent binaries are bundled in the Neo4j package and reside in the `products` folder of the Neo4j installation directory.
23+
*** It can be extracted from the package and installed into a system binaries directory for access to the `agent` command.
1324
+
14-
[source, shell, role=noheader]
25+
[source, shell]
1526
----
16-
docker exec -dit <neo4j container name> /bin/sh 'tar -xzf products/neo4j-ops-manager-agent-*-linux-amd64.tar.gz --strip-components 1 \
17-
&& mv bin/agent /bin/agent \
18-
&& CONFIG_AGENT_NAME="test_local" \
19-
CONFIG_AGENT_DESCRIPTION="local self-registered agent" \
20-
CONFIG_SERVER_GRPC_ADDRESS="server:9090" \
21-
CONFIG_SERVER_HTTP_ADDRESS="https://server:8080" \
22-
CONFIG_LOG_LEVEL="debug" \
23-
CONFIG_INSTANCE_1_NAME="test_local" \
24-
CONFIG_INSTANCE_1_BOLT_URI="bolt://localhost:7687" \
25-
CONFIG_INSTANCE_1_BOLT_USERNAME="neo4j" \
26-
CONFIG_INSTANCE_1_BOLT_PASSWORD="passw0rd" \
27-
CONFIG_INSTANCE_1_QUERY_LOG_PORT="9500" \
28-
CONFIG_INSTANCE_1_LOG_CONFIG_PATH="/var/lib/neo4j/conf/server-logs.xml" \
29-
CONFIG_INSTANCE_1_QUERY_LOG_MIN_DURATION="100" \
30-
NEO4J_ACCEPT_LICENSE_AGREEMENT="yes" \
31-
NEO4J_AUTH=neo4j/passw0rd \
32-
NEO4J_EDITION="enterprise" \
33-
NEO4J_server_metrics_prometheus_enabled="true" \
34-
NEO4J_server_metrics_prometheus_endpoint="localhost:2004" \
35-
NEO4J_server_metrics_filter="*" \
36-
agent console -s'
27+
tar -xzf $NEO4J_HOME/products/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz --strip-components 1 && mv bin/agent-<NOM_VERSION>-linux-amd64 /bin/agent
28+
----
29+
30+
** Mounted
31+
*** NOM agent binaries can be mounted into the container as a volume mount and installed into a system binaries directory for access to `agent` command.
32+
+
33+
[source, shell]
34+
----
35+
docker run -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/agent/binary>:/agent/bin neo4j/neo4j:latest --name <neo4j container name>
36+
----
37+
+
38+
Wait for the container to be ready and then run :
39+
+
40+
[source, shell]
41+
----
42+
docker exec -it <neo4j container name> -- /bin/sh -c "mv /agent/bin/agent-*-linux-amd64 /bin/agent"
43+
----
44+
45+
** Downloaded
46+
*** NOM agent package can be directly downloaded on to the Neo4j container and then extracted and installed.
47+
+
48+
[source, shell]
49+
----
50+
wget -O /tmp/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz https://dist.neo4j.org/ops-manager/<NOM_VERSION>/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz
51+
tar -xzf /tmp/neo4j-ops-manager-agent-*-linux-amd64.tar.gz --strip-components 1 && mv bin/agent-*-linux-amd64 /bin/agent
3752
----
3853

39-
. Start a Neo4j instance Docker container with NOM agent using a custom entry point:
54+
* Run the NOM agent, which involves executing the agent binary with appropriate agent configuration.
55+
** Agent configuration can be provided as environment variables set during docker `exec` command or as part of container startup command.
56+
** Run agent in console mode
4057
+
58+
[source, shell]
59+
----
60+
docker exec -dit <neo4j container name> -- /bin/sh 'CONFIG_AGENT_NAME="test_local" \
61+
CONFIG_AGENT_DESCRIPTION="local self-registered agent" \
62+
CONFIG_SERVER_GRPC_ADDRESS="server:9090" \
63+
CONFIG_SERVER_HTTP_ADDRESS="https://server:8080" \
64+
CONFIG_TOKEN_URL=https://server:8080/api/login \
65+
CONFIG_TOKEN_CLIENT_ID=rj34hg6456ghrhjthet45hg6 \
66+
CONFIG_TOKEN_CLIENT_SECRET=5h346jhrjkthkjthjerhtkrt \
67+
CONFIG_TLS_TRUSTED_CERTS=/path/to/trusted/certs/pem/file \
68+
CONFIG_LOG_FILE=/path/to/nom-agent/log.txt \
69+
CONFIG_LOG_LEVEL="debug" \
70+
CONFIG_INSTANCE_1_NAME="test_local" \
71+
CONFIG_INSTANCE_1_BOLT_URI="bolt://localhost:7687" \
72+
CONFIG_INSTANCE_1_BOLT_USERNAME="neo4j" \
73+
CONFIG_INSTANCE_1_BOLT_PASSWORD="passw0rd" \
74+
CONFIG_INSTANCE_1_QUERY_LOG_PORT="9500" \
75+
CONFIG_INSTANCE_1_LOG_CONFIG_PATH="/var/lib/neo4j/conf/server-logs.xml" \
76+
CONFIG_INSTANCE_1_QUERY_LOG_MIN_DURATION="100" \
77+
agent console -s'
78+
----
79+
4180
[[entrypoint]]
81+
== Start a Neo4j container with NOM agent using a custom entry point
82+
83+
Start the Neo4j container by running the entrypoint script `/startup/docker-entrypoint.sh` with the command `neo4j`.
84+
To run the NOM agent on the container startup, this entrypoint needs to be overridden with a custom entrypoint script as shown:
85+
86+
* In console mode:
87+
+
4288
.entrypoint.sh
4389
[source, shell]
4490
----
4591
#!/bin/bash -eu
4692
4793
/startup/docker-entrypoint.sh neo4j &
48-
tar -xzf products/neo4j-ops-manager-agent-*-linux-amd64.tar.gz --strip-components 1 \
49-
&& mv bin/agent /bin/agent \
50-
&& CONFIG_AGENT_NAME="test_local" \
94+
CONFIG_AGENT_NAME="test_local" \
5195
CONFIG_AGENT_DESCRIPTION="local self-registered agent" \
5296
CONFIG_SERVER_GRPC_ADDRESS="server:9090" \
5397
CONFIG_SERVER_HTTP_ADDRESS="https://server:8080" \
98+
CONFIG_TOKEN_URL=https://server:8080/api/login \
99+
CONFIG_TOKEN_CLIENT_ID=rj34hg6456ghrhjthet45hg6 \
100+
CONFIG_TOKEN_CLIENT_SECRET=5h346jhrjkthkjthjerhtkrt \
101+
CONFIG_TLS_TRUSTED_CERTS=/path/to/trusted/certs/pem/file \
102+
CONFIG_LOG_FILE=/path/to/nom-agent/log.txt \
54103
CONFIG_LOG_LEVEL="debug" \
55104
CONFIG_INSTANCE_1_NAME="test_local" \
56105
CONFIG_INSTANCE_1_BOLT_URI="bolt://localhost:7687" \
@@ -59,16 +108,63 @@ CONFIG_INSTANCE_1_BOLT_PASSWORD="passw0rd" \
59108
CONFIG_INSTANCE_1_QUERY_LOG_PORT="9500" \
60109
CONFIG_INSTANCE_1_LOG_CONFIG_PATH="/var/lib/neo4j/conf/server-logs.xml" \
61110
CONFIG_INSTANCE_1_QUERY_LOG_MIN_DURATION="100" \
62-
NEO4J_ACCEPT_LICENSE_AGREEMENT="yes" \
63-
NEO4J_AUTH=neo4j/passw0rd \
64-
NEO4J_EDITION="enterprise" \
65-
NEO4J_server_metrics_prometheus_enabled="true" \
66-
NEO4J_server_metrics_prometheus_endpoint="localhost:2004" \
67-
NEO4J_server_metrics_filter="*" \
68111
agent console -s
69112
----
70113
+
71114
[source, shell, role=noheader]
72115
----
73116
docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/custom/entrypoint>:/custom $NEO4J_IMAGE
74117
----
118+
119+
* In service mode:
120+
+
121+
.entrypoint.sh
122+
[source, shell]
123+
----
124+
#!/bin/bash -eu
125+
126+
/startup/docker-entrypoint.sh neo4j &
127+
agent service install
128+
cat > neo4j-ops-manager-agent.service <<EOF
129+
[Service]
130+
Environment="CONFIG_SERVER_GRPC_ADDRESS=<server grpc address>"
131+
Environment="CONFIG_SERVER_HTTP_ADDRESS=<server http address>"
132+
Environment="CONFIG_TOKEN_URL=<server http login url>"
133+
Environment="CONFIG_TOKEN_CLIENT_ID=<client id>"
134+
Environment="CONFIG_TOKEN_CLIENT_SECRET=<client secret>"
135+
Environment="CONFIG_TLS_TRUSTED_CERTS=</path/to/trusted/certs/pem/file>"
136+
Environment="CONFIG_LOG_FILE=</path/to/nom-agent/log.txt>"
137+
Environment="CONFIG_INSTANCE_1_NAME=<instance name>"
138+
Environment="CONFIG_INSTANCE_1_BOLT_URI=<bolt uri of the local instance>"
139+
Environment="CONFIG_INSTANCE_1_BOLT_USERNAME=<local instance user name>"
140+
Environment="CONFIG_INSTANCE_1_BOLT_PASSWORD=<local instance password>"
141+
Environment="CONFIG_INSTANCE_1_QUERY_LOG_PORT=<an available port>"
142+
Environment="CONFIG_INSTANCE_1_LOG_CONFIG_PATH=<path to server-logs.xml>"
143+
EOF
144+
systemctl start neo4j-ops-manager-agent.service
145+
----
146+
+
147+
[source, shell, role=noheader]
148+
----
149+
docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/custom/entrypoint>:/custom $NEO4J_IMAGE
150+
----
151+
152+
== External monitoring
153+
154+
A NOM agent can be run in either the console or service mode on the Docker host.
155+
It can be configured to have access to Neo4j container resources.
156+
Apply the following additional configurations to the Neo4j container run config to enable an external NOM agent to monitor the instance correctly:
157+
158+
[source, shell, role=nocopy]
159+
----
160+
docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes \
161+
-v <path/to/custom/entrypoint>:/custom \
162+
-v </path/to/local/neo4j/home>:/var/lib/neo4j \ <1>
163+
-p "8884:2004" \ <2>
164+
-p "9500:9500" \ <3>
165+
neo4j/neo4j:latest
166+
----
167+
168+
<1> Neo4j home directory needs to be mounted back on the Docker host to enable access to agent.
169+
<2> Neo4j prometheus endpoint port (2004) needs to be exposed via port mapping.
170+
<3> Query log port (9500) needs to be mapped for log appender to forward query logs.

0 commit comments

Comments
 (0)