Skip to content

Commit 5d6ce32

Browse files
valeriy42valeriy42
authored
[ML] How to profile autodetect on MacOS (#2526)
This PR adds bash scripts and a description of how to profile the anomaly detection process started from the Elasticsearch Java backend.
1 parent 22cddab commit 5d6ce32

File tree

3 files changed

+202
-0
lines changed

3 files changed

+202
-0
lines changed

‎dev-tools/autodetect_profiling_macos/autodetect_profiling_on_macos.md‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Anomaly detection profiling on MacOS <!-- omit in toc -->
2+
3+
---
4+
- Author: Valeriy Khakhutskyy
5+
- Date: 2023-06-15
6+
---
7+
8+
- [Objective](#objective)
9+
- [Recording Information](#recording-information)
10+
- [Using `leaks`](#using-leaks)
11+
- [Using `xctrace` with Allocations template](#using-xctrace-with-allocations-template)
12+
- [Common issues](#common-issues)
13+
- [Changing the Executable Signature](#changing-the-executable-signature)
14+
- [If nothing works](#if-nothing-works)
15+
- [Sources](#sources)
16+
17+
## Objective
18+
19+
Profiling anomaly detection executables can be challenging because their execution depends on input from Java. This document describes how to do this using XCode Instruments.
20+
21+
To access Instruments, you must do the following:
22+
```
23+
XCode->Open Developer Tools->Instruments
24+
```
25+
26+
## Recording Information
27+
28+
Instruments tools such as Allocations allow you to run an executable or attach to a process. However, trying to attach to a running process using the UI is cumbersome because the process may be short-lived and the UI doesn't support filtering by process name.
29+
30+
So we need a bash command that periodically checks if a process named `autodetect` exists. If so, it grabs the PID of that process. Then it immediately starts Xcode's recording utility to create a recording and save it to disk.
31+
32+
### Using `leaks`
33+
34+
```bash
35+
#!/bin/bash
36+
# filename: record_memgraph.sh
37+
while true; do
38+
autodetect_pid=$(pgrep autodetect)
39+
40+
if [[ -n $autodetect_pid ]]; then
41+
echo "Process 'autodetect' found with PID $autodetect_pid."
42+
43+
# Start 'leaks' utility from Xcode to generate memgraph
44+
leaks_command="leaks $autodetect_pid --outputGraph=autodetect_${autodetect_pid}"
45+
eval $leaks_command
46+
47+
if [[ $? -eq 0 ]]; then
48+
echo "Memgraph generated successfully."
49+
break
50+
else
51+
echo "Failed to generate memgraph."
52+
break
53+
fi
54+
fi
55+
56+
# Sleep for 0.1 seconds before checking again
57+
sleep 0.1
58+
done
59+
```
60+
61+
The recording can be opened in the UI with
62+
```sh
63+
open autodetect_XXXX.memgraph
64+
```
65+
66+
For more information:
67+
- [Using Xcode's visual debugger and Instruments' modules to prevent memory overuse](https://rderik.com/blog/using-xcode-s-visual-debugger-and-instruments-modules-to-prevent-memory-overuse/)
68+
69+
70+
### Using `xctrace` with Allocations template
71+
72+
```bash
73+
#!/bin/bash
74+
# filename: record_xctrace.sh
75+
76+
while true; do
77+
autodetect_pid=$(pgrep autodetect)
78+
79+
if [[ -n $autodetect_pid ]]; then
80+
echo "Process 'autodetect' found with PID $autodetect_pid."
81+
82+
# Start 'xctrace' to record memgraph
83+
xctrace_command="xctrace record --output autodetect_${autodetect_pid}.trace --template 'Allocations' --attach $autodetect_pid"
84+
eval $xctrace_command
85+
86+
if [[ $? -eq 0 ]]; then
87+
echo "Trace file generated successfully."
88+
break
89+
else
90+
echo "Failed to generate the trace file."
91+
break
92+
fi
93+
fi
94+
95+
# Sleep for 0.1 seconds before checking again
96+
sleep 0.1
97+
done
98+
99+
```
100+
101+
## Common issues
102+
103+
### Changing the Executable Signature
104+
105+
You may receive the following error message when you select the executable:
106+
```
107+
Required kernel recording resources are in use by another document
108+
```
109+
110+
To fix the permission problems, you need to apply **in zsh** the following command to the executable (instead of `pytorch_inference`):
111+
```zsh
112+
codesign -s - -v -f --entitlements =(echo -n '<?xml version="1.0" encoding="UTF-8"?>
113+
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "[https://www.apple.com/DTDs/PropertyList-1.0.dtd](https://www.apple.com/DTDs/PropertyList-1.0.dtd)"\>
114+
<plist version="1.0">
115+
<dict>
116+
<key>com.apple.security.get-task-allow</key>
117+
<true/>
118+
</dict>
119+
</plist>') build/distribution/platform/darwin-aarch64/controller.app/Contents/MacOS/pytorch_inference
120+
```
121+
122+
If this doesn't work, create an `entitlements.xml' file in VSCode with the following content
123+
```xml
124+
<?xml version="1.0" encoding="UTF-8"?>
125+
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "https://www.apple.com/DTDs/PropertyList-1.0.dtd"\>
126+
<plist version="1.0">
127+
<dict>
128+
<key>com.apple.security.get-task-allow</key>
129+
<true/>
130+
</dict>
131+
</plist>
132+
```
133+
134+
and then codesign using the xml file:
135+
```zsh
136+
codesign -s - -v -f --entitlements ./build/distribution/platform/darwin-aarch64/controller.app/Contents/MacOS/autodetect
137+
```
138+
139+
#### If nothing works
140+
141+
Restart your computer in recovery mode (start the laptop and hold down the power button). Then go to Utilities->Terminal and run
142+
```sh
143+
csrutil disable
144+
```
145+
146+
Don't forget to re-enable the signature check by running
147+
```sh
148+
csrutil enable
149+
```
150+
when you are done.
151+
152+
153+
#### Sources
154+
1. https://stackoverflow.com/questions/74262518/why-required-kernel-recording-resources-are-in-use-by-another-document-error-i
155+
2. https://sourceware.org/gdb/wiki/PermissionsDarwin#Sign_and_entitle_the_gdb_binary

‎dev-tools/autodetect_profiling_macos/record_memgraph.sh‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#!/bin/bash
2+
while true; do
3+
autodetect_pid=$(pgrep autodetect)
4+
5+
if [[ -n $autodetect_pid ]]; then
6+
echo "Process 'autodetect' found with PID $autodetect_pid."
7+
8+
# Start 'leaks' utility from Xcode to generate memgraph
9+
leaks_command="leaks $autodetect_pid --outputGraph=autodetect_${autodetect_pid}"
10+
eval $leaks_command
11+
12+
if [[ $? -eq 0 ]]; then
13+
echo "Memgraph generated successfully."
14+
break
15+
else
16+
echo "Failed to generate memgraph."
17+
break
18+
fi
19+
fi
20+
21+
# Sleep for 0.1 seconds before checking again
22+
sleep 0.1
23+
done

‎dev-tools/autodetect_profiling_macos/record_xctrace.sh‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
#!/bin/bash
2+
3+
while true; do
4+
autodetect_pid=$(pgrep autodetect)
5+
6+
if [[ -n $autodetect_pid ]]; then
7+
echo "Process 'autodetect' found with PID $autodetect_pid."
8+
9+
# Start 'xctrace' to record memgraph
10+
xctrace_command="xctrace record --output autodetect_${autodetect_pid}.trace --template 'Allocations' --attach 'autodetect'"
11+
eval $xctrace_command
12+
13+
if [[ $? -eq 0 ]]; then
14+
echo "Trace file generated successfully."
15+
break
16+
else
17+
echo "Failed to generate the trace file."
18+
break
19+
fi
20+
fi
21+
22+
# Sleep for 0.1 seconds before checking again
23+
sleep 0.1
24+
done

0 commit comments

Comments
 (0)