Skip to content

Conversation

@precla
Copy link
Collaborator

@precla precla commented Nov 8, 2024

@precla
Copy link
Collaborator Author

precla commented Nov 8, 2024

Valgrind test (shortened to the summary):

$ valgrind --leak-check=full --show-leak-kinds=all --errors-for-leak-kinds=all --track-origins=yes --read-var-info=yes ./out/bin/ssh

...


==49912== LEAK SUMMARY:
==49912==    definitely lost: 2,992 bytes in 39 blocks
==49912==    indirectly lost: 176 bytes in 11 blocks
==49912==      possibly lost: 2,168 bytes in 8 blocks
==49912==    still reachable: 18,497 bytes in 241 blocks
==49912==         suppressed: 0 bytes in 0 blocks

@plajjan
Copy link
Contributor

plajjan commented Nov 22, 2024

@precla I'm not sure how well Valgrind works for an Acton app given that we have a GC. I wonder how it determines something is lost?

We have our own memory measurements already as part of the built-in testing functionality, so perhaps that can be used?

@precla
Copy link
Collaborator Author

precla commented Nov 29, 2024

@plajjan furher TODO's would be having the possibility to start an SSH session from another Acton program and, for example, execute commands on the connected SSH server?

@precla precla self-assigned this Jan 16, 2025

def get_ssh_session():
return _ssh_session

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
action def get_affinity() -> u64:
NotImplemented

then need to implement the necessary C call to return the actual affinity of the Client actor. We need this as action so the Channel actor can query the Client about affinity, so the Channel actor in turn can make sure it gets pinned to the same worker thread as the Client.

Comment on lines +57 to +60
proc def _init() -> None:
"""Initialize the SSH Channel"""
NotImplemented
_init()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
proc def _init() -> None:
"""Initialize the SSH Channel"""
NotImplemented
_init()
proc def _ssh_init() -> None:
"""Initialize the SSH Channel"""
NotImplemented
def _set_affinity(affinity: u64) -> None:
NotImplemented
def _init():
aff = client.get_affinity()
set_affinity(aff)
after 0: _ssh_init()
_init()

the channel actor needs to be running on the same worker thread as the Client, since it accesses the same socket and accessing the socket in our libuv IO loop means we need to be running on the same worker therad. libuv is not thread safe, so accessing the IO loop of another worker thread will cause crashes. It might be working for you while developing today, but it's because there is so little going on. If we write a test with 1000 concurrently running SSH clients, it would mostly likely segfault.

Actor continuations are run by worker threads. Per default they go to the global shared readyQ and then a "random" worker thread picks it up from there. The actor initialization (running the code body of the actor) always runs in a random worker, so the first thing it should do is set affinity, then any subsequent continuations will run in the correct worker thread. Thus, we must NOT run the ssh init stuff too early, that's why I'm suggesting here to first set affinity then do after 0: ... which schedules that method to run, so the correct worker thread can pick it up.

There are some affinity related functions in rts.h or around there. See the other IO actors that interact with it. We can read out our affinity by reading the $Actor->affinity attribute (or whatever it is called).

Let me know if you have any questions and we can walk through it! :)

@precla
Copy link
Collaborator Author

precla commented Feb 28, 2025

@plajjan see latest two commits. Added affinity and the capability to return data that is retrieved via netconf.

but i face the following issue: the retrieved data via ssh can not be properly stored in a string. see the following example:

for reasons yet unknown the string is in the last iteration reduced by length, compared to the iteration before the last one...

output
SSH extension successfully initialized
Connecting to SSH server
setting payload
sending payload


strlen buf: 0
strlen tmp: 1018
sizeof tmp: 1024
strlen buffer after snprintf: 1018

<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><capabilities><capability>urn:ietf:params:netconf:base:1.0</capability><capability>urn:ietf:params:netconf:base:1.1</capability><capability>urn:ietf:params:netconf:capability:writable-running:1.0</capability><capability>urn:ietf:params:netconf:capability:candidate:1.0</capability><capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability><capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability><capability>urn:ietf:params:netconf:capability:validate:1.1</capability><capability>urn:ietf:params:netconf:capability:startup:1.0</capability><capability>urn:ietf:params:netconf:capability:xpath:1.0</capability><capability>urn:ietf:params:netconf:capability:with-defaults:1.0?basic-mode=explicit&amp;also-supported=report-all,report-all-tagged,trim,explicit</capability><capability>urn:ietf:params:netconf:capability:notification:1.0</capability><capability>urn:ietf:params:netconf:capability:interleave:1.0</capability>

strlen buf: 1018
strlen tmp: 1011
sizeof tmp: 1024
strlen buffer after snprintf: 2029

<capability>urn:ietf:params:netconf:capability:url:1.0?scheme=scp,http,https,ftp,sftp,ftps,file</capability><capability>urn:ietf:params:xml:ns:yang:ietf-yang-metadata?module=ietf-yang-metadata&amp;revision=2016-08-05</capability><capability>urn:ietf:params:xml:ns:yang:ietf-inet-types?module=ietf-inet-types&amp;revision=2013-07-15</capability><capability>urn:ietf:params:xml:ns:yang:ietf-yang-types?module=ietf-yang-types&amp;revision=2013-07-15</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-acm?module=ietf-netconf-acm&amp;revision=2018-02-14</capability><capability>urn:ietf:params:netconf:capability:yang-library:1.1?revision=2019-01-04&amp;content-id=3204730553</capability><capability>urn:sysrepo:plugind?module=sysrepo-plugind&amp;revision=2022-08-26</capability><capability>urn:ietf:params:xml:ns:netconf:base:1.0?module=ietf-netconf&amp;revision=2013-09-29&amp;features=writable-running,candidate,confirmed-commit,rollback-on-error,validate,startup,url,xpath</capability><capability>

strlen buf: 2029
strlen tmp: 1028
sizeof tmp: 1024
strlen buffer after snprintf: 3053

urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults?module=ietf-netconf-with-defaults&amp;revision=2011-06-01</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-notifications?module=ietf-netconf-notifications&amp;revision=2012-02-06</capability><capability>urn:ietf:params:xml:ns:netconf:notification:1.0?module=notifications&amp;revision=2008-07-14</capability><capability>urn:ietf:params:xml:ns:netmod:notification?module=nc-notifications&amp;revision=2008-07-14</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring?module=ietf-netconf-monitoring&amp;revision=2010-10-04</capability><capability>urn:ietf:params:xml:ns:yang:ietf-x509-cert-to-name?module=ietf-x509-cert-to-name&amp;revision=2014-12-10</capability><capability>urn:ietf:params:xml:ns:yang:iana-crypt-hash?module=iana-crypt-hash&amp;revision=2014-04-04&amp;features=crypt-hash-md5,crypt-hash-sha-256,crypt-hash-sha-512</capability><capability>urn:ieee:std:802.1Q:yang:ieee802-dot1q-bridge?module=ieee802-dot1q-bridge&amp

strlen buf: 3053
strlen tmp: 873
sizeof tmp: 1024
strlen buffer after snprintf: 3926

;revision=2020-11-06&amp;deviations=r3-bridges</capability><capability>urn:ieee:std:802.1Q:yang:ieee802-types?module=ieee802-types&amp;revision=2020-06-04</capability><capability>urn:ietf:params:xml:ns:yang:iana-if-type?module=iana-if-type&amp;revision=2021-06-21</capability><capability>urn:ieee:std:802.1Q:yang:ieee802-dot1q-types?module=ieee802-dot1q-types&amp;revision=2020-06-04</capability><capability>http://openconfig.net/yang/interfaces?module=openconfig-interfaces&amp;revision=2021-04-06</capability><capability>http://openconfig.net/yang/types/yang?module=openconfig-yang-types&amp;revision=2021-07-14</capability><capability>http://openconfig.net/yang/openconfig-ext?module=openconfig-extensions&amp;revision=2020-06-16</capability><capability>http://openconfig.net/yang/openconfig-types?module=openconfig-types&amp;revision=2019-04-16</capability><capability>

strlen buf: 3926
strlen tmp: 986
sizeof tmp: 1024
strlen buffer after snprintf: 4912

http://openconfig.net/yang/types/inet?module=openconfig-inet-types&amp;revision=2021-08-17</capability><capability>http://openconfig.net/yang/local-routing?module=openconfig-local-routing&amp;revision=2022-11-01</capability><capability>http://openconfig.net/yang/policy-types?module=openconfig-policy-types&amp;revision=2022-11-08</capability><capability>http://openconfig.net/yang/bfd?module=openconfig-bfd&amp;revision=2023-02-06</capability><capability>urn:ietf:params:xml:ns:yang:ietf-system?module=ietf-system&amp;revision=2014-08-06&amp;deviations=r3-system</capability><capability>http://openconfig.net/yang/wifi/wifi-types?module=openconfig-wifi-types&amp;revision=2022-03-24</capability><capability>http://openconfig.net/yang/wifi-phy?module=openconfig-wifi-phy&amp;revision=2022-03-24&amp;deviations=r3-wifi-mods</capability><capability>http://openconfig.net/yang/wifi-mac?module=openconfig-wifi-mac&amp;revision=2022-03-24&amp;deviations=r3-wifi-mods</capability><capability>

strlen buf: 3824
strlen tmp: 160
sizeof tmp: 1024
strlen buffer after snprintf: 3824

http://openconfig.net/yang/vlan-types?module=openconfig-vlan-types&amp;revision=2022-05-24</capability></capabilities><session-id>116</session-id></hello>]]>]]>

buffer:
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><capabilities><capability>urn:ietf:params:netconf:base:1.0</capability><capability>urn:ietf:params:netconf:base:1.1</capability><capability>urn:ietf:params:netconf:capability:writable-running:1.0</capability><capability>urn:ietf:params:netconf:capability:candidate:1.0</capability><capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability><capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability><capability>urn:ietf:params:netconf:capability:validate:1.1</capability><capability>urn:ietf:params:netconf:capability:startup:1.0</capability><capability>urn:ietf:params:netconf:capability:xpath:1.0</capability><capability>urn:ietf:params:netconf:capability:with-defaults:1.0?basic-mode=explicit&amp;also-supported=report-all,report-all-tagged,trim,explicit</capability><capability>urn:ietf:params:netconf:capability:notification:1.0</capability><capability>urn:ietf:params:netconf:capability:interleave:1.0</capability><capability>urn:ietf:params:netconf:capability:url:1.0?scheme=scp,http,https,ftp,sftp,ftps,file</capability><capability>urn:ietf:params:xml:ns:yang:ietf-yang-metadata?module=ietf-yang-metadata&amp;revision=2016-08-05</capability><capability>urn:ietf:params:xml:ns:yang:ietf-inet-types?module=ietf-inet-types&amp;revision=2013-07-15</capability><capability>urn:ietf:params:xml:ns:yang:ietf-yang-types?module=ietf-yang-types&amp;revision=2013-07-15</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-acm?module=ietf-netconf-acm&amp;revision=2018-02-14</capability><capability>urn:ietf:params:netconf:capability:yang-library:1.1?revision=2019-01-04&amp;content-id=3204730553</capability><capability>urn:sysrepo:plugind?module=sysrepo-plugind&amp;revision=2022-08-26</capability><capability>urn:ietf:params:xml:ns:netconf:base:1.0?module=ietf-netconf&amp;revision=2013-09-29&amp;features=writable-running,candidate,confirmed-commit,rollback-on-error,validate,startup,url,xpath</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults?module=ietf-netconf-with-defaults&amp;revision=2011-06-01</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-notifications?module=ietf-netconf-notifications&amp;revision=2012-02-06</capability><capability>urn:ietf:params:xml:ns:netconf:notification:1.0?module=notifications&amp;revision=2008-07-14</capability><capability>urn:ietf:params:xml:ns:netmod:notification?module=nc-notifications&amp;revision=2008-07-14</capability><capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring?module=ietf-netconf-monitoring&amp;revision=2010-10-04</capability><capability>urn:ietf:params:xml:ns:yang:ietf-x509-cert-to-name?module=ietf-x509-cert-to-name&amp;revision=2014-12-10</capability><capability>urn:ietf:params:xml:ns:yang:iana-crypt-hash?module=iana-crypt-hash&amp;revision=2014-04-04&amp;features=crypt-hash-md5,crypt-hash-sha-256,crypt-hash-sha-512</capability><capability>urn:ieee:std:802.1Q:yang:ieee802-dot1q-bridge?module=ieee802-dot1q-bridge&amp;revision=2020-11-06&amp;deviations=r3-bridges</capability><capability>urn:ieee:std:802.1Q:yang:ieee802-types?module=ieee802-types&amp;revision=2020-06-04</capability><capability>urn:ietf:params:xml:ns:yang:iana-if-type?module=iana-if-type&amp;revision=2021-06-21</capability><capability>urn:ieee:std:802.1Q:yang:ieee802-dot1q-types?module=ieee802-dot1q-types&amp;revision=2020-06-04</capability><capability>http://openconfig.net/yang/interfaces?module=openconfig-interfaces&amp;revision=2021-04-06</capability><capability>http://openconfig.net/yang/types/yang?module=openconfig-yang-types&amp;revision=2021-07-14</capability><capability>http://openconfig.net/yang/openconfig-ext?module=openconfig-extensions&amp;revision=2020-06-16</capability><capability>http://opencon


ERROR: This is the automatic crash handler for ./out/bin/ssh
ERROR: segmentation fault
NOTE: this is likely a bug in acton, please report this at:
NOTE: https://github.com/actonlang/acton/issues/new
NOTE: include the backtrace printed below between -- 8< -- lines

-- 8< --------- BACKTRACE --------------------
(lldb) process attach --pid 96073
Process 96073 stopped
* thread #1, name = 'ssh', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
  thread #2, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #3, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #4, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #5, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #6, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #7, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #8, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #9, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #10, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
  thread #11, name = 'GC-marker-0', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
  thread #12, name = 'GC-marker-1', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
  thread #13, name = 'GC-marker-2', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
  thread #14, name = 'GC-marker-3', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
  thread #15, name = 'GC-marker-4', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
  thread #16, name = 'GC-marker-5', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
  thread #17, name = 'GC-marker-6', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
  thread #18, name = 'Worker 1', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
  thread #19, name = 'Worker 2', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
  thread #20, name = 'Worker 3', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
  thread #21, name = 'Worker 4', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
  thread #22, name = 'Worker 5', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
  thread #23, name = 'Worker 6', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
  thread #24, name = 'Worker 7', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
  thread #25, name = 'Worker 8', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
Executable module set to "~/acton-ssh/out/bin/ssh".
Architecture set to: x86_64-pc-linux-gnu.
(lldb) thread backtrace all
* thread #1, name = 'ssh', stop reason = signal SIGSTOP
  * frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=3, a2=140729489190336, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=3, a2=140729489190336, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:75:11
    frame #3: 0x0000734ffe77dd25 libc.so.6`__GI_epoll_pwait(epfd=3, events=0x00007ffe23366dc0, maxevents=1024, timeout=<unavailable>, set=0x0000000000000000) at epoll_pwait.c:40:10
    frame #4: 0x000000000119d536 ssh`uv__io_poll(loop=0x0000734ffe548800, timeout=-1) at linux.c:1322:12
    frame #5: 0x000000000118e2f0 ssh`uv_run(loop=0x0000734ffe548800, mode=UV_RUN_DEFAULT) at core.c:448:5
    frame #6: 0x0000000001144128 ssh`main(argc=1, argv=<unavailable>) at rts.c:2884:13
    frame #7: 0x0000734ffe68b488 libc.so.6`__libc_start_call_main(main=(ssh`main at rts.c:2318), argc=1, argv=0x00007ffe2336a958) at libc_start_call_main.h:58:16
    frame #8: 0x0000734ffe68b54c libc.so.6`__libc_start_main_impl(main=(ssh`main at rts.c:2318), argc=1, argv=0x00007ffe2336a958, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007ffe2336a948) at libc-start.c:360:3
    frame #9: 0x000000000110b83a ssh`_start at start-2.33.S:120
  thread #2, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #3, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #4, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #5, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #6, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #7, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #8, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #9, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #10, name = 'iou-sqp-96073', stop reason = signal SIGSTOP
    frame #0: 0x0000000000000000
  thread #11, name = 'GC-marker-0', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
    frame #1: 0x0000734ffe6f64bc libc.so.6`__futex_abstimed_wait_common [inlined] __futex_abstimed_wait_common64(private=<unavailable>, futex_word=<unavailable>, expected=<unavailable>, op=<unavailable>, abstime=<unavailable>, cancel=<unavailable>) at futex-internal.c:57:12
    frame #2: 0x0000734ffe6f64a0 libc.so.6`__futex_abstimed_wait_common(futex_word=<unavailable>, expected=<unavailable>, clockid=<unavailable>, abstime=<unavailable>, private=<unavailable>, cancel=<unavailable>) at futex-internal.c:87:9
    frame #3: 0x0000734ffe6f8c0e libc.so.6`___pthread_cond_wait [inlined] __pthread_cond_wait_common(cond=0x00000000014106a8, mutex=0x0000000001410a78, clockid=0, abstime=0x0000000000000000) at pthread_cond_wait.c:426:13
    frame #4: 0x0000734ffe6f8bb0 libc.so.6`___pthread_cond_wait(cond=0x00000000014106a8, mutex=0x0000000001410a78) at pthread_cond_wait.c:458:10
    frame #5: 0x0000000001185b84 ssh`GC_wait_marker at pthread_support.c:2948:9
    frame #6: 0x00000000011797c5 ssh`GC_help_marker(my_mark_no=0) at mark.c:1264:7
    frame #7: 0x0000000001183be9 ssh`GC_mark_thread(id=<unavailable>) at pthread_support.c:493:7
    frame #8: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #9: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #12, name = 'GC-marker-1', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
    frame #1: 0x0000734ffe6f64bc libc.so.6`__futex_abstimed_wait_common [inlined] __futex_abstimed_wait_common64(private=<unavailable>, futex_word=<unavailable>, expected=<unavailable>, op=<unavailable>, abstime=<unavailable>, cancel=<unavailable>) at futex-internal.c:57:12
    frame #2: 0x0000734ffe6f64a0 libc.so.6`__futex_abstimed_wait_common(futex_word=<unavailable>, expected=<unavailable>, clockid=<unavailable>, abstime=<unavailable>, private=<unavailable>, cancel=<unavailable>) at futex-internal.c:87:9
    frame #3: 0x0000734ffe6f8c0e libc.so.6`___pthread_cond_wait [inlined] __pthread_cond_wait_common(cond=0x00000000014106a8, mutex=0x0000000001410a78, clockid=0, abstime=0x0000000000000000) at pthread_cond_wait.c:426:13
    frame #4: 0x0000734ffe6f8bb0 libc.so.6`___pthread_cond_wait(cond=0x00000000014106a8, mutex=0x0000000001410a78) at pthread_cond_wait.c:458:10
    frame #5: 0x0000000001185b84 ssh`GC_wait_marker at pthread_support.c:2948:9
    frame #6: 0x00000000011797c5 ssh`GC_help_marker(my_mark_no=0) at mark.c:1264:7
    frame #7: 0x0000000001183be9 ssh`GC_mark_thread(id=<unavailable>) at pthread_support.c:493:7
    frame #8: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #9: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #13, name = 'GC-marker-2', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
    frame #1: 0x0000734ffe6f64bc libc.so.6`__futex_abstimed_wait_common [inlined] __futex_abstimed_wait_common64(private=<unavailable>, futex_word=<unavailable>, expected=<unavailable>, op=<unavailable>, abstime=<unavailable>, cancel=<unavailable>) at futex-internal.c:57:12
    frame #2: 0x0000734ffe6f64a0 libc.so.6`__futex_abstimed_wait_common(futex_word=<unavailable>, expected=<unavailable>, clockid=<unavailable>, abstime=<unavailable>, private=<unavailable>, cancel=<unavailable>) at futex-internal.c:87:9
    frame #3: 0x0000734ffe6f8c0e libc.so.6`___pthread_cond_wait [inlined] __pthread_cond_wait_common(cond=0x00000000014106a8, mutex=0x0000000001410a78, clockid=0, abstime=0x0000000000000000) at pthread_cond_wait.c:426:13
    frame #4: 0x0000734ffe6f8bb0 libc.so.6`___pthread_cond_wait(cond=0x00000000014106a8, mutex=0x0000000001410a78) at pthread_cond_wait.c:458:10
    frame #5: 0x0000000001185b84 ssh`GC_wait_marker at pthread_support.c:2948:9
    frame #6: 0x00000000011797c5 ssh`GC_help_marker(my_mark_no=0) at mark.c:1264:7
    frame #7: 0x0000000001183be9 ssh`GC_mark_thread(id=<unavailable>) at pthread_support.c:493:7
    frame #8: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #9: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #14, name = 'GC-marker-3', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
    frame #1: 0x0000734ffe6f64bc libc.so.6`__futex_abstimed_wait_common [inlined] __futex_abstimed_wait_common64(private=<unavailable>, futex_word=<unavailable>, expected=<unavailable>, op=<unavailable>, abstime=<unavailable>, cancel=<unavailable>) at futex-internal.c:57:12
    frame #2: 0x0000734ffe6f64a0 libc.so.6`__futex_abstimed_wait_common(futex_word=<unavailable>, expected=<unavailable>, clockid=<unavailable>, abstime=<unavailable>, private=<unavailable>, cancel=<unavailable>) at futex-internal.c:87:9
    frame #3: 0x0000734ffe6f8c0e libc.so.6`___pthread_cond_wait [inlined] __pthread_cond_wait_common(cond=0x00000000014106a8, mutex=0x0000000001410a78, clockid=0, abstime=0x0000000000000000) at pthread_cond_wait.c:426:13
    frame #4: 0x0000734ffe6f8bb0 libc.so.6`___pthread_cond_wait(cond=0x00000000014106a8, mutex=0x0000000001410a78) at pthread_cond_wait.c:458:10
    frame #5: 0x0000000001185b84 ssh`GC_wait_marker at pthread_support.c:2948:9
    frame #6: 0x00000000011797c5 ssh`GC_help_marker(my_mark_no=0) at mark.c:1264:7
    frame #7: 0x0000000001183be9 ssh`GC_mark_thread(id=<unavailable>) at pthread_support.c:493:7
    frame #8: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #9: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #15, name = 'GC-marker-4', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
    frame #1: 0x0000734ffe6f64bc libc.so.6`__futex_abstimed_wait_common [inlined] __futex_abstimed_wait_common64(private=<unavailable>, futex_word=<unavailable>, expected=<unavailable>, op=<unavailable>, abstime=<unavailable>, cancel=<unavailable>) at futex-internal.c:57:12
    frame #2: 0x0000734ffe6f64a0 libc.so.6`__futex_abstimed_wait_common(futex_word=<unavailable>, expected=<unavailable>, clockid=<unavailable>, abstime=<unavailable>, private=<unavailable>, cancel=<unavailable>) at futex-internal.c:87:9
    frame #3: 0x0000734ffe6f8c0e libc.so.6`___pthread_cond_wait [inlined] __pthread_cond_wait_common(cond=0x00000000014106a8, mutex=0x0000000001410a78, clockid=0, abstime=0x0000000000000000) at pthread_cond_wait.c:426:13
    frame #4: 0x0000734ffe6f8bb0 libc.so.6`___pthread_cond_wait(cond=0x00000000014106a8, mutex=0x0000000001410a78) at pthread_cond_wait.c:458:10
    frame #5: 0x0000000001185b84 ssh`GC_wait_marker at pthread_support.c:2948:9
    frame #6: 0x00000000011797c5 ssh`GC_help_marker(my_mark_no=0) at mark.c:1264:7
    frame #7: 0x0000000001183be9 ssh`GC_mark_thread(id=<unavailable>) at pthread_support.c:493:7
    frame #8: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #9: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #16, name = 'GC-marker-5', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
    frame #1: 0x0000734ffe6f64bc libc.so.6`__futex_abstimed_wait_common [inlined] __futex_abstimed_wait_common64(private=<unavailable>, futex_word=<unavailable>, expected=<unavailable>, op=<unavailable>, abstime=<unavailable>, cancel=<unavailable>) at futex-internal.c:57:12
    frame #2: 0x0000734ffe6f64a0 libc.so.6`__futex_abstimed_wait_common(futex_word=<unavailable>, expected=<unavailable>, clockid=<unavailable>, abstime=<unavailable>, private=<unavailable>, cancel=<unavailable>) at futex-internal.c:87:9
    frame #3: 0x0000734ffe6f8c0e libc.so.6`___pthread_cond_wait [inlined] __pthread_cond_wait_common(cond=0x00000000014106a8, mutex=0x0000000001410a78, clockid=0, abstime=0x0000000000000000) at pthread_cond_wait.c:426:13
    frame #4: 0x0000734ffe6f8bb0 libc.so.6`___pthread_cond_wait(cond=0x00000000014106a8, mutex=0x0000000001410a78) at pthread_cond_wait.c:458:10
    frame #5: 0x0000000001185b84 ssh`GC_wait_marker at pthread_support.c:2948:9
    frame #6: 0x00000000011797c5 ssh`GC_help_marker(my_mark_no=0) at mark.c:1264:7
    frame #7: 0x0000000001183be9 ssh`GC_mark_thread(id=<unavailable>) at pthread_support.c:493:7
    frame #8: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #9: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #17, name = 'GC-marker-6', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe6f5e56 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=<unavailable>, a3=<unavailable>, a4=<unavailable>, a5=<unavailable>, a6=<unavailable>, nr=202) at cancellation.c:64:1
    frame #1: 0x0000734ffe6f64bc libc.so.6`__futex_abstimed_wait_common [inlined] __futex_abstimed_wait_common64(private=<unavailable>, futex_word=<unavailable>, expected=<unavailable>, op=<unavailable>, abstime=<unavailable>, cancel=<unavailable>) at futex-internal.c:57:12
    frame #2: 0x0000734ffe6f64a0 libc.so.6`__futex_abstimed_wait_common(futex_word=<unavailable>, expected=<unavailable>, clockid=<unavailable>, abstime=<unavailable>, private=<unavailable>, cancel=<unavailable>) at futex-internal.c:87:9
    frame #3: 0x0000734ffe6f8c0e libc.so.6`___pthread_cond_wait [inlined] __pthread_cond_wait_common(cond=0x00000000014106a8, mutex=0x0000000001410a78, clockid=0, abstime=0x0000000000000000) at pthread_cond_wait.c:426:13
    frame #4: 0x0000734ffe6f8bb0 libc.so.6`___pthread_cond_wait(cond=0x00000000014106a8, mutex=0x0000000001410a78) at pthread_cond_wait.c:458:10
    frame #5: 0x0000000001185b84 ssh`GC_wait_marker at pthread_support.c:2948:9
    frame #6: 0x00000000011797c5 ssh`GC_help_marker(my_mark_no=0) at mark.c:1264:7
    frame #7: 0x0000000001183be9 ssh`GC_mark_thread(id=<unavailable>) at pthread_support.c:493:7
    frame #8: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #9: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #18, name = 'Worker 1', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=<unavailable>, a2=0, a3=0, a4=0, a5=0, a6=0, nr=61) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=<unavailable>, a2=0, a3=0, a4=0, a5=0, a6=0, nr=61) at cancellation.c:75:11
    frame #3: 0x0000734ffe766d6f libc.so.6`__GI___wait4(pid=<unavailable>, stat_loc=0x0000000000000000, options=0, usage=0x0000000000000000) at wait4.c:30:10
    frame #4: 0x0000734ffe766dbb libc.so.6`__GI___waitpid(pid=<unavailable>, stat_loc=<unavailable>, options=<unavailable>) at waitpid.c:38:10 [artificial]
    frame #5: 0x0000000001142809 ssh`print_trace at rts.c:2188:9 [artificial]
    frame #6: 0x0000000001142b86 ssh`crash_handler(signum=11) at rts.c:2234:5
    frame #7: 0x0000734ffe6a1cd0 libc.so.6`__restore_rt
    frame #8: 0x000000000113c1b2 ssh`ENQ_ready [inlined] spinlock_lock(f=0x0000000ceef90308) at q.c:5:12
    frame #9: 0x000000000113c1a2 ssh`ENQ_ready(a=0x0000734ffe554e60) at q.c:20:5
    frame #10: 0x000000000113f209 ssh`wt_work_cb(ev=<unavailable>) at rts.c:0
    frame #11: 0x0000000001193edd ssh`uv__run_check(loop=0x0000734ffe548400) at loop-watcher.c:67:1
    frame #12: 0x000000000118e375 ssh`uv_run(loop=0x0000734ffe548400, mode=UV_RUN_DEFAULT) at core.c:462:5
    frame #13: 0x000000000113f504 ssh`main_loop(idx=<unavailable>) at rts.c:1753:13
    frame #14: 0x0000000001185e61 ssh`GC_pthread_start_inner(sb=<unavailable>, arg=<unavailable>) at pthread_start.c:55:12
    frame #15: 0x000000000117d970 ssh`GC_call_with_stack_base(fn=(ssh`GC_pthread_start_inner at pthread_start.c:45), arg=<unavailable>) at misc.c:2307:14
    frame #16: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #17: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #19, name = 'Worker 2', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=17, a2=126787338775824, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=17, a2=126787338775824, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:75:11
    frame #3: 0x0000734ffe77dd25 libc.so.6`__GI_epoll_pwait(epfd=17, events=0x0000734ffa4a2d10, maxevents=1024, timeout=<unavailable>, set=0x0000000000000000) at epoll_pwait.c:40:10
    frame #4: 0x000000000119d536 ssh`uv__io_poll(loop=0x0000734ffe548000, timeout=-1) at linux.c:1322:12
    frame #5: 0x000000000118e2f0 ssh`uv_run(loop=0x0000734ffe548000, mode=UV_RUN_DEFAULT) at core.c:448:5
    frame #6: 0x000000000113f504 ssh`main_loop(idx=<unavailable>) at rts.c:1753:13
    frame #7: 0x0000000001185e61 ssh`GC_pthread_start_inner(sb=<unavailable>, arg=<unavailable>) at pthread_start.c:55:12
    frame #8: 0x000000000117d970 ssh`GC_call_with_stack_base(fn=(ssh`GC_pthread_start_inner at pthread_start.c:45), arg=<unavailable>) at misc.c:2307:14
    frame #9: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #10: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #20, name = 'Worker 3', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=23, a2=126787330383120, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=23, a2=126787330383120, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:75:11
    frame #3: 0x0000734ffe77dd25 libc.so.6`__GI_epoll_pwait(epfd=23, events=0x0000734ff9ca1d10, maxevents=1024, timeout=<unavailable>, set=0x0000000000000000) at epoll_pwait.c:40:10
    frame #4: 0x000000000119d536 ssh`uv__io_poll(loop=0x0000734ffe54dc00, timeout=-1) at linux.c:1322:12
    frame #5: 0x000000000118e2f0 ssh`uv_run(loop=0x0000734ffe54dc00, mode=UV_RUN_DEFAULT) at core.c:448:5
    frame #6: 0x000000000113f504 ssh`main_loop(idx=<unavailable>) at rts.c:1753:13
    frame #7: 0x0000000001185e61 ssh`GC_pthread_start_inner(sb=<unavailable>, arg=<unavailable>) at pthread_start.c:55:12
    frame #8: 0x000000000117d970 ssh`GC_call_with_stack_base(fn=(ssh`GC_pthread_start_inner at pthread_start.c:45), arg=<unavailable>) at misc.c:2307:14
    frame #9: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #10: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #21, name = 'Worker 4', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=29, a2=126787321990416, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=29, a2=126787321990416, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:75:11
    frame #3: 0x0000734ffe77dd25 libc.so.6`__GI_epoll_pwait(epfd=29, events=0x0000734ff94a0d10, maxevents=1024, timeout=<unavailable>, set=0x0000000000000000) at epoll_pwait.c:40:10
    frame #4: 0x000000000119d536 ssh`uv__io_poll(loop=0x0000734ffe54d800, timeout=-1) at linux.c:1322:12
    frame #5: 0x000000000118e2f0 ssh`uv_run(loop=0x0000734ffe54d800, mode=UV_RUN_DEFAULT) at core.c:448:5
    frame #6: 0x000000000113f504 ssh`main_loop(idx=<unavailable>) at rts.c:1753:13
    frame #7: 0x0000000001185e61 ssh`GC_pthread_start_inner(sb=<unavailable>, arg=<unavailable>) at pthread_start.c:55:12
    frame #8: 0x000000000117d970 ssh`GC_call_with_stack_base(fn=(ssh`GC_pthread_start_inner at pthread_start.c:45), arg=<unavailable>) at misc.c:2307:14
    frame #9: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #10: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #22, name = 'Worker 5', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=35, a2=126787313597712, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=35, a2=126787313597712, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:75:11
    frame #3: 0x0000734ffe77dd25 libc.so.6`__GI_epoll_pwait(epfd=35, events=0x0000734ff8c9fd10, maxevents=1024, timeout=<unavailable>, set=0x0000000000000000) at epoll_pwait.c:40:10
    frame #4: 0x000000000119d536 ssh`uv__io_poll(loop=0x0000734ffe54d400, timeout=-1) at linux.c:1322:12
    frame #5: 0x000000000118e2f0 ssh`uv_run(loop=0x0000734ffe54d400, mode=UV_RUN_DEFAULT) at core.c:448:5
    frame #6: 0x000000000113f504 ssh`main_loop(idx=<unavailable>) at rts.c:1753:13
    frame #7: 0x0000000001185e61 ssh`GC_pthread_start_inner(sb=<unavailable>, arg=<unavailable>) at pthread_start.c:55:12
    frame #8: 0x000000000117d970 ssh`GC_call_with_stack_base(fn=(ssh`GC_pthread_start_inner at pthread_start.c:45), arg=<unavailable>) at misc.c:2307:14
    frame #9: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #10: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #23, name = 'Worker 6', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=41, a2=126787233234192, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=41, a2=126787233234192, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:75:11
    frame #3: 0x0000734ffe77dd25 libc.so.6`__GI_epoll_pwait(epfd=41, events=0x0000734ff3ffbd10, maxevents=1024, timeout=<unavailable>, set=0x0000000000000000) at epoll_pwait.c:40:10
    frame #4: 0x000000000119d536 ssh`uv__io_poll(loop=0x0000734ffe54d000, timeout=-1) at linux.c:1322:12
    frame #5: 0x000000000118e2f0 ssh`uv_run(loop=0x0000734ffe54d000, mode=UV_RUN_DEFAULT) at core.c:448:5
    frame #6: 0x000000000113f504 ssh`main_loop(idx=<unavailable>) at rts.c:1753:13
    frame #7: 0x0000000001185e61 ssh`GC_pthread_start_inner(sb=<unavailable>, arg=<unavailable>) at pthread_start.c:55:12
    frame #8: 0x000000000117d970 ssh`GC_call_with_stack_base(fn=(ssh`GC_pthread_start_inner at pthread_start.c:45), arg=<unavailable>) at misc.c:2307:14
    frame #9: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #10: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #24, name = 'Worker 7', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=47, a2=126787224841488, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=47, a2=126787224841488, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:75:11
    frame #3: 0x0000734ffe77dd25 libc.so.6`__GI_epoll_pwait(epfd=47, events=0x0000734ff37fad10, maxevents=1024, timeout=<unavailable>, set=0x0000000000000000) at epoll_pwait.c:40:10
    frame #4: 0x000000000119d536 ssh`uv__io_poll(loop=0x0000734ffe54fc00, timeout=-1) at linux.c:1322:12
    frame #5: 0x000000000118e2f0 ssh`uv_run(loop=0x0000734ffe54fc00, mode=UV_RUN_DEFAULT) at core.c:448:5
    frame #6: 0x000000000113f504 ssh`main_loop(idx=<unavailable>) at rts.c:1753:13
    frame #7: 0x0000000001185e61 ssh`GC_pthread_start_inner(sb=<unavailable>, arg=<unavailable>) at pthread_start.c:55:12
    frame #8: 0x000000000117d970 ssh`GC_call_with_stack_base(fn=(ssh`GC_pthread_start_inner at pthread_start.c:45), arg=<unavailable>) at misc.c:2307:14
    frame #9: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #10: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
  thread #25, name = 'Worker 8', stop reason = signal SIGSTOP
    frame #0: 0x0000734ffe701be2 libc.so.6`__syscall_cancel_arch at syscall_cancel.S:56
    frame #1: 0x0000734ffe6f5e33 libc.so.6`__internal_syscall_cancel(a1=53, a2=126787216448784, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:49:12
    frame #2: 0x0000734ffe6f5e74 libc.so.6`__syscall_cancel(a1=53, a2=126787216448784, a3=1024, a4=<unavailable>, a5=0, a6=8, nr=281) at cancellation.c:75:11
    frame #3: 0x0000734ffe77dd25 libc.so.6`__GI_epoll_pwait(epfd=53, events=0x0000734ff2ff9d10, maxevents=1024, timeout=<unavailable>, set=0x0000000000000000) at epoll_pwait.c:40:10
    frame #4: 0x000000000119d536 ssh`uv__io_poll(loop=0x0000734ffe54f800, timeout=-1) at linux.c:1322:12
    frame #5: 0x000000000118e2f0 ssh`uv_run(loop=0x0000734ffe54f800, mode=UV_RUN_DEFAULT) at core.c:448:5
    frame #6: 0x000000000113f504 ssh`main_loop(idx=<unavailable>) at rts.c:1753:13
    frame #7: 0x0000000001185e61 ssh`GC_pthread_start_inner(sb=<unavailable>, arg=<unavailable>) at pthread_start.c:55:12
    frame #8: 0x000000000117d970 ssh`GC_call_with_stack_base(fn=(ssh`GC_pthread_start_inner at pthread_start.c:45), arg=<unavailable>) at misc.c:2307:14
    frame #9: 0x0000734ffe6f970a libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:448:8
    frame #10: 0x0000734ffe77daac libc.so.6`__clone3 at clone3.S:78
(lldb) exit

-- 8< --------- END BACKTRACE ----------------

ERROR: segmentation fault
NOTE: this is likely a bug in acton, please report this at:
NOTE: https://github.com/actonlang/acton/issues/new
NOTE: include the backtrace printed above between -- 8< -- lines
[1]    96073 segmentation fault (core dumped)  ./out/bin/ssh

EDIT:
does not happen if instead of a calloc()'d buffer, i use a static size one:
replace:

char *buffer = (char*)acton_calloc(BUF_SIZE, sizeof(char));

with:

char buffer[BUF_SIZE] = {0};

edit:

is it related to this?

    // TODO: can we avoid custom malloc in libssh? like let libssh use stock
    // malloc and instead we would explicitly call free() from a finalizer()
    // All things related to buffers for receiving data and similarly would have
    // to be allocated on the GC-heap though since that data is passed outside

@precla precla changed the title Channel actor SSH module Feb 28, 2025
@precla
Copy link
Collaborator Author

precla commented Mar 5, 2025

@plajjan can you please take a look at this commit (another branch) -> 2eb43af

i can't get it to return the string that was dynamically allocated to the acton code, using return $R_CONT(c$cont, to$str(response));

do i have to create a variable in Channel and then copy the string from response to self->something? i guess here: https://github.com/actonlang/acton-ssh/blob/ssh_nc_parse_malloc/src/ssh.ext.c#L381

@precla
Copy link
Collaborator Author

precla commented Jul 8, 2025

hi @plajjan , i'm trying to figure out how to implement this affinity code. So far the changes from the diff below are added to the current code.

diff

this libssh_libuv branch + diff:

diff --git a/src/ssh.act b/src/ssh.act
index 3b8e6b4..f618cdf 100755
--- a/src/ssh.act
+++ b/src/ssh.act
@@ -34,13 +34,14 @@ actor Client(cap: net.TCPConnectCap,
     def get_subsystem() -> str:
         return subsystem
 
+    action def get_affinity() -> u64:
+        NotImplemented
+
     proc def _init() -> None:
         """Initialize the SSH client"""
         NotImplemented
     _init()
 
-    # print("SSH Client connected")
-
 #    action def close(on_close: action(TLSConnection) -> None) -> None:
 #        """Close the connection"""
 #        NotImplemented
@@ -75,12 +76,18 @@ actor Channel(client: Client):
         NotImplemented
     _pin_affinity()
 
-    proc def _init() -> None:
+    proc def _set_affinity(affinity: u64) -> None:
+        NotImplemented
+
+    proc def _ssh_init() -> None:
         """Initialize the SSH Channel"""
         NotImplemented
-    _init()
 
-    # print("SSH Channel created")
+    proc def _init() -> None:
+        aff = client.get_affinity()
+        _set_affinity(aff)
+        after 0: _ssh_init()
+    _init()
 
     def setPayload(p: str):
         payload = p

diff --git a/src/ssh.ext.c b/src/ssh.ext.c
index 7b90f56..b89eecd 100644
--- a/src/ssh.ext.c
+++ b/src/ssh.ext.c
@@ -274,6 +274,11 @@ $R sshQ_ClientD__initG_local (sshQ_Client self, $Cont c$cont) {
     return $R_CONT(c$cont, B_None);
 }
 
+$R sshQ_ClientD__get_affinityG_local (sshQ_Client self, $Cont c$cont) {
+    printf("sshQ_ClientD__get_affinityG_local, affinity=%ld\n", self->$affinity);
+    return $R_CONT(c$cont, B_None);
+}
+
 $R sshQ_ClientD_disconnectG_local (sshQ_Client self, $Cont c$cont) {
     ssh_disconnect((ssh_session)fromB_u64(self->_ssh_session));
     ssh_free((ssh_session)fromB_u64(self->_ssh_session));
@@ -335,6 +340,21 @@ $R sshQ_ChannelD__initG_local (sshQ_Channel self, $Cont c$cont) {
     return $R_CONT(c$cont, B_None);
 }
 
+$R sshQ_ChannelD_get_affinityG_local (sshQ_Channel self, $Cont c$cont) {
+    printf("sshQ_ChannelD_get_affinityG_local, affinity=%ld\n", self->$affinity);
+    return $R_CONT(c$cont, to$int(self->$affinity));
+}
+
+$R sshQ_ChannelD__set_affinityG_local (sshQ_Channel self, $Cont C_cont, B_u64 affinity) {
+    printf("sshQ_ChannelD__set_affinityG_local, affinity=%ld\n", self->$affinity);
+    return $R_CONT(C_cont, B_None);
+}
+
+$R sshQ_ChannelD__ssh_initG_local (sshQ_Channel self, $Cont c$cont) {
+    printf("sshQ_ChannelD__ssh_initG_local, affinity=%ld\n", self->$affinity);
+    return $R_CONT(c$cont, B_None);
+}
+
 $R sshQ_ChannelD_disconnectG_local (sshQ_Channel self, $Cont c$cont) {
     int err = 0;
     ssh_channel channel = (ssh_channel)fromB_u64(self->_ssh_channel);

this compiles and runs fine. Whereas as soon when this is added to actor Channel(client: Client):, see here: #1 (comment)

    def _init():
        aff = client.get_affinity()
        set_affinity(aff)
        after 0: _ssh_init()

the following compile error occurs

Building executable from: .../acton-ssh/out/types/ssh.root.c -> ssh
install
+- install ActonProject
   +- zig build-lib ActonProject Debug x86_64-linux-gnu.2.27 1 errors
.../acton-ssh/out/types/ssh.c:1604:4: error: redefinition of 'sshQ_ChannelD__initG_local'
.../acton-ssh/src/ssh.ext.c:294:4: note: previous definition is here
error: the following command failed with 1 compilation errors:

i'm using Acton 0.26.0

@plajjan
Copy link
Contributor

plajjan commented Jul 8, 2025

@precla the _init method is called sshQ_ChannelD__initG_local in C and before your change it was written in C, i.e. in Acton we say NotImplemented

    proc def _init() -> None:
        """Initialize the SSH Channel"""
        NotImplemented
    _init()

and in your C code you have a function called sshQ_ChannelD__initG_local. Now you are creating some new functions and have implemented the C part yourself, and the _init method has changed from being a C function to being an Acton function - so you must remove the manually define sshQ_ChannelD__initG_local in the .ext.c file!

@precla
Copy link
Collaborator Author

precla commented Jul 9, 2025

@precla the _init method is called sshQ_ChannelD__initG_local in C and before your change it was written in C, i.e. in Acton we say NotImplemented

    proc def _init() -> None:
        """Initialize the SSH Channel"""
        NotImplemented
    _init()

and in your C code you have a function called sshQ_ChannelD__initG_local. Now you are creating some new functions and have implemented the C part yourself, and the _init method has changed from being a C function to being an Acton function - so you must remove the manually define sshQ_ChannelD__initG_local in the .ext.c file!

Thanks, adapted and it's compiling!

@precla precla mentioned this pull request Aug 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants