Multi-thread debugging, the main thread is suspended, the child thread can not be suspended?

[YuhongZeng]

Multi-thread debugging, the main thread is suspended, the child thread can not be suspended? You can perform continue, parse, and step control on each thread.
Our expected multi-threaded debugging feature for VSCode includes the following:

1. The main thread is suspended, but the sub-threads are not suspended at the same time.
2. Each thread can be separately controlled to continue, pause, and step.
   The following video demonstrates the implementation of this multi-threaded debugging feature.

[int19h]

The main reason why we don't support this is because not all DAP clients can handle this behavior - e.g. VSCode can, but VS cannot, hence why the DAP spec makes individual thread suspend optional. That said, this functionality is actually implemented, just disabled. There's no feature flag for this, unfortunately, but it can be re-enabled fairly easily in the code:

debugpy/src/debugpy/adapter/clients.py

Lines 630 to 645 in ef9a67f

	@message_handler
	def pause_request(self, request):
	request.arguments["threadId"] = "*"
	return self.server.channel.delegate(request)
	@message_handler
	def continue_request(self, request):
	request.arguments["threadId"] = "*"
	try:
	return self.server.channel.delegate(request)
	except messaging.NoMoreMessages:
	# pydevd can sometimes allow the debuggee to exit before the queued
	# "continue" response gets sent. Thus, a failed "continue" response
	# indicating that the server disconnected should be treated as success.
	return {"allThreadsContinued": True}

Could you give it a try by changing your local copy to see if the resulting behavior actually does what you want? In our experience, having the other threads running ended up incredibly confusing in most scenarios - that was the other reason for dropping it, and presumably why some clients don't even bother supporting it. But if you do find it useful even so, we could add a feature flag for launch.json to enable this behavior.

[YuhongZeng]

I understand having the other threads running ended up incredibly confusing in most scenarios. I believe that this confusion is mainly caused by the vendored PyDev.Debugger. Here are some reasons we think this is happening.

[YuhongZeng]

1. Breakpoint hits cause all threads to be suspended

• When setting breakpoints, pydevd forcibly set suspend policy to ALL.
  https://github.com/microsoft/debugpy/blob/ef9a67fe150179ee4df9997f9273723c26687fab/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_process_net_command_json.py#L728C1-L813C81

• When suspend_policy = 'ALL' causes the suspend_other_threads parameter to be set to True.
  https://github.com/microsoft/debugpy/blob/ef9a67fe150179ee4df9997f9273723c26687fab/src/debugpy/_vendored/pydevd/pydevd.py#L1845C1-L1892C60

• The self.multi_threads_single_notification=True sets suspend_other_threads to True when suspend_policy ='NONE'. The multi_threads_single_notification`cannot be set to False. If it is set to False, Development Tools cannot accept debugging signals from pydevd.

        if not suspend_other_threads and self.multi_threads_single_notification:
            # In the mode which gives a single notification when all threads are
            # stopped, stop all threads whenever a set_suspend is issued.
            suspend_other_threads = True

[YuhongZeng]

2. StoppedEvent and ContinuedEvent returned by pydevd to Development Tools
   https://github.com/microsoft/debugpy/blob/ef9a67fe150179ee4df9997f9273723c26687fab/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_net_command_factory_json.py#L347C1-L397C89

        body = pydevd_schema.StoppedEventBody(
            reason=stop_reason,
            description=exc_desc,
            threadId=thread_id,
            text=exc_name,
            allThreadsStopped=True,
            preserveFocusHint=preserve_focus_hint,
        )

body = ContinuedEventBody(threadId=thread_id, allThreadsContinued=True)

The allThreadsStopped and allThreadsContinued are forcibly set to True, which causes Development Tools to respond to continue and stop for all threads, regardless of whether continue and stop of the debugger work for all threads.

[YuhongZeng]

3. Step resumes all threads,the default value is true.Step also applies to all threads.
   https://github.com/microsoft/debugpy/blob/ef9a67fe150179ee4df9997f9273723c26687fab/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_process_net_command_json.py#L328C1-L331C87
   stepping_resumes_all_threads = args.get('steppingResumesAllThreads', True)

[YuhongZeng]

4. The debuppy forcibly sets the thread ID to * for the received pause request and continue request, which also continues and parses all threads.
   

   debugpy/src/debugpy/adapter/clients.py

   Lines 630 to 645 in ef9a67f

   	@message_handler
   	def pause_request(self, request):
   	request.arguments["threadId"] = "*"
   	return self.server.channel.delegate(request)
   	@message_handler
   	def continue_request(self, request):
   	request.arguments["threadId"] = "*"
   	try:
   	return self.server.channel.delegate(request)
   	except messaging.NoMoreMessages:
   	# pydevd can sometimes allow the debuggee to exit before the queued
   	# "continue" response gets sent. Thus, a failed "continue" response
   	# indicating that the server disconnected should be treated as success.
   	return {"allThreadsContinued": True}

[int19h]

The reason why step resumes all threads is because there are many cases where a thread is dependent on another thread doing something (e.g. releasing a lock), so resuming just the single thread would often translate to the debugged problem hanging with no clear indication of the problem and no good way to resolve it at that point (since we can't pause the thread again once it's inside a native call that is blocked).

As I recall now, originally pydevd worked as you described, only pausing/resuming a single thread in all circumstances. It was changed based on real world experience with that. Unfortunately it does seem like this isn't just a one-line change, so we'll have to treat it as an enhancement - which for practical purposes means that it'll likely only be available in the next major version of debugpy that is currently in progress.

[YuhongZeng]

We have added a configurations args multiThread, when multiThread is set to true, multi-thread debugging phenomenon is shown in the following video:

78.mp4

[YuhongZeng]

When multiThread is set to false, multi-thread debugging behaves as before, as shown in the following video:

82.mp4

[YuhongZeng]

We found that when multi-threaded debugging, pause will occasionally have bugs.the control thread suspend, but the vscode interface does not respond, this is because the thread sleeps when pause, if it exceeds 0.5s, it will think that the request has timed out, and then return failed to the IDE, but the thread will still suspend after sleeping. After using debugging another thread, it can be resumed after pausing the thread again.

83.mp4

[int19h]

I have tried to implement this cleanly in the new debug adapter, and found DAP itself to be underspecified in the presence of individual threads; no wonder that clients get into weird states with it. Between that and lack of VS support, we're not going to support this feature going forward until DAP spec makes it possible for clients to clearly indicate which behavior they expect, and specifies what happens in various corner cases (e.g. what if client requests all threads suspend, then resumes individual threads which spawn more threads).