Connecting to LSP with docker-in-docker setups.

[paul]

Description

We've got a Rails app with some expensive and complicated dependencies (libarrow, among other things), so we run the app in docker-compose. Additionally, for most of our devs using macs, we've started using gitpod to develop the app. They run their own gidpod container on their own infrastructure, and expose the docker socket to that container, and we can docker-compose our app from there. Then vscode can connect to the gitpod docker infrastructure and interact with the files, etc...

I'm trying to figure out how to convince the vscode that connects to the gitpod container to run the ruby-lsp command within our dev container. I can write a wrapper script that would do something like docker compose run -it dev ruby_lsp, but I'm not sure how to get the extension to run my wrapper. Also, its gonna try to boot rails, so I'd need to wrap that command, too.

Not a bug report exactly, but maybe a feature request, to allow for a setting in the vscode extension that would let me override the command used to invoke ruby_lsp. (and another to control how it boots the rails app). Alternatively, there's other issues on here to expose a socket or port for vscode to connect to, so that might work if we can start ruby-lsp independently as part of docker-compose, and configure vscode to connect there.

Reproduction steps

1. Have an app with a complicated set of dependencies, such that it can't bundle install or boot on a local machine or in a "normal" container, only within a purpose-built container with the deps needed.
2. Get vscode to run ruby_lsp inside that container.

[willtcarey]

We've got a similar setup to what you're describing, except we remote into linux machines from our macOS machines (using the VS Code Remote SSH extensino) and then run our app code in docker.

Ruby LSP appears to have support for everything running inside of a devcontainer, but not this case where you are running VS Code outside of the container which is running your app code/LSP instance.

I managed to make the VS Code extension run my custom wrapper script by running the Ruby LSP: Select ruby version manager -> Change manual ruby configuration then picking my wrapper script. However, it's a bit odd that the extension only stores that in a workspaceState rather than a configuration option. I'd like to share that configuration with my team, but I don't know a way to do that with workspaceState.

ruby-lsp/vscode/src/ruby.ts

Lines 106 to 136 in caa4a40

	const workspaceRubyPath = this.context.workspaceState.get<
	string | undefined
	>(`rubyLsp.workspaceRubyPath.${this.workspaceFolder.name}`);
	if (workspaceRubyPath) {
	// If a workspace specific Ruby path is configured, then we use that to activate the environment
	await this.runActivation(
	new None(this.workspaceFolder, this.outputChannel, workspaceRubyPath),
	);
	} else {
	// If the version manager is auto, discover the actual manager before trying to activate anything
	if (this.versionManager.identifier === ManagerIdentifier.Auto) {
	await this.discoverVersionManager();
	this.outputChannel.info(
	`Discovered version manager ${this.versionManager.identifier}`,
	);
	}
	try {
	await this.runManagerActivation();
	} catch (error: any) {
	// If an error occurred and a global Ruby path is configured, then we can try to fallback to that
	const globalRubyPath = vscode.workspace
	.getConfiguration("rubyLsp")
	.get<string | undefined>("rubyExecutablePath");
	if (globalRubyPath) {
	await this.runActivation(
	new None(this.workspaceFolder, this.outputChannel, globalRubyPath),
	);
	} else {

The second thing that's odd here is that I could avoid going the workspaceState route at all if I could set my version manager to none and set a custom rubyExecutablePath. But the custom ruby path is not passed to the none version manager on the first pass.

ruby-lsp/vscode/src/ruby.ts

Lines 293 to 297 in caa4a40

	case ManagerIdentifier.None:
	await this.runActivation(
	new None(this.workspaceFolder, this.outputChannel),
	);
	break;

So that means that the extension attempts to execute ruby on my machine outside of docker. I have a system installed ruby so the command works and the extension tries to go down that activation path before erroring out because that's not the correct ruby it should be using. I think that we should always pass the custom rubyExecutablePath if it is set for consistency.

Ok, so while I think some of the above things should be fixed, I've managed to work around them by using a custom version manager which ran a script to add some custom ruby, gem and bundler shims to my path which executed the respective command inside my docker container. This seems like a temporary hack, but at least it got me to this point. When I did that, however, I got errors because my Dockerfile adds LD_PRELOAD environment variables for jemalloc, and the extension was trying to add that to my shell environment when executing other commands, but that jemalloc library doesn't exist on the host machine, so it immediately errors. Maybe we could configure what env params get sanitized during env sanitization process?

So it appears there are several layers to this and several different locations where I need to tell ruby-lsp that I want to run it inside my container. I believe this should technically work as the LSP protocol runs through plain text communication on STDOUT (please correct me if I'm wrong there). But I'm definitely running into several walls. I'm not sure the best way to approach this as it looks like it would take changes in several places, but I wanted to leave my experience of where I got to.

[butsjoh]

I have a different use case but i think i boils down to the same issue. We have a monorepo with multiple projects (some of them are written in ruby). We run the projects through docker-compose. We have vscode locally installed with the ruby-lsp extension and currently we can only get things to work if we locally install ruby. It would be nice if we could use the ruby that is already available in the docker containers.

[vinistock]

For simpler docker setups, VS Code already supports connecting the editor directly to the container, which makes everything in the editor works, including the integrated terminal. We have docs for this use case https://github.com/Shopify/ruby-lsp/tree/main/vscode#developing-on-containers.

For more complex setups, like the one mentioned, I believe it ultimately boils down to the LSP supporting connecting through sockets #1782. The LSP has to run inside wherever Ruby and your dependencies are installed and where the code lives - since it needs to read it all from disk to index the declarations.

If someone is interested in taking a stab at this, my understanding is that the server needs to support CLI arguments to connect via ports. I'm not exactly sure if the language client package still owns the life cycle of the process in this scenario or if there's some custom code that needs to exist to make it work. It might be worth checking how language server implementations for other languages support this.

[willtcarey]

@vinistock I'm not sure that any of these setups require the LSP to support socket communication. From what I can tell, each one of us can run a command which would start up ruby-lsp, but the Ruby LSP extension appears to only support starting the LSP by calling the ruby-lsp command.

Some other LSPs that I've looked into do allow you to customize the commands that are run to start the LSP

• Sorbet https://github.com/sorbet/sorbet/blob/16f043e37b92b5de0b1b9d228826ac2230b38153/vscode_extension/src/sorbetLspConfig.ts#L60
• StandardRB (which has been deprecated in favor of a Ruby LSP add-on) https://github.com/standardrb/vscode-standard-ruby/blob/e898ee80d90dc6a0526a7d6428d287c254aa06b2/src/extension.ts#L254
• Python https://github.com/palantir/python-language-server/blob/2982b42644282c5be54d3cfa7f5c3bd8d85c4ba2/vscode-client/src/extension.ts#L46

And there are instances where the command is not customizable, but the path is overridable

• Elixir https://github.com/elixir-lsp/vscode-elixir-ls/blob/14190e87475ef2b12386104cc7ac4323cbcf9107/src/executable.ts#L17

I started down the path of adding a manual adapter to customize path/env variables. But I wonder if it should simply be extended to support a manual adapter that allows you to override the lsp command. That would let the language server package still own the process and give the user quite a bit of leeway with their setup

[vinistock]

I'm not opposed to allowing the command to be configurable, but I have some concerns:

1. First, we need to ensure not only that we are able to spawn the LSP in the right place, but that communication is working properly. My concern is that depending on how the command is written the pipes used for communication may not be set up as needed. If someone is willing to test that, our CONTRIBUTING guide has instructions on how to verify changes locally
2. The second part is helping to ensure that people aren't configuring the command incorrectly. The Ruby LSP has to be launched with ruby-lsp (and not bundle exec ruby-lsp) so that we set up the composed bundle. I'm not sure how we would be able to do this since people could be pointing the executable to arbitrary commands. If you have ideas on how to help guide users, then please let us know
3. We also support pointing to a custom bundle in order to launch the LSP, which essentially just runs bundle exec ruby-lsp skipping the composed bundle and pointing to the one configured by the user. A custom command will add another dimension to this matrix and we need to ensure that it works for custom bundle paths
4. Finally, we're going to need to document this use case so that others can use similar techniques without reinventing the wheel

[github-actions]

This issue is being marked as stale because there was no activity in the last 2 months

[github-actions]

This issue is being marked as stale because there was no activity in the last 2 months