Enable setting DNS options through global nerdctl config #4378

swagatbora90 · 2025-06-26T21:26:28Z

This PR adds support for configuring DNS settings globally through the nerdctl configuration file (nerdctl.toml), allowing users to set default DNS servers, search domains, and options that apply to all containers unless explicitly overridden by command-line flags.

This is for a usecase where I want all containers to use a specific dns server instead of system default. Currently, I have to use --dns flag with every container create/run commands. This functionality is available in docker which allows setting global dns config value in the daemon config file.

Testing

Added a end to end test TestDNSWithGlobalConfig() that checks 4 scenarios:

Global DNS settings used when no command-line options provided
Command-line DNS options override global config (precedence test)
Global DNS settings apply with host networking
Global DNS settings apply with none networking

apostasie · 2025-06-26T21:37:08Z

Hey @swagatbora90

Noting that:

docker run -ti --dns="" debian cat /etc/resolv.conf
nerdctl run -ti --dns="" debian cat /etc/resolv.conf

Different behaviors on this ^.
(also same difference on dns-search="")

Appreciate your PR is not about that, but maybe, if you wish, this PR could fix that discrepancy as well since it is touching that part?

apostasie · 2025-06-26T21:42:14Z

cmd/nerdctl/container/container_run_network_linux_test.go

+	nerdtest.Setup()
+
+	testCase := &test.Case{
+		Env: map[string]string{"NERDCTL_TOML": tomlPath},


Tigron has a helper for that:

Just add this to your testCase instead of the Env:

Config: test.WithConfig(nerdtest.NerdctlToml, configContent),

(also takes care of writing the file in a temp location for you, and make it easier to retrieve)

apostasie · 2025-06-26T21:51:47Z

cmd/nerdctl/container/container_run_network_linux_test.go

+			{
+				Description: "Global DNS settings are used when command line options are not provided",
+				Command: func(data test.Data, helpers test.Helpers) test.TestableCommand {
+					nerdctlTomlPath := os.Getenv("NERDCTL_TOML")


Is this going to work as you think?
os.Getenv points to global env, not the test / commands env.

If you use the config from above ^^^

helpers.Read(nerdtest.NerdctlToml)

Should give you what you want?

If I read it correctly, the Env var should be passed to all Subtest. I need all the Subtest to use the global env, so it does work.

However, your suggestion to use Config should work too. Let me change it like you suggested.

Yes, testCase.Env is propagated to subtests, but this is not the global os.Environ (otherwise, you would pollute other unrelated tests), so os.Getenv will not give you the test environment...

Not completely sure I was clear, so, in a shell:

do not rely on os.Getenv or os.Setenv - that will cross pollute other tests and is specifically frowned-upon

the testCase Env is private to the test, but it is inherited by subtests

every test has a custom nerdctl.toml, regardless of the fact you want one explicitly or not (this is to guarantee that we are isolated from a possible, pre-existing, host nerdctl.toml)

Okay, I see your point. I did check that the NERDCTL_TOML config values are correct when running the test. Anyways, I have changed to use the Config attribute like you suggested.

apostasie · 2025-06-26T21:53:46Z

cmd/nerdctl/container/container_run_network_linux_test.go

+					cmd := helpers.Command("run", "--rm", testutil.CommonImage, "cat", "/etc/resolv.conf")
+					return cmd
+				},
+				Expected: func(data test.Data, helpers test.Helpers) *test.Expected {


More compact, using the test.Expects helper and comparators instead of naked asserts (better debugging info):

Expected: test.Expects(expect.ExitCodeSuccess, nil, expect.All( expect.Contains("foo"), expect.Contains("bla"), )),

apostasie · 2025-06-26T22:03:39Z

@swagatbora90 left a few comments on tests.

Tigron testing has some (semblance of) documentation here: https://github.com/containerd/nerdctl/blob/main/docs/testing/tools.md

Very much work in progress, so, it is what it is... but then, am always happy to cosplay "testing-helper-AI-bot" until the doc and API are settled :-) (also, feedback on Tigron is always welcome)

swagatbora90 · 2025-06-26T22:38:44Z

Hey @swagatbora90

Noting that:
docker run -ti --dns="" debian cat /etc/resolv.conf
nerdctl run -ti --dns="" debian cat /etc/resolv.conf
Different behaviors on this ^. (also same difference on dns-search="")

Appreciate your PR is not about that, but maybe, if you wish, this PR could fix that discrepancy as well since it is touching that part?

Looks like there is no validation today on the DNS servers and domain names that are being passed. I can add a follow up PR to add validation, I would prefer to keep this PR only for adding the global config.

apostasie · 2025-06-26T22:40:09Z

Looks like there is no validation today on the DNS servers and domain names that are being passed. I can add a follow up PR to add validation, I would prefer to keep this PR only for adding the global config.

Fair.

apostasie · 2025-06-27T21:44:07Z

Windows failure is likely due to [CI]: TestNetworkInspect is flaky in windows #4322
I did workaround it for some tests ([CI] Add delay for windows network test #4360), but we probably want to do a full review all of network tests - or better, actually fix the issue, but I am not sure it is a nerdctl issue or an hcsshim one.
Rootless failure is yet another variant of content digest not found
TestComposeExec: [CI] TestComposeExec test is flaky #4380

@swagatbora90 re-run the CI? (eg: rebase or close/open)

apostasie · 2025-07-01T18:42:55Z

Failures:

soci (fixed upstream)
TestLogs (possibly a similar issue to the recent IO fixes that got merged)
2x the infamous content digest not found (the good news part on this one is that it now shows a lot more, which should make it easier to repro)
[Flaky] TestComposeExecWithIndex #4373 - somehow, ip addr show dev eth0 returns nothing - could be a case of failing to wait on task IO

All unrelated to this PR ^

apostasie · 2025-07-01T18:45:05Z

Note: #4392 just got in - curious is this is going to help TestComposeExecWithIndex

swagatbora90 · 2025-07-01T23:57:35Z

New set of failing tests:

TestIPFSCompNoBuild - Is this known?
TestKubeCommitSave
Almalinux TestPush/plain_http_with_localhost - I have seen this consistently failing across many test runs
Rootful canary TestImagePullSoci - This is known and currently being worked on.

apostasie · 2025-07-02T00:43:57Z

Number 4 yeah ack

Number 3 Content digest not found (which you have seen on TestPush) has a long history of pain.
In a shell: containerd does garbage collect layers it should not (either because nerdctl does not hold correctly on references, or because containerd GC has issues - very likely on convert, which is clearly wrong).

Number 2: #4106

Number 1: is one of these legacy tests - we should start by rewriting the test with the new test toolkit and ensure the test itself is not broken - that being said, IPFS overall is not stable IMHO, and our compose implementation (shelling out) is a serious problem - so, I would not expect miracles out of the test rewrite. Also, IPFS might involve number 3 as well.

apostasie · 2025-07-02T00:48:59Z

Content digest issue maybe starts here: #4062 (and then a bunch of linked issues)
We did fix a number of cases, but clearly there are more.
While there is still clearly a containerd issue on convert that is unfixed (PR upstream was not accepted), it seemed like we did have a workaround, but apparently not...

apostasie · 2025-07-02T00:52:30Z

@swagatbora90 generally speaking, here is the central collection point for all CI evil: #4120

apostasie · 2025-07-02T00:56:59Z

Correction: kube issue is #4364

Shubhranshu153 · 2025-07-02T07:31:14Z

@AkihiroSuda This one too if we can add to milestone 2.1.3.
Thanks

Signed-off-by: Swagat Bora <[email protected]>

swagatbora90 · 2025-07-02T23:05:09Z

The failed test does not appear to be related to the changes

=== Failed
=== FAIL: cmd/nerdctl/image TestPush/with_hosts_dir,_with_login (0.87s)
    image_push_linux_test.go:282:

and is related to #4062

AkihiroSuda

Thanks

apostasie reviewed Jun 26, 2025

View reviewed changes

apostasie mentioned this pull request Jun 26, 2025

--dns* flags are inconsistent with docker #4379

Closed

swagatbora90 force-pushed the add-global-dns-config branch 3 times, most recently from c5bba54 to 1028d42 Compare June 26, 2025 23:14

swagatbora90 mentioned this pull request Jun 27, 2025

[CI] TestComposeExec test is flaky #4380

Closed

swagatbora90 force-pushed the add-global-dns-config branch from 1028d42 to 1d93303 Compare June 30, 2025 04:41

Shubhranshu153 closed this Jun 30, 2025

Shubhranshu153 reopened this Jun 30, 2025

swagatbora90 closed this Jul 1, 2025

swagatbora90 reopened this Jul 1, 2025

swagatbora90 force-pushed the add-global-dns-config branch from 1d93303 to 1cb05ec Compare July 1, 2025 17:25

apostasie approved these changes Jul 1, 2025

View reviewed changes

swagatbora90 closed this Jul 1, 2025

swagatbora90 reopened this Jul 1, 2025

swagatbora90 force-pushed the add-global-dns-config branch from 1cb05ec to 2bc1aa9 Compare July 1, 2025 22:47

Shubhranshu153 approved these changes Jul 2, 2025

View reviewed changes

Shubhranshu153 closed this Jul 2, 2025

Shubhranshu153 reopened this Jul 2, 2025

AkihiroSuda added this to the v2.1.3 milestone Jul 2, 2025

Enable setting DNS options through global nerdctl config

1ac8bb4

Signed-off-by: Swagat Bora <[email protected]>

swagatbora90 force-pushed the add-global-dns-config branch from 2bc1aa9 to 1ac8bb4 Compare July 2, 2025 16:48

swagatbora90 closed this Jul 2, 2025

swagatbora90 reopened this Jul 2, 2025

AkihiroSuda approved these changes Jul 3, 2025

View reviewed changes

AkihiroSuda merged commit 14a59a2 into containerd:main Jul 3, 2025
110 of 116 checks passed

github-actions bot mentioned this pull request Jul 13, 2025

Bump nerdctl from 1.7.6 to 2.1.3 kokyhm/kubespray#244

Open

Enable setting DNS options through global nerdctl config #4378

Enable setting DNS options through global nerdctl config #4378

Uh oh!

Conversation

swagatbora90 commented Jun 26, 2025

Testing

Uh oh!

apostasie commented Jun 26, 2025

Uh oh!

apostasie Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apostasie Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

swagatbora90 Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

apostasie Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

apostasie Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swagatbora90 Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

apostasie Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apostasie commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swagatbora90 commented Jun 26, 2025

Uh oh!

apostasie commented Jun 26, 2025

Uh oh!

apostasie commented Jun 27, 2025

Uh oh!

apostasie commented Jul 1, 2025

Uh oh!

apostasie commented Jul 1, 2025

Uh oh!

swagatbora90 commented Jul 1, 2025

Uh oh!

apostasie commented Jul 2, 2025

Uh oh!

apostasie commented Jul 2, 2025

Uh oh!

apostasie commented Jul 2, 2025

Uh oh!

apostasie commented Jul 2, 2025

Uh oh!

Shubhranshu153 commented Jul 2, 2025

Uh oh!

swagatbora90 commented Jul 2, 2025

Uh oh!

AkihiroSuda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

apostasie Jun 26, 2025 •

edited

Loading

apostasie Jun 26, 2025 •

edited

Loading

apostasie Jun 26, 2025 •

edited

Loading

apostasie commented Jun 26, 2025 •

edited

Loading