Populate `/etc` for zones

[smklein]

• /etc/hosts
• /etc/resolv.conf - transfer information about nameservers to zones, rather than having zones infer information from the /48 subnet

[luqmana]

What's the split for things hardcoded into /etc/hosts vs resolved using internal DNS listed in /etc/resolv.conf?

[smklein]

Good question, I dunno.

[luqmana]

Ok, RFD 248 was the one I was thinking of and based on that it seems for the most part we'd want things to resolve via DNS.

[jclulow]

It is generally good UNIX hygiene to map whatever the system's name is (as seen via hostname or uname -n) to some IP interface on the local system via /etc/inet/hosts. This could just be ::1 and 127.0.0.1 though, at which point the file could be essentially static.

I would do everything else through DNS.

[leftwo]

From https://github.com/oxidecomputer/rfd/tree/0206/rfd/0206/#41-how-do-you-find-the-dns-servers
The DNS server I.P. can be determined/calculated using the address of the system/zone that needs it.

Do we want to put that into /etc/resolv.conf for a zone when it is created?

[smklein]

From https://github.com/oxidecomputer/rfd/tree/0206/rfd/0206/#41-how-do-you-find-the-dns-servers The DNS server I.P. can be determined/calculated using the address of the system/zone that needs it.

Do we want to put that into /etc/resolv.conf for a zone when it is created?

This seems like the right call - as far as I understand it, adding the resolv.conf server basically just short-cuts some of the manual address calculation that a zone would need to do, by providing the address of the nameserver up-front.

[jclulow]

I'm not sure of the ordering of startup here. It seems like it would be possible that at least some zones (and certainly the global zone) to start prior to our knowing what the local /48 actually is. We might need a service on each system (or, at least, on a subset of systems) that waits for the prefix to appear and then updates the file?

[citrus-it]

We could also make use of the network/dns/install SMF service for this if we wanted - configure the service with the values we want in the final /etc/resolv.conf then refresh/enable the service and let it populate things. For zones that start early (global and switch) this could be deferred until the underlay addresses are known.

The network/dns/client service itself is really just a marker as a pre-requisite to the name services milestone. I've opened oxidecomputer/helios-omicron-brand#5 to disable this by default in omicron zones, and then it can be enabled as part of configuring the resolver for any zones that do that. This stops the service from just remaining offline and showing up in the list of service faults.

[davepacheco]

Thanks @citrus-it for the pointer to network/dns/install. This is really convenient because we can put the configuration into the SMF profile that we're going to deliver even into otherwise self-assembling zones, rather than reaching into the zone at runtime (which is fraught with stuff like #3407, plus makes it hard to inject into the boot sequence). In #2954, I've updated Sled Agent to configure network/dns/install so that /etc/resolv.conf gets populated in CockroachDB zones. I expect we can do the same thing for other zones. This may depend on commonizing some of the stuff that's currently in the cockroachdb start method that configures the necessary networking objects and routes.

[davepacheco]

To elaborate on that last bit, veering slightly off-topic, here's a suggestion:

• Create smf/oxide-netconfig/{manifest.xml,start.sh} that does basically this part of the CockroachDB start script. This creates a well-defined, declarative interface in the form of a particular named property group on a particular SMF service. The start method would apply the requested config. This service should probably become a dependency of one of the network services (milestone/network seems too late but I'm not sure) so that it's always run before anything that depends on the network.
• Ship this service in most control plane zones.
• For these other control plane zones, have Sled Agent startup configure this service as well as network/dns/install the way it does for CockroachDB today.

The net result is that these control plane zones would have a uniform, safe way to cause networking and DNS to be configured.