DSC Settings and Policy should be more strongly typed

[michaeltlombardi]

Summary of the new feature / enhancement

As a maintainer of DSC,
I want to have strongly defined types for the available settings/policy,
so that I can reason about the structure of DSC settings without needing to search through the code and find keys/values to explore how they're used.

As a user and integrating developer,
I want to be able to reference a JSON Schema for DSC settings/policy,
so that I can validate that data and safely use/integrate with it.

Proposed technical implementation details (optional)

Currently, the implementation uses the get_setting function, which returns a result object where the inner data is a struct containing two optional arbitrary JSON values - one for the definition in settings, one for the definition in policy:

DSC/dsc_lib/src/util.rs

Line 110 in 728b529

pub fn get_setting(value_name: &str) -> Result<DscSettingValue, DscError> {

DSC/dsc_lib/src/util.rs

Lines 15 to 18 in 728b529

	pub struct DscSettingValue {
	pub setting: Value,
	pub policy: Value,
	}

We use this setting in two places:

• We retrieve resourcePath during command discovery:

  DSC/dsc_lib/src/discovery/command_discovery.rs

  Lines 88 to 110 in 728b529

  	fn get_resource_path_setting() -> Result<ResourcePathSetting, DscError>
  	{
  	if let Ok(v) = get_setting("resourcePath") {
  	// if there is a policy value defined - use it; otherwise use setting value
  	if v.policy != serde_json::Value::Null {
  	match serde_json::from_value::<ResourcePathSetting>(v.policy) {
  	Ok(v) => {
  	return Ok(v);
  	},
  	Err(e) => { return Err(DscError::Setting(format!("{e}"))); }
  	}
  	} else if v.setting != serde_json::Value::Null {
  	match serde_json::from_value::<ResourcePathSetting>(v.setting) {
  	Ok(v) => {
  	return Ok(v);
  	},
  	Err(e) => { return Err(DscError::Setting(format!("{e}"))); }
  	}
  	}
  	}
  	Err(DscError::Setting(t!("discovery.commandDiscovery.couldNotReadSetting").to_string()))
  	}

  Which we then deserialize as a JSON value into the ResourcePathSetting struct defined in the same file:

  DSC/dsc_lib/src/discovery/command_discovery.rs

  Lines 49 to 59 in 728b529

  	#[derive(Deserialize)]
  	pub struct ResourcePathSetting {
  	/// whether to allow overriding with the `DSC_RESOURCE_PATH` environment variable
  	#[serde(rename = "allowEnvOverride")]
  	allow_env_override: bool,
  	/// whether to append the PATH environment variable to the list of resource directories
  	#[serde(rename = "appendEnvPath")]
  	append_env_path: bool,
  	/// array of directories that DSC should search for non-built-in resources
  	directories: Vec<String>
  	}

• We retrieve the tracing setting in the enable_tracing() function in the DSC CLI itself:

  DSC/dsc/src/util.rs

  Lines 321 to 340 in 728b529

  	if let Ok(v) = get_setting("tracing") {
  	if v.policy != serde_json::Value::Null {
  	match serde_json::from_value::<TracingSetting>(v.policy) {
  	Ok(v) => {
  	tracing_setting = v;
  	policy_is_used = true;
  	},
  	Err(e) => { error!("{e}"); }
  	}
  	} else if v.setting != serde_json::Value::Null {
  	match serde_json::from_value::<TracingSetting>(v.setting) {
  	Ok(v) => {
  	tracing_setting = v;
  	},
  	Err(e) => { error!("{e}"); }
  	}
  	}
  	} else {
  	error!("{}", t!("util.failedToReadTracingSetting"));
  	}

  Where we again deserialize the JSON value into a struct defined in the same file, in this case TracingSetting:

  DSC/dsc/src/util.rs

  Lines 79 to 87 in 728b529

  	pub struct TracingSetting {
  	/// Trace level to use - see pub enum `TraceLevel` in `dsc_lib\src\dscresources\command_resource.rs`
  	level: TraceLevel,
  	/// Trace format to use - see pub enum `TraceFormat` in `dsc\src\args.rs`
  	format: TraceFormat,
  	/// Whether the 'level' can be overrridden by `DSC_TRACE_LEVEL` environment variable
  	#[serde(rename = "allowOverride")]
  	allow_override: bool
  	}

We don't perform any caching and we don't have a way to retrieve every setting once. We don't derive serialization or JSON Schema for these settings and they're not easily discoverable without following the flow of code that uses the get_setting() function.

Instead, we should define a new module for dsc_lib where we centrally define settings/policy. Ideally, we should be able to parse the policy and settings once during an invocation and quickly be able to retrieve the relevant settings/policy from the resolved struct (which is separate from the data a user specifies, like how we separate the manifests from resource/extension info).

The current representation of available settings/policy would look something like:

#[derive(Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "camelCase")]
pub struct DefinedMetaConfiguration {
    pub tracing: Option<TracingSetting>,
    pub resource_path: Option<ResourcePathSetting>,
}

#[derive(Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "camelCase")]
pub struct TracingSetting {
    /// Trace level to use - see pub enum `TraceLevel` in `dsc_lib\src\dscresources\command_resource.rs`
    pub level:  TraceLevel,
    /// Trace format to use - see pub enum `TraceFormat` in `dsc\src\args.rs`
    pub format: TraceFormat,
    /// Whether the 'level' can be overrridden by `DSC_TRACE_LEVEL` environment variable
    pub allow_env_override: bool
}

#[derive(Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "camelCase")]
pub struct ResourcePathSetting {
    /// whether to allow overriding with the `DSC_RESOURCE_PATH` environment variable
    pub allow_env_override: bool,
    /// whether to append the PATH environment variable to the list of resource directories
    pub append_env_path: bool,
    /// array of directories that DSC should search for non-built-in resources
    pub directories: Vec<String>
}

While we could probably just cache two instances of DefinedMetaConfiguration in the context - one for policy, one for settings - and resolve things that way, it would probably better to have a longer discussion about designing the meta configuration for maintainability. Currently we only consider meta configuration from policy, settings, and environment variables. We probably also want to consider things like:

• Cases where definable fields in policy and settings are different (does it make sense to have some meta configuration field only at the policy level?)
• Cases where we want to merge meta configuration as opposed to replace it (e.g. if I define a block list at the policy level, I should be able to add to it in settings but not allow an item blocked at the policy level).
• Being able to define meta configuration in a document (override anything but policy).
• Strongly associating environment variables with meta configuration
• Showing the resolved meta configuration from policy/settings/environment variables.
• Performance and caching - can we cache the resolved settings to disk and see whether we actually need to re-process policy or settings during an invocation? Can we pass resolved settings to a nested invocation?
• Should we ever enable some subset of settings to be modified on the per-resource level (e.g. I want to get tracing for this resource, not the whole run)?
• If we have schemas for various settings, we can probably define resources to set the policy/settings for a machine - but should we have any special handling for updating the originally-resolved context afterward, or clearly indicate they only apply on subsequent executions?
• Which settings, if any, should be surfaced in results/diagnostics? Can we usefully indicate meta configuration to callers for debugging/investigation purposes without requiring the use of a second command?
• Should we have a command that emits the resolved meta configuration at the terminal?