From 51fa5d5cf70b2a00634f27f634bb91669360545e Mon Sep 17 00:00:00 2001 From: Dave Armstrong Date: Tue, 5 Mar 2019 15:44:19 +0000 Subject: [PATCH 1/8] (FM-7819) - update readme to include transports This commit updates the readme to mention the transport class and gives an example of a pass through on the device class. --- README.md | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index a389a6ab..fa5a8e67 100644 --- a/README.md +++ b/README.md @@ -172,19 +172,38 @@ The `create`/`update`/`delete` methods get called by the `SimpleProvider` base-c The generated unit tests in `spec/unit/puppet/provider/foo_spec.rb` get automatically evaluated with `pdk test unit`. +## Remote Resources + +Support for remote resources is enabled through the use of a `transport`. A transport class conatins the code for managing connections and processing information to/from the remote resource. Please see the [RSAPI specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport) document on how to create a transport. + ### `puppet device` support -To support remote resources using `puppet device`, a few more steps are needed. First a `Puppet::Util::NetworkDevice::::Device` class needs to exist, which provides facts and connection management . That device class can inherit from `Puppet::Util::NetworkDevice::Simple::Device` to receive a simple default configuration parser using hocon. +To connect to a remote resource through `puppet device` a `transport` must be called through a device shim. -The provider needs to specify the `remote_resource` feature to enable the second part of the machinery. +```ruby +# lib/puppet/util/network_device/remote_thing.rb + +require 'puppet' +require 'puppet/resource_api/transport/wrapper' +# force registering the transport +require 'puppet/transport/schema/remote_thing' + +module Puppet::Util::NetworkDevice::RemoteThing + class Device < Puppet::ResourceApi::Transport::Wrapper + def initialize(url_or_config, _options = {}) + super('remote_thing', url_or_config) + end + end +end +``` After this, `puppet device` will be able to use the new provider, and supply it (through the device class) with the URL specified in the [`device.conf`](https://puppet.com/docs/puppet/5.3/config_file_device.html). -#### Device-specific providers +#### Transport/Device specific providers -To allow modules to deal with different backends independently of each other, the Resource API also implements a mechanism to use different API providers side-by-side. For a given device type (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::DeviceType` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. +To allow modules to deal with different backends independently of each other, the Resource API also implements a mechanism to use different API providers side-by-side. For a given transport/device (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::DeviceType` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. -### Further Reading +## Further Reading The [Resource API](https://github.com/puppetlabs/puppet-specifications/blob/master/language/resource-api/README.md) describes details of all the capabilities of this gem. From e081b935706392169dc4cb873293af8dada229c3 Mon Sep 17 00:00:00 2001 From: Will Meek Date: Tue, 5 Mar 2019 17:12:36 +0000 Subject: [PATCH 2/8] (FM-7819) Readme updates with transport class example This commit updates the readme with examples of the transport class and an example transport schema. --- README.md | 79 +++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 71 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index fa5a8e67..3869c7d3 100644 --- a/README.md +++ b/README.md @@ -174,34 +174,97 @@ The generated unit tests in `spec/unit/puppet/provider/foo_spec.rb` get automati ## Remote Resources -Support for remote resources is enabled through the use of a `transport`. A transport class conatins the code for managing connections and processing information to/from the remote resource. Please see the [RSAPI specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport) document on how to create a transport. +Support for remote resources is enabled through the use of a `transport` class. A transport class contains the code for managing connections and processing information to/from the remote resource. Please see the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport) document on how to create a transport class. ### `puppet device` support -To connect to a remote resource through `puppet device` a `transport` must be called through a device shim. +To connect to a remote resource through `puppet device` a `transport` class must be called through a device shim. + +For example, the `device` class will be a pass through to `transport`: ```ruby -# lib/puppet/util/network_device/remote_thing.rb +# lib/puppet/util/network_device/device_type/device.rb require 'puppet' require 'puppet/resource_api/transport/wrapper' -# force registering the transport -require 'puppet/transport/schema/remote_thing' +# force registering the transport schema +require 'puppet/transport/schema/device_type' -module Puppet::Util::NetworkDevice::RemoteThing +module Puppet::Util::NetworkDevice::DeviceType class Device < Puppet::ResourceApi::Transport::Wrapper def initialize(url_or_config, _options = {}) - super('remote_thing', url_or_config) + super('device_type', url_or_config) + end + end +end +``` + +This requires a `transport` class and schema, as detailed in the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport), for example a transport class: + +```ruby +# lib/puppet/transport/device_type.rb +module Puppet::Transport + # The main connection class to a PAN-OS API endpoint + class DeviceType + def initialize(context, connection_info) + # Initialization eg. validate connection_info + end + + def verify(context) + # Test that transport can talk to the remote target + end + + def facts(context) + # Access target, return a Facter facts hash + end + + def close(context) + # Close connection, free up resources end end end ``` +An example of a corresponding schema may look like: + +```ruby +# lib/puppet/transport/device_type.rb +Puppet::ResourceAPI.register_transport( + name: 'device_type', # points at class Puppet::Transport::DeviceType + desc: 'Connects to a device_type', + # features: [], # future extension points + connection_info: { + hostname: { + type: 'String', + desc: 'The host to connect to.', + }, + username: { + type: 'String', + desc: 'The user.', + }, + password: { + type: 'String', + sensitive: true, + desc: 'The password to connect.', + }, + enable_password: { + type: 'String', + sensitive: true, + desc: 'The password escalate to enable access.', + }, + port: { + type: 'Integer', + desc: 'The port to connect to.', + }, + }, +) +``` + After this, `puppet device` will be able to use the new provider, and supply it (through the device class) with the URL specified in the [`device.conf`](https://puppet.com/docs/puppet/5.3/config_file_device.html). #### Transport/Device specific providers -To allow modules to deal with different backends independently of each other, the Resource API also implements a mechanism to use different API providers side-by-side. For a given transport/device (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::DeviceType` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. +To allow modules to deal with different backends independently of each other, the Resource API also implements a mechanism to use different API providers side-by-side. For a given transport/device class (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::DeviceType` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. ## Further Reading From 364a8459c78776d5aded2b968777a4101914f01f Mon Sep 17 00:00:00 2001 From: clairecadman Date: Thu, 7 Mar 2019 12:21:58 +0000 Subject: [PATCH 3/8] (puppet-resource_api) Docs edit This commit makes minor edits to the puppet-resource_api docs. --- README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 3869c7d3..09727258 100644 --- a/README.md +++ b/README.md @@ -172,15 +172,13 @@ The `create`/`update`/`delete` methods get called by the `SimpleProvider` base-c The generated unit tests in `spec/unit/puppet/provider/foo_spec.rb` get automatically evaluated with `pdk test unit`. -## Remote Resources +## Remote resources -Support for remote resources is enabled through the use of a `transport` class. A transport class contains the code for managing connections and processing information to/from the remote resource. Please see the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport) document on how to create a transport class. +Support for remote resources is enabled through a `transport` class. A transport class contains the code for managing connections and processing information to and from the remote resource. For information on how to create a transport class, see the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport). ### `puppet device` support -To connect to a remote resource through `puppet device` a `transport` class must be called through a device shim. - -For example, the `device` class will be a pass through to `transport`: +To connect to a remote resource through `puppet device`, you must call a `transport` class through a device shim. For example, the `device` class will be passed through to `transport`: ```ruby # lib/puppet/util/network_device/device_type/device.rb @@ -199,7 +197,9 @@ module Puppet::Util::NetworkDevice::DeviceType end ``` -This requires a `transport` class and schema, as detailed in the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport), for example a transport class: +This requires a `transport` class and schema, as detailed in the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport). + +An example of a transport class: ```ruby # lib/puppet/transport/device_type.rb @@ -225,7 +225,7 @@ module Puppet::Transport end ``` -An example of a corresponding schema may look like: +An example of a corresponding schema: ```ruby # lib/puppet/transport/device_type.rb @@ -262,11 +262,11 @@ Puppet::ResourceAPI.register_transport( After this, `puppet device` will be able to use the new provider, and supply it (through the device class) with the URL specified in the [`device.conf`](https://puppet.com/docs/puppet/5.3/config_file_device.html). -#### Transport/Device specific providers +#### Transport/device specific providers -To allow modules to deal with different backends independently of each other, the Resource API also implements a mechanism to use different API providers side-by-side. For a given transport/device class (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::DeviceType` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. +To allow modules to deal with different backends independently, the Resource API implements a mechanism to use different API providers side by side. For a given transport/device class (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::DeviceType` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. -## Further Reading +## Further reading The [Resource API](https://github.com/puppetlabs/puppet-specifications/blob/master/language/resource-api/README.md) describes details of all the capabilities of this gem. From 728d8f8c1ac695368160664b468decb95dc24a15 Mon Sep 17 00:00:00 2001 From: Will Meek Date: Thu, 7 Mar 2019 15:33:00 +0000 Subject: [PATCH 4/8] (FM-7819) Clarify device shim and difference between device and transport classes --- README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 09727258..f3fa5377 100644 --- a/README.md +++ b/README.md @@ -178,7 +178,15 @@ Support for remote resources is enabled through a `transport` class. A transport ### `puppet device` support -To connect to a remote resource through `puppet device`, you must call a `transport` class through a device shim. For example, the `device` class will be passed through to `transport`: +To connect to a remote resource through `puppet device`, you must call a `transport` class through a device shim to maintain compatibility with Puppet Resource. + +A `device` class is created that Puppet Resource will call, however this will inherit from the `transport` class which will contain the following methods (as detailed in the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport)): +* `initialize` +* `verify` +* `facts` +* `connect` + +For example, the `device` class will be a pass through to `transport`: ```ruby # lib/puppet/util/network_device/device_type/device.rb From 7369c9d1c577d0b8dcf747987fa3263fe6a19dcb Mon Sep 17 00:00:00 2001 From: Will Meek Date: Fri, 8 Mar 2019 10:46:26 +0000 Subject: [PATCH 5/8] (FM-7819) Detail known Bolt keywords Detail the keywords currently known from Bolt --- README.md | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f3fa5377..86d9524b 100644 --- a/README.md +++ b/README.md @@ -268,7 +268,34 @@ Puppet::ResourceAPI.register_transport( ) ``` -After this, `puppet device` will be able to use the new provider, and supply it (through the device class) with the URL specified in the [`device.conf`](https://puppet.com/docs/puppet/5.3/config_file_device.html). +##### Transport Schema keywords + +Please note that within the transport schema, the following keywords are reserved words: + +###### Usable within the schema + +The following keywords are encouraged within the Transport schema: + +* `uri` - Use when you need to specify a specific URL to connect to. All of the following keys will be computed from the `uri` if possible. In the future more url parts may be computed from the URI as well. +* `host` - Use to specify and IP or address to connect to. +* `protocol` - Use to specify which protocol the transport should use for example `http`, `https`, `ssh` or `tcp` +* `user` - The user the transport should connect as. +* `port` - The port the transport should connect to. + +###### Non-Usable within the schema + +The following keywords are keywords that must not be used by the transport schema: + +* `name` - transports should use `uri` instead of name. +* `path` +* `query` +* `run-on` - This is used by bolt to determine which target to proxy to. Transports should not rely on this key. +* `remote-transport` - This is used to determine which transport to load. It should always be the transport class name "declassified". +* `remote-*` Any key starting with `remote-` is reserved for future use. + +Note: Currently bolt inventory requires that a name be set for every target and always uses that name as the URI. This means there is no way to specify `host` separately from the host section of the `name` when parsed as a URI. + +After the device class, transport class and transport schema have been implemented, `puppet device` will be able to use the new provider, and supply it (through the device class) with the URL specified in the [`device.conf`](https://puppet.com/docs/puppet/5.3/config_file_device.html). #### Transport/device specific providers From c53b20159f112b0cd4e5bd232acb21dfd3c3f646 Mon Sep 17 00:00:00 2001 From: Will Meek Date: Wed, 13 Mar 2019 10:14:29 +0000 Subject: [PATCH 6/8] (FM-7819) Updates on keywords --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 86d9524b..3f528355 100644 --- a/README.md +++ b/README.md @@ -242,11 +242,11 @@ Puppet::ResourceAPI.register_transport( desc: 'Connects to a device_type', # features: [], # future extension points connection_info: { - hostname: { + host: { type: 'String', desc: 'The host to connect to.', }, - username: { + user: { type: 'String', desc: 'The user.', }, @@ -287,8 +287,8 @@ The following keywords are encouraged within the Transport schema: The following keywords are keywords that must not be used by the transport schema: * `name` - transports should use `uri` instead of name. -* `path` -* `query` +* `path` - reserved as a uri part +* `query` - reserved as a uri part * `run-on` - This is used by bolt to determine which target to proxy to. Transports should not rely on this key. * `remote-transport` - This is used to determine which transport to load. It should always be the transport class name "declassified". * `remote-*` Any key starting with `remote-` is reserved for future use. @@ -299,7 +299,7 @@ After the device class, transport class and transport schema have been implement #### Transport/device specific providers -To allow modules to deal with different backends independently, the Resource API implements a mechanism to use different API providers side by side. For a given transport/device class (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::DeviceType` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. +To allow modules to deal with different backends independently, the Resource API implements a mechanism to use different API providers side by side. For a given transport/device class (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. ## Further reading From d1f4d8d9e0637fb10c503b989c94e3915b0735ab Mon Sep 17 00:00:00 2001 From: David Schmitt Date: Wed, 13 Mar 2019 13:51:55 +0000 Subject: [PATCH 7/8] (FM-7819) rearrange for better reading flow --- README.md | 88 +++++++++++++++++++++++++++---------------------------- 1 file changed, 43 insertions(+), 45 deletions(-) diff --git a/README.md b/README.md index 3f528355..8b84b91b 100644 --- a/README.md +++ b/README.md @@ -176,37 +176,6 @@ The generated unit tests in `spec/unit/puppet/provider/foo_spec.rb` get automati Support for remote resources is enabled through a `transport` class. A transport class contains the code for managing connections and processing information to and from the remote resource. For information on how to create a transport class, see the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport). -### `puppet device` support - -To connect to a remote resource through `puppet device`, you must call a `transport` class through a device shim to maintain compatibility with Puppet Resource. - -A `device` class is created that Puppet Resource will call, however this will inherit from the `transport` class which will contain the following methods (as detailed in the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport)): -* `initialize` -* `verify` -* `facts` -* `connect` - -For example, the `device` class will be a pass through to `transport`: - -```ruby -# lib/puppet/util/network_device/device_type/device.rb - -require 'puppet' -require 'puppet/resource_api/transport/wrapper' -# force registering the transport schema -require 'puppet/transport/schema/device_type' - -module Puppet::Util::NetworkDevice::DeviceType - class Device < Puppet::ResourceApi::Transport::Wrapper - def initialize(url_or_config, _options = {}) - super('device_type', url_or_config) - end - end -end -``` - -This requires a `transport` class and schema, as detailed in the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport). - An example of a transport class: ```ruby @@ -215,17 +184,17 @@ module Puppet::Transport # The main connection class to a PAN-OS API endpoint class DeviceType def initialize(context, connection_info) - # Initialization eg. validate connection_info + # Initialization eg. validate connection_info end - + def verify(context) # Test that transport can talk to the remote target end - + def facts(context) # Access target, return a Facter facts hash end - + def close(context) # Close connection, free up resources end @@ -268,31 +237,31 @@ Puppet::ResourceAPI.register_transport( ) ``` -##### Transport Schema keywords +### Transport Schema keywords Please note that within the transport schema, the following keywords are reserved words: -###### Usable within the schema - +#### Usable within the schema + The following keywords are encouraged within the Transport schema: - + * `uri` - Use when you need to specify a specific URL to connect to. All of the following keys will be computed from the `uri` if possible. In the future more url parts may be computed from the URI as well. -* `host` - Use to specify and IP or address to connect to. +* `host` - Use to specify and IP or address to connect to. * `protocol` - Use to specify which protocol the transport should use for example `http`, `https`, `ssh` or `tcp` * `user` - The user the transport should connect as. * `port` - The port the transport should connect to. - -###### Non-Usable within the schema - + +#### Non-Usable within the schema + The following keywords are keywords that must not be used by the transport schema: - + * `name` - transports should use `uri` instead of name. * `path` - reserved as a uri part * `query` - reserved as a uri part * `run-on` - This is used by bolt to determine which target to proxy to. Transports should not rely on this key. * `remote-transport` - This is used to determine which transport to load. It should always be the transport class name "declassified". * `remote-*` Any key starting with `remote-` is reserved for future use. - + Note: Currently bolt inventory requires that a name be set for every target and always uses that name as the URI. This means there is no way to specify `host` separately from the host section of the `name` when parsed as a URI. After the device class, transport class and transport schema have been implemented, `puppet device` will be able to use the new provider, and supply it (through the device class) with the URL specified in the [`device.conf`](https://puppet.com/docs/puppet/5.3/config_file_device.html). @@ -301,6 +270,35 @@ After the device class, transport class and transport schema have been implement To allow modules to deal with different backends independently, the Resource API implements a mechanism to use different API providers side by side. For a given transport/device class (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. +### `puppet device` support + +To connect to a remote resource through `puppet device`, you must call a `transport` class through a device shim to maintain compatibility with Puppet Resource. + +A `device` class is created that Puppet Resource will call, however this will inherit from the `transport` class which will contain the following methods (as detailed in the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport)): +* `initialize` +* `verify` +* `facts` +* `connect` + +For example, the `device` class will be a pass through to `transport`: + +```ruby +# lib/puppet/util/network_device/device_type/device.rb + +require 'puppet' +require 'puppet/resource_api/transport/wrapper' +# force registering the transport schema +require 'puppet/transport/schema/device_type' + +module Puppet::Util::NetworkDevice::DeviceType + class Device < Puppet::ResourceApi::Transport::Wrapper + def initialize(url_or_config, _options = {}) + super('device_type', url_or_config) + end + end +end +``` + ## Further reading The [Resource API](https://github.com/puppetlabs/puppet-specifications/blob/master/language/resource-api/README.md) describes details of all the capabilities of this gem. From aae6604029ab4dbf54427be7601dad3b44085e96 Mon Sep 17 00:00:00 2001 From: David Schmitt Date: Wed, 13 Mar 2019 15:31:58 +0000 Subject: [PATCH 8/8] (FM-7819) clarify `puppet device` compatibility --- README.md | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 8b84b91b..f2c8329a 100644 --- a/README.md +++ b/README.md @@ -270,17 +270,11 @@ After the device class, transport class and transport schema have been implement To allow modules to deal with different backends independently, the Resource API implements a mechanism to use different API providers side by side. For a given transport/device class (see above), the Resource API will first try to load a `Puppet::Provider::TypeName::` class from `lib/puppet/provider/type_name/device_type.rb`, before falling back to the regular provider at `Puppet::Provider::TypeName::TypeName`. -### `puppet device` support +### Puppet backwards compatibility -To connect to a remote resource through `puppet device`, you must call a `transport` class through a device shim to maintain compatibility with Puppet Resource. +To connect to a remote resource through `puppet device`, you must provide a device shim to maintain compatibility with Puppet. The device shim needs to interface the transport to puppet's config and runtime expectations. -A `device` class is created that Puppet Resource will call, however this will inherit from the `transport` class which will contain the following methods (as detailed in the [Resource API specification](https://github.com/puppetlabs/puppet-specifications/tree/master/language/resource-api#transport)): -* `initialize` -* `verify` -* `facts` -* `connect` - -For example, the `device` class will be a pass through to `transport`: +In the simplest case you can use the provided `Puppet::ResourceApi::Transport::Wrapper` like this: ```ruby # lib/puppet/util/network_device/device_type/device.rb @@ -290,7 +284,7 @@ require 'puppet/resource_api/transport/wrapper' # force registering the transport schema require 'puppet/transport/schema/device_type' -module Puppet::Util::NetworkDevice::DeviceType +module Puppet::Util::NetworkDevice::Device_type class Device < Puppet::ResourceApi::Transport::Wrapper def initialize(url_or_config, _options = {}) super('device_type', url_or_config)