Validation (#86)

* Added validation using jsonschema

* Update version

* Default to using the poetry --all-extras flag

Author: tr11

.github/workflows/run_tests.yml

@@ -49,7 +49,7 @@ jobs:
 
       - name: Install dependencies
         if: steps.cache.outputs.cache-hit != 'true'
-        run: poetry install -v -E toml -E yaml -E azure -E aws -E gcp -E vault
+        run: poetry install -v --all-extras
 
       - name: Run pytest
         run: poetry run pytest --cov=./ --cov-report=xml

CHANGELOG.md

@@ -4,9 +4,15 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+
+## [0.9.1] - 2023-08-06
+
 ### Added
 
 - Allow to pass a `ignore_missing_paths` parameter to each config method
+- Support for Hashicorp Vault credentials (in `config.contrib`)
+- Added a `validate` method to validate `Configuration` instances against a [json schema](https://json-schema.org/understanding-json-schema/basics.html#basics).
+
 
 ## [0.9.0] - 2023-08-04
 
@@ -31,12 +37,14 @@ All notable changes to this project will be documented in this file.
 
 - Configurations from ini file won't be converted to lower case if `lowercase_keys = False`
 
+
 ## [0.8.2] - 2021-01-30
 
 ### Fixed
 
 - The behavior of merging sets was incorrect since version 0.8.0
 
+
 ## [0.8.0] - 2020-08-01
 
 ### Changed
@@ -60,24 +68,28 @@ with cfg.dotted_iter():
 - Support for _.env_-type files
 - Option for deep interpolation. To activate that mode, use one of the enum values in `InterpolateEnumType` as the `interpolate_type` parameter. This allows for hierachical _templates_, in which configuration objects use the values from lower ones to interpolate instead of simply overriding.
 
+
 ## [0.7.1] - 2020-07-05
 
 ### Fixed
 
 - Installation with `poetry` because of changes to pytest-black
 
+
 ## [0.7.0] - 2020-05-06
 
 ### Added
 
 - New string interpolation feature
 
+
 ## [0.6.1] - 2020-04-24
 
 ### Changed
 
 - Added a `separator` argument to `config` function
 
+
 ## [0.6.0] - 2020-01-22
 
 ### Added
@@ -86,6 +98,7 @@ with cfg.dotted_iter():
 - Added a `reload` method to refresh a `Configuration` instance (can be used to reload a configuration from a file that may have changed).
 - Added a `configs` method to expose the underlying instances of a `ConfigurationSet`
 
+
 ## [0.5.0] - 2020-01-08
 
 ### Added
@@ -98,13 +111,15 @@ with cfg.dotted_iter():
 
 - Changed the `__repr__` and `__str__` methods so possibly sensitive values are not printed by default.
 
+
 ## [0.4.0] - 2019-10-11
 
 ### Added
 
 - Allow path-based failures using the `config` function.
 - Added a levels option to the dict-like objects.
 
+
 ## [0.3.1] - 2019-08-20
 
 ### Added
@@ -113,26 +128,30 @@ with cfg.dotted_iter():
 - TravisCI support
 - Codecov
 
+
 ## [0.3.0] - 2019-08-16
 
 ### Changed
 
 - Changed the old behavior in which every key was converted to lower case.
 
+
 ## [0.2.0] - 2019-07-16
 
 ### Added
 
 - Added Sphinx documentation
 - Added a `remove_levels` parameter to the config function
 
+
 ## [0.1.0] - 2019-01-16
 
 ### Added
 
 - Initial version
 
-[unreleased]: https://github.com/tr11/python-configuration/compare/0.9.0...HEAD
+[unreleased]: https://github.com/tr11/python-configuration/compare/0.9.1...HEAD
+[0.9.1]: https://github.com/tr11/python-configuration/compare/0.9.0...0.9.1
 [0.9.0]: https://github.com/tr11/python-configuration/compare/0.8.3...0.9.0
 [0.8.3]: https://github.com/tr11/python-configuration/compare/0.8.2...0.8.3
 [0.8.2]: https://github.com/tr11/python-configuration/compare/0.8.0...0.8.2

README.md

@@ -26,6 +26,7 @@ and optionally
 * Azure Key Vault credentials
 * AWS Secrets Manager credentials
 * GCP Secret Manager credentials
+* Hashicorp Vault credentials
 
 ## Installing
 
@@ -276,13 +277,62 @@ When setting the `interpolate` parameter in any `Configuration` instance, the li
 cfg = config_from_dict({
     "percentage": "{val:.3%}",
     "with_sign": "{val:+f}",
-    "val": 1.23456}, interpolate=True)
+    "val": 1.23456,
+    }, interpolate=True)
 
 assert cfg.val == 1.23456
 assert cfg.with_sign == "+1.234560"
 assert cfg.percentage == "123.456%"
 ```
 
+###### Validation
+
+Validation relies on the [jsonchema](https://github.com/python-jsonschema/jsonschema) library, which is automatically installed using the extra `validation`. To use it, call the `validate` method on any `Configuration` instance in a manner similar to what is described on the `jsonschema` library:
+
+```python
+schema = {
+    "type" : "object",
+    "properties" : {
+        "price" : {"type" : "number"},
+        "name" : {"type" : "string"},
+    },
+}
+
+cfg = config_from_dict({"name" : "Eggs", "price" : 34.99})
+assert cfg.validate(schema)
+
+cfg = config_from_dict({"name" : "Eggs", "price" : "Invalid"})
+assert not cfg.validate(schema)
+
+# pass the `raise_on_error` parameter to get the traceback of validation failures
+cfg.validate(schema, raise_on_error=True)
+# ValidationError: 'Invalid' is not of type 'number'
+```
+
+To use the [format](https://python-jsonschema.readthedocs.io/en/latest/validate/#validating-formats) feature of the `jsonschema` library, the extra dependencies must be installed separately as explained in the documentation of `jsonschema`.   
+
+```python
+from jsonschema import Draft202012Validator
+
+schema = {
+    "type" : "object",
+    "properties" : {
+        "ip" : {"format" : "ipv4"},
+    },
+}
+
+cfg = config_from_dict({"ip": "10.0.0.1"})
+assert cfg.validate(schema, format_checker=Draft202012Validator.FORMAT_CHECKER)
+
+cfg = config_from_dict({"ip": "10"})
+assert not cfg.validate(schema, format_checker=Draft202012Validator.FORMAT_CHECKER)
+
+# with the `raise_on_error` parameter:
+c.validate(schema, raise_on_error=True, format_checker=Draft202012Validator.FORMAT_CHECKER)
+# ValidationError: '10' is not a 'ipv4'
+```
+
+
 ## Extras
 
 The `config.contrib` package contains extra implementations of the `Configuration` class used for special cases. Currently the following are implemented:
@@ -311,6 +361,14 @@ The `config.contrib` package contains extra implementations of the `Configuratio
   pip install python-configuration[gcp]
   ```
 
+* `HashicorpVaultConfiguration` in `config.contrib.vault`, which takes Hashicorp Vault
+  credentials into a `Configuration`-compatible instance. To install the needed dependencies
+  execute
+
+  ```shell
+  pip install python-configuration[vault]
+  ```
+
 ## Features
 
 * Load multiple configuration types

config/configuration.py

@@ -391,6 +391,21 @@ def reload(self) -> None:  # pragma: no cover
         """
         raise NotImplementedError()
 
+    def validate(
+        self, schema: Any, raise_on_error: bool = False, **kwargs: Mapping[str, Any]
+    ) -> bool:
+        try:
+            from jsonschema import validate, ValidationError
+        except ImportError:  # pragma: no cover
+            raise RuntimeError("Validation requires the `jsonschema` library.")
+        try:
+            validate(self.as_dict(), schema, **kwargs)
+        except ValidationError as err:
+            if raise_on_error:
+                raise err
+            return False
+        return True
+
     @contextmanager
     def dotted_iter(self) -> Iterator["Configuration"]:
         """

pyproject.toml

@@ -9,7 +9,7 @@ name = "python-configuration"
 packages = [{ include = "config" }]
 readme = 'README.md'
 repository = "https://github.com/tr11/python-configuration"
-version = "0.9.0"
+version = "0.9.1"
 
 [tool.poetry.dependencies]
 python = "^3.8.1"
@@ -20,6 +20,7 @@ google-cloud-secret-manager = { version = "^2.16.3", optional = true }
 hvac = { version ="^1.1.1", optional = true }
 pyyaml = { version = "^6.0", optional = true }
 toml = { version = "^0.10.0", optional = true }
+jsonschema = { version = "^4.18.6", optional = true }
 
 [tool.poetry.group.dev.dependencies]
 flake8-blind-except = "^0.2.0"
@@ -47,6 +48,7 @@ gcp = ["google-cloud-secret-manager"]
 toml = ["toml"]
 vault = ["hvac"]
 yaml = ["pyyaml"]
+validation = ["jsonschema"]
 
 [tool.black]
 line-length = 88
@@ -60,7 +62,7 @@ envlist = py38, py39, py310, py311
 [testenv]
 allowlist_externals = poetry
 commands =
-    poetry install -v -E toml -E yaml -E azure -E aws -E gcp -E vault
+    poetry install -v --all-extras
     poetry run pytest
 """
 
@@ -89,6 +91,8 @@ module= [
     'botocore.exceptions', 
     'hvac',
     'hvac.exceptions',
+    'jsonschema',
+    'jsonschema.exceptions'
 ]
 ignore_missing_imports = true
 

tests/test_validation.py

@@ -0,0 +1,93 @@
+from config import (
+    Configuration,
+    ConfigurationSet,
+    EnvConfiguration,
+    config,
+    config_from_dict,
+)
+import pytest
+
+try:
+    import jsonschema
+except ImportError:
+    jsonschema = None
+
+
+@pytest.mark.skipif("jsonschema is None")
+def test_validation_ok():  # type: ignore
+    d = {"items": [1, 3]}
+    cfg = config_from_dict(d)
+
+    schema = {
+        "type": "object",
+        "properties": {
+            "items": {
+                "type": "array",
+                "items": {"enum": [1, 2, 3]},
+                "maxItems": 2,
+            }
+        },
+    }
+
+    assert cfg.validate(schema)
+
+
+@pytest.mark.skipif("jsonschema is None")
+def test_validation_fail():  # type: ignore
+    from jsonschema.exceptions import ValidationError
+
+    schema = {
+        "type": "object",
+        "properties": {
+            "items": {
+                "type": "array",
+                "items": {"enum": [1, 2, 3]},
+                "maxItems": 2,
+            }
+        },
+    }
+
+    with pytest.raises(ValidationError) as err:
+        d = {"items": [1, 4]}
+        cfg = config_from_dict(d)
+        assert not cfg.validate(schema)
+        cfg.validate(schema, raise_on_error=True)
+    assert "4 is not one of [1, 2, 3]" in str(err)
+
+    with pytest.raises(ValidationError) as err:
+        d = {"items": [1, 2, 3]}
+        cfg = config_from_dict(d)
+        assert not cfg.validate(schema)
+        cfg.validate(schema, raise_on_error=True)
+    assert "[1, 2, 3] is too long" in str(err)
+
+
+@pytest.mark.skipif("jsonschema is None")
+def test_validation_format():  # type: ignore
+    from jsonschema import Draft202012Validator
+    from jsonschema.exceptions import ValidationError
+
+    schema = {
+        "type": "object",
+        "properties": {
+            "ip": {"format": "ipv4"},
+        },
+    }
+
+    cfg = config_from_dict({"ip": "10.0.0.1"})
+    assert cfg.validate(schema, format_checker=Draft202012Validator.FORMAT_CHECKER)
+
+    # this passes since we didn't specify the format checker
+    cfg = config_from_dict({"ip": "10"})
+    assert cfg.validate(schema)
+
+    # fails with the format checker
+    with pytest.raises(ValidationError) as err:
+        cfg = config_from_dict({"ip": "10"})
+        cfg.validate(
+            schema,
+            raise_on_error=True,
+            format_checker=Draft202012Validator.FORMAT_CHECKER,
+        )
+    print(str(err))
+    assert "'10' is not a 'ipv4'" in str(err)