Advanced Usage¶
Creating your own types¶
Config Suite supports you creating your own basic types. This is done by
creating a new instance of configsuite.BasicType
. The constructor takes a
type name, as well as a validator of the type. For instance, the Date
type in Config Suite could be implemented as follows:
import configsuite
import datetime
@configsuite.validator_msg("Is x a date")
def _is_date(x):
return isinstance(x, datetime.date)
Date = configsuite.BasicType("date", _is_date)
After this, you can use Date
as a MetaKeys.Type
value in your schema as
displayed in the cars
-schema.
import yaml
from configsuite import MetaKeys as MK
schema = {MK.Type: Date}
config = yaml.load("1988-06-05")
suite = configsuite.ConfigSuite(config, schema)
print(suite.valid)
print(suite.snapshot)
True
1988-06-05
Context validators¶
Context validators enables validation of an entry based on the value of entries located elsewhere in the configuration. This is a two-step procedure; where the first step is a consumer-defined function for extracting context of a snapshot. The given snapshot is guaranteed to be readable and is extracted after all transformations have been applied. Then, after an element have been validated recursively and the element validator has been applied, if the element is still deemed valid, the context validator is applied.
The typical application of a context validator is when you have multiple, central concepts in your configuration. An example would be a configuration of students and classes. You want to enable specification of both students and classes, yet you also want to list the students attending each class.
import collections
import configsuite
from configsuite import MetaKeys as MK
from configsuite import types
_student_schema = {
MK.Type: types.List,
MK.Content: {
MK.Item: {
MK.Type: types.NamedDict,
MK.Content: {
"name": {MK.Type: types.String},
"age": {MK.Type: types.Integer},
"favourite_lunch": {MK.Type: types.String}
},
},
},
}
@configsuite.validator_msg("Is x a student name")
def _is_student(name, context):
return name in context.student_names
_course_schema = {
MK.Type: types.List,
MK.Content: {
MK.Item: {
MK.Type: types.NamedDict,
MK.Content: {
"name": {MK.Type: types.String},
"max_size": {MK.Type: types.Integer},
"students": {
MK.Type: types.List,
MK.Content: {
MK.Item: {
MK.Type: types.String,
MK.ContextValidators: (_is_student,),
},
},
},
},
},
},
}
schema = {
MK.Type: types.NamedDict,
MK.Content: {
"students": _student_schema,
"courses": _course_schema,
},
}
def _extract_student_names(snapshot):
Context = collections.namedtuple("Context", ("student_names",))
student_names = tuple(
student.name for student in snapshot.students
)
return Context(student_names=student_names)
Note that we have split the schema in two in this example to keep it a bit more
manageable. Also, observe how _is_student
takes both a name
(which we know
is a string
) as well as a context
. Besides that, we had to implement a
function that extracts the context
from a snapshot. Notice that we are in
complete control of the context
, which means that we could have returned the
entire snapshot, or a tuple containing just the student
names. The first is
not recommended as it is good to be conscious regarding what the context
contains. In particular, a user will have to carry the same amount of
information in their head while auditing a configuration file. The latter,
because we recommend to make the context extendable without having to change
all the context validators. And by putting all student names under an
attribute, we achieve just that.
Now, to create a suite with the configuration config
, we do as follows:
config = {
"students": [
{
"name": "Per",
"age": 21,
"favourite_lunch": "graut",
},
{
"name": "Espen",
"age": 17,
"favourite_lunch": "troll",
},
],
"courses": [
{
"name": "adventures-101",
"max_size": 50,
"students": ["Per", "Espen"],
},
],
}
suite = configsuite.ConfigSuite(
config,
schema,
extract_validation_context=_extract_student_names,
)
print(suite.valid)
True
However, if we add a course with an unknown student:
invalid_suite = suite.push({
"courses": [
{
"name": "impossible-101",
"max_size": 0,
"students": "Pål",
},
],
})
print(invalid_suite.valid)
print(invalid_suite.errors)
False
(InvalidTypeError(msg=Is x a list is false on input 'Pål', key_path=('courses', 0, 'students'), layer=1),)
Context transformations¶
Context transformations allows for transforming elements based on the values of other elements. This was implemented to support the following scenario; you want to define variables in one part of the configuration and then substitute values in another part based on these variables. We will now display a simple implementation of such a system.
First, we must implement functionality for given a template
and definitions
to render the templates. That can be done as follows:
import collections
import copy
import jinja2
import configsuite
from configsuite import MetaKeys as MK
from configsuite import types
# To avoid collision with dict and set syntax in yaml
_VAR_START = "<"
_VAR_END = ">"
def _render_variables(variables, jinja_env):
"""Repeatedly render the variables to support the scenario when one
variable refers to another one.
"""
variables = copy.deepcopy(variables)
for _ in enumerate(variables):
rendered_values = []
for key, value in variables.items():
try:
variables[key] = jinja_env.from_string(value).render(variables)
rendered_values.append(variables[key])
except TypeError:
continue
if any([_VAR_START in val for val in rendered_values]):
raise ValueError("Circular dependencies")
return variables
def _render(template, definitions):
"""Render a template with the given definitions."""
if definitions is None:
definitions = {}
variables = copy.deepcopy(definitions)
jinja_env = jinja2.Environment(
variable_start_string=_VAR_START, variable_end_string=_VAR_END, autoescape=True
)
try:
variables = _render_variables(variables, jinja_env)
jinja_template = jinja_env.from_string(template)
return jinja_template.render(variables)
except TypeError:
return template
@configsuite.transformation_msg("Renders Jinja template using definitions")
def _context_render(elem, context):
return _render(elem, definitions=context.definitions)
Second, we must implement a context extractor.
def extract_templating_context(configuration):
Context = collections.namedtuple("TemplatingContext", ["definitions"])
definitions = {key: value for (key, value) in configuration.definitions}
return Context(definitions=definitions)
Third, we define the schema.
schema = {
MK.Type: types.NamedDict,
MK.Content: {
"definitions": {
MK.Type: types.Dict,
MK.Content: {
MK.Key: {MK.Type: types.String},
MK.Value: {MK.Type: types.String},
},
},
"templates": {
MK.Type: types.List,
MK.Content: {
MK.Item: {
MK.Type: types.String,
MK.ContextTransformation: _context_render,
}
},
},
},
}
And then, given the following yaml-configuration:
definitions:
animal: pig
habitants: <animal>, cow and monkey
color: blue
secret_number: "42"
templates:
- This is a story about a <animal>.
- It had a <color> house.
- And the password to enter was <secret_number>.
- If you entered the house you would meet: <habitants>.
- The end.
we obtain the following rendered templates
after feeding the config
,
schema
and the extract_templating_context
through
configsuite.ConfigSuite
.
import yaml
config = yaml.safe_load("""
definitions:
animal: pig
habitants: <animal>, cow and monkey
color: blue
secret_number: "42"
templates:
- This is a story about a <animal>.
- It had a <color> house.
- And the password to enter was <secret_number>.
- "If you entered the house you would meet: <habitants>."
- The end.
""")
suite = configsuite.ConfigSuite(
config,
schema,
extract_transformation_context=extract_templating_context,
)
print(suite.valid)
True
Furthermore, if we print the rendered templates we get the following:
for line in suite.snapshot.templates:
print(line)
This is a story about a pig.
It had a blue house.
And the password to enter was 42.
If you entered the house you would meet: pig, cow and monkey.
The end.
Notice that we can now merge multiple layers, with definitions in higher levels taking precedence.
A note on contexts¶
In these section we cover some advance topics that should be used with care. The contexts due to the fact that parts of the configuration can no longer be validated independently. Which implies that the user might have to make changes far apart to keep a configuration file consistent, data that is naturally displayed together in a UI cannot be validated without taking the rest of the configuration into account and the difficulty of understanding your configuration increases.
Layer transformations¶
A layer transformation is applied to an element of a layer before the various layers are merged. The observant reader might notice that hence the layer transformations provide no real benefit over a standard transformation for Basic Types. The natural application of a layer transformation is to transform a collection such that merging can be carried out. Due to this, Config Suite can give no guarantee what so ever on the content of the data provided to a layer transformation.
A natural application of layer transformations is to support ranges where lists
of integers are expected. In particular, one would like to be able to write
1-3, 5-7, 9
and get [1, 2, 3, 5, 6, 7, 9]
as a result. Assume one implements a
function _realize_list
that takes as input a string of ranges and singletons
and returns a list as in the example above. And that one in addition decorates it with a
transformation_msg
. Then, the following schema
from configsuite import MetaKeys as MK
from configsuite import types
{
MK.Type: types.List,
MK.LayerTransformation: _realize_list,
MK.Content: {MK.Item: {MK.Type: types.Integer}},
}
gives the specification of a list of integers for which one can instead provide strings of ranges and singletons in some of the layers. For a full implementation, where in addition the final list (after the layers have been merged) is sorted and duplicates are removed we refer the reader to the test data.
Note that for the layer transformations to give the intended functionality they are applied in a top down manner. This is another distinction from the other transformations (including the context transformations further down) that are all applied in a bottom up manner.