emann / flex-config Goto Github PK

Is your feature request related to a problem? Please describe.
Currently, there is no way (included with flex-config) to dynamically load values into a config. We commonly will load the default sources, check if the environment is non-local, and then load SSM params.

Describe the solution you'd like
Allow a callable which returns a ConfigSource to be passed in as a source, which could also optionally take in the parsed dict so far to allow it to dynamically load sources.

Is your feature request related to a problem? Please describe.
Currently FlexConfig's AWSSource only supports SSM params.

Describe the solution you'd like
Support AppConfig, either in AWSSource or split AWSSource into SSMSource and AppConfigSource

Is your feature request related to a problem? Please describe.
Currently a FlexConfig object is simply a dictionary and while simple types are inferred/converted, the return type of __getitem__ is Any and thus mypy will complain about using config values directly without converting or casting them. In addition, there isn't any validation for complex types such as a list of dictionaries that should all follow the same structure and the interface for defining a required base structure is a bit cumbersome.

Describe the solution you'd like
Specify a ConfigSchema, which is a subclass of Pydantic's BaseModel and give it as well as the sources to load from to a function that combines all the sources into a dict, creates an instance of the schema with said dict, and returns it.

Describe alternatives you've considered
Theoretically this could be done with dataclasses, but pydantic already has nested dict parsing/validation built in.

Is your feature request related to a problem? Please describe.
Currently only YAML files are supported - it'd be nice (and quite easy) to have support for JSON and TOML as well. No pressing need for it, but its an easy to implement nice to have that we can slip into 2.0

Describe the solution you'd like
Due to the similarities between loading YAML, JSON, or TOML into a dict, they should share a common FileSource parent whose .items() looks something like:

def items(self) -> Iterable[Tuple[str, Any]]:
    """ Returns a generator for getting all path, value pairs """
    if not self.path.exists():
        return []
    config_dict = self._load_file()
    return config_dict.items()

Where ._load_file() imports the respective parsing library, parses the file at self.path, and returns the parsed data dict.

2.0.0 is a complete redo on how stuff works from a user perspective. Config is now based on Pydantic models. We'll need to update the docs to show how to use the new version.

Is your feature request related to a problem? Please describe.
The default __repr__ for BaseModel is pretty ugly and it'd be nice to have a prettier way to print it out the values in a constructed config for debugging purposes and/or interactive applications.

Describe the solution you'd like
Make ConfigSchema an actual subclass of BaseModel and override the __str__ method which would call .dict() and then builds a string with each field/sub-field with the appropriate indentation.

Describe alternatives you've considered
In theory json.dumps or pydantic's built it .json() would work, but both have issues serializing much more than the basic built it types (it even struggles to do enums and paths).

Is your feature request related to a problem? Please describe.
It'd be nice to have a way to automatically generate an example yaml config file from a config schema to include in a project for future contributors. Not something necessary any time soon but would be a nice convenience.

Describe the solution you'd like

def generate_example_file(config_schema: ConfigSchema, path: Path = os.getcwd(), file_types: Optional[Set[str]]=None):
    ...

Generates an instance of the schema, then turns it into a dictionary with .dict() and dumps it into a file of each of the supplied file types (or all supported types if file_types is omitted) at the specified path. Example value precedence for each field would be the following:

1. Default value
2. Example value
3. An example generated off of the type hints

Generating example values might get a little icky, but could be either unsupported initially or have minimal support, etc.