rconradharris / envparse Goto Github PK

Currently having to install from github, would rather install from PyPi.

I haven't found a way to parse dictionary with boolean subtypes.
The variable in .env file is:
FOO=bar1=true, bar2=false
and I parse it with env.dict('FOO', subcast=bool).

Expected result is
{'bar1':True, 'bar2':False}
but I get
{'bar1':True, 'bar2':True} instead.

The reason why this happens is that subcast is called as constructor and dict value is read as string so bool('false') evaluates to True.

The only way I found this working is specifing variable as
FOO=bar1=true, bar2=, but this would be too messy if i had more dict keys.

Another way to parse it as desired is to specify environment variable as json, but I don't want to do that.

Hi,

instead of raising an error in pre processing method if env variable is not set, it would be useful to set the variable to None.

this raise an error

import random

from envparse import env


def get_secret_key(secret_key):
    if not secret_key:
        return "".join([random.choice("abcdefghijklmnopqrstuvwxyz0123456789!@#$^&*(-_=+)") for i in range(50)])
    return secret_key


SECRET_KEY = env('SECRET_KEY', preprocessor=get_secret_key)

print(SECRET_KEY)

the solution I use, make the API is not pretty

SECRET_KEY = env('SECRET_KEY', preprocessor=get_secret_key, default=None)

Is there a reason to raise an exception instead or setting the env var to None ?

Thanks for reading me

I'm curious if there is a reason to not include pre/post processors in the schema?

Could add a few more lines after https://github.com/rconradharris/envparse/blob/master/envparse.py#L85:

if preprocessor is None:
    preprocessor = params.get('preprocessor', None)
if postprocessor is None:
    postprocessor = params.get('postprocessor', None)

Problem

In Kubernetes, sensitive values cannot be safely stored in the containers environment as this is exposed to any who can view a pod's spec: this is usually a broad selection of people. However, they can be safely stored in Secrets, which can be safely mounted to files in the container: Secrets are usually encrypted at rest, and can have much tighter restriction on who can read/write them without impairing the maintenance of the cluster.

Unfortunately, file contents are lower priority than environment variables in envparse, meaning that anyone with the ability to write environment variables to a pod can supersede more sensitive/controlled values that have been set in an env file sourced from a mounted Secret.

Proposal

We can add a flag to the read_envfile method that will switch the behaviour from setdefault to a straight assignment, allowing people to choose to prioritise environment variables or env file contents when there is a conflict.

We're using envparse in a Django project that uses consul-template to provide the env file. When config is updated in Consul, the env file is updated and we touch the WSGI file to (attempt to) pick up the new configuration.

However this doesn't work as env.read_envfile uses os.environ.setdefault so updates are never applied (as the env var already exists).

What's the reasoning for using setdefault instead of just setting directly. If that can't be changed, would you be open to passing an additional flag to read_envfile to allow updates? Am happy to write pull request.

Would involve:

• mapping a Env.__getitem__ function that maps to Env.__call__
• a Env.__dict__ that outputs schema values
• a Env.__str__ prints out dict

I have this secret key (an example): DJANGO_SECRET_KEY=YwRqr;u|g_Qb3hEPRm/B6ry#bMb;-c,YmFaGsi>/s?rdhnc$M`

It contains #. When I try to use it, that's what I see:

>>> env('DJANGO_SECRET_KEY')
'YwRqr;u|g_Qb3hEPRm/B6ry'

Hi @rconradharris,

I noticed that there's a number of hanging issues and PRs nobody paid attention to in 3 years.

Have you considered finding another pair of hands to help you maintain envparse?

Better yet — there's a project called @jazzband which could be a new home for this repo. It's a community of maintainers where multiple interested parties can collaborate on moving forward the projects that the original authors/maintainers have no resources to maintain anymore.

For more info, please read more at https://jazzband.co/about. I hope that it'll help reduce the frustration of many folks who'd like to see a couple of improvements to this project.

Have a nice day :)

Negative numbers are converted into positive ones. env.list seem to work with subcast
I think the problem is here, the minus sign is stripped:

Refer to this line, https://github.com/rconradharris/envparse/blob/master/envparse.py#L54

Can we have "True" as truthy value? It's python's format. If yes, I could do a PR.

Cheers

Thank you for envparse. I liked its API so much that I implemented a successor library, environs, which uses marshmallow to do the heavy lifting for casting, validation, and serialization.

environ's API is near-identical to envparse, with a few differences:

• environs supports a few more cast types, including decimal, datetime, date, timedelta, and uuid
• environs adds validation. https://github.com/sloria/environs#validation
• environs implements serialization: https://github.com/sloria/environs#serialization
• environs allows the user to register custom parser methods: https://github.com/sloria/environs#defining-custom-parser-behavior . This makes it easy to create plugins; see https://github.com/sloria/environs/blob/master/examples/plugin_example.py
• environs handles prefixes: https://github.com/sloria/environs#handling-prefixes
• You cannot pass a schema to the Env constructor. (Env(FOO=bool)). This was intentional; I did not see an advantage in having the redundant API.
• environs does not implement env('VARNAME', cast=str); you must do env.str('VARNAME'). Again, the intention was to avoid redundant API.
• preprocessor and postprocessor are not supported. Any pre- or post-processing can happen in a custom parser function
• environs cannot read .env files (yet). Up for discussion here: sloria/environs#1
• environs does not support proxied variables (yet)

environs was mostly written as a proof-of-concept, though at this point it is stable enough to use (it's only ~100SLOC after all).

I wonder if we could reduce duplication of effort by merging these two libraries. I haven't yet thought about the particulars; just wanted to open the table for discussion first.

Thoughts?

Hey, I notice the code for boolean casting comes from a limited set of hard-coded strings, I've recently become aware of a python-standard strtobool, I believe the only one missing would be "ok".

Would you object to a small PR migrating to that?

I would like to be able to access a full list of variables or just their names in order to log the configuration of the app at startup.
Is there some way to do this already? If not, can we add it?

Using envparse 0.16 on Python 2.7:

$ export foo="abc,def"
$ python
...
>>> from envparse import env
>>> env("foo", cast=list)
['a', 'b', 'c', ',', 'd', 'e', 'f']

Whereas I would have expected:
['abc', 'def']

Am I doing something wrong, or is this a bug?

When using read_envfile with path defined, the function still recurses up the directory tree if the given path is not found.

Consider the following code:

import envparse
envparse.env.read_envfile('myenv.conf')

The app runs from /home/test/code, which doesn't have myenv.conf file. The following debug log:

DEBUG:envfile not found at myenv.conf, looking in parent dir.
DEBUG:envfile not found at /home/test/myenv.conf, looking in parent dir.
DEBUG:envfile not found at /home/myenv.conf, looking in parent dir.
DEBUG:envfile not found at /myenv.conf, looking in parent dir.
UserWarning: Could not any envfile.

This is contrary to the documentation of the function. The function should not recurse if path is given, but file missing, or if it does this needs to be documented.

It would be helpful to add some description parameter to env vars so that on printing warning/exceptions we can show more details on env vars

See https://github.com/rconradharris/envparse/blob/master/envparse.py#L195. It should read

Could not find any envfile

I'm trying to raise the warning level of the envparse module but the logger is named after the file and so doesn't have a consistent name. Maybe just call it "envparse"?

You need to change your readme.md, read_env --> read_envfile