rigetti / paranormal

A declarative, parameter-parsing library that provides multiple parsing interfaces (YAML, command line, and JSON)

paranormal

A declarative, parameter-parsing library that provides multiple parsing interfaces (YAML, command line, and JSON) for loading parameters.

Python Install:

Just install from PyPi using pip install paranormal.

Running unit tests

Unit tests can be executed by running pytest from top folder of the repository.

Using the Library

The following code samples show how this library is meant to be used.

Subclass Params

from paranormal.parameter_interface import *
from paranormal.params import *

# Note that only direct inheritance from Params is currently supported
class FrequencySweep(Params):
    """
    A frequency sweep measurement
    """
    freqs = SpanArangeParam(help='A list of frequencies to scan as [center, width, step]',
                            default=(1e9, 2e9, 0.1e9), unit='GHz')
    power = FloatParam(help='Power to transmit', default=-20, unit='dBm')
    pulse_samples = IntParam(help='Samples in the pulse', default=100)
    averages = IntParam(help='Number of sweeps to average over', default=10)
    is_test = BoolParam(help='Is this just a test', default=False)
    unseen_test = BoolParam(help='Hidden from the command line', default = False, hide=True)
    _unseen_test_2 = BoolParam(help='Hidden from the command line bc of the _', default=False)

Reading From the Command Line

parser = to_argparse(FrequencySweep)
# argparse will grab the command line arguments
# Ex command line: '--freqs 150 100 2 --power -40 --pulse_samples 200 --is_test'
args = parser.parse_args()
sweep_params = from_parsed_args(FrequencySweep, params_namespace=args)[0]

# even_simpler
sweep_params = create_parser_and_parse_args(FrequencySweep)

Setting and getting parameters

print(sweep_params.freqs)  # prints a numpy array of size (20,) array([0.0e+00, 1.0e+08, 2.0e+08, …
sweep_params.freqs = [5e9, 10e9, 2e9]
print(sweep_params.freqs)  # prints array([0.e+00, 2.e+09, 4.e+09, 6.e+09, 8.e+09])
print(sweep_params.is_test)  # prints True
sweep_params.freqs = [None, 10e9, 2e9]
print(sweep_params.freqs)  # prints [None, 10e9, 2e9]

JSON and YAML serialization

import json
import yaml  # pyyaml

d = to_json_serializable_dict(sweep_params)
s = json.dumps(d)
sweep_params = from_json_serializable_dict(json.loads(s))


to_yaml_file(sweep_params, 'test_params.yaml')
sweep_params = from_yaml_file('test_params.yaml')

Nested Params

class MultipleFreqSweeps(Params):
    sweep_1 = FrequencySweep(freqs=[0, 1e9, 0.1e9])  # overwrites default freqs value
    sweep_2 = FrequencySweep(freqs=[1e9, 2e9, 0.1e9])  # overwrites default freqs value
    
# Omit a nested param from the outer class:
class MultipleFreqSweepsHidden(Params):
    sweep_1 = FrequencySweep(freqs='__omit__')
    sweep_2 = FrequencySweep(power='__omit__')
        
# Customize prefixes used for command line parsing    
class MultipleFreqSweepsCustom(Params):
    sweep_1 = FrequencySweep()
    sweep_2 = FrequencySweep()
    __nested_prefixes__ = {'sweep_1': None, 'sweep_2': 'second_'}
    # Ex command line: '--freqs 100 120 2 --power -30 --second_power -40'
    
# Hide a nested param from the command line, but keep it in the class:
class MultipleFreqSweepsCustom(Params):
    sweep_1 = FrequencySweep()
    sweep_2 = FrequencySweep()
    __nested_prefixes__ = {'sweep_1': None, 'sweep_2': 'second_'}
    __params_to_hide__ = ['sweep_2_power']

paranormal's Issues

Remove __add__

paranormal/paranormal/parameter_interface.py

Line 353 in 5758336

def __add__(self, other):

prevent mutating defaults in numpy fn param

Should pass back the argument as a tuple

README example has incorrect usage of `SpanArangeParam`

It uses it as as [center, width, num] when it's implemented as [center, width, step]

Hide vs Omit

When creating nested params, want to be able to hide some from the command line but not omit them

Overwriting default is a permanent overwrite

• Should not be permanent

There should be an easy way to assign a parameter with SI units

Right now, if you have a value in SI units, you need to call params.x = unconvert_si_units(value, get_param_unit(cls, 'x')), which is unnecessarily verbose. It would be nice to use something like param.si_set('x', value) instead

Function to concatenate attributes from one params class into another (with option to leave some out or replace the attrs of others)

Ex.

class A(Params):
    i = IntParam(help='int', default=1)

class B(Params):
    f = FloatParam(help='float', default=2.0)
    z = IntParam(help='another int')


concatenate_param_attributes(A, B, do_not_copy=['z'], override_dictionary={'f' : {'help': 'a better float', 'default': 3.0}})
a = A()
print(a.f)  # prints 3.0
print(a.z)  # throws error

allow numpy param with center, width, num

docstrings on params classes

remove dependency upper bounds in pyproject.toml

The package pyproject toml is using ^ limit some packages which is causing versioning conflicts.
^ in poetry prevents changes on the leftmost non-zero digit. In the case of, for example, pampy = "^0.2.1", we're stuck with the minor revision 0.2.x and only have wiggle room to change the patch versions; which is problematic.
Setting some packages to >= could be useful.

prefix _ gets added twice

in __nested_prefixes__, example is given in readme as:

# Customize prefixes used for command line parsing    
class MultipleFreqSweepsCustom(Params):
    sweep_1 = FrequencySweep()
    sweep_2 = FrequencySweep()
    __nested_prefixes__ = {'sweep_1': None, 'sweep_2': 'second_'}
    # Ex command line: '--freqs 100 120 2 --power -30 --second_power -40'

but the _ gets added again by _create_param_name_prefix when it shouldn't

Resolving name conflicts for nested class params that are expanded doesn't append a prefix to every expanded argument

Streamline types for descriptors set and get

Instead of Lists, we should allow both a list and a tuple for params like SpanArange, etc.

Ex.

class A(Params):
    freqs = SpanArangeParam(default=(1, 2, 0.1), help='freqs')

a = A()
a.freqs = [0, 1, 0.2]
a.freqs = (0, 2, 0.5)

Be able to override the descriptor behavior by setting the attribute equal to anything

If you have something like:

class FreqSweep(Params):
    freqs = SpanArangeParam(help='freqs', default=[1, 1, 0.1], unit='GHz')

and you don't want freqs to be a span arange for some reason, you should be able to override the descriptor behavior like so:

f = FreqSweep()
f.override('freqs', np.linspace(1e9, 20e9, 100), to_json_hook=lambda x: x.tolist(), from_json_hook=lambda x: np.array(x))
print(f.freqs)  # prints an array of 100pts from 1e9 to 20e9
f.freqs = [1, 1, 0.1]  # returns to default behavior 

Note that overriding the SpanArangeParam will break JSON serializability guarantees, so we add extra arguments to give the user an option to provide their own JSON hooks.

This will require refactoring all descriptor get and set methods as well as JSON serialization routines.

Allow hide as param input instead of requiring leading underscore

When initializing a param, it would be nice to have the option to set it as a hidden param without naming it using a leading underscore.

Class attributes aren't present in subclasses

This breaks the Params __init__ function when you subclass it twice

def __init__(self, **kwargs):
        cls = type(self)
        for key, value in kwargs.items():
            if not key in cls.__dict__:
                raise KeyError(f'{cls.__name__} does not take {key} as an argument')
            setattr(self, key, value)
        _check_for_required_arguments(cls, kwargs)

ex.

class A(Params):
    x = BoolParams(default=True, help='')

class B(A):
    y = IntParam(help='')

# raises a KeyError
b = B(x=False)

Hide or override defaults of nested params classes

Ex:

class A(Params):
    x = IntParam(help='int')

class B(Params):
   a = A()
   remove_params(a, 'x')
   x = IntParam(help='a better int')

Nested params classes are mutable and shouldn't be

Ex.

class A(Params):
    f = FloatParam(help='float', default=2)

class B(Params):
    a = A(f = 5)

b = B()
b.a.f = 7
x = B()
print(x.a.f)  # prints 7 and not 5

to_json_serializable_dict doesn't correctly serialize nested classes

This happens when enclosing classes set the nested class param value within its constructor

Failure mode:

class A(Params):
    f = FloatParam(help='float', default=2)

class B(Params):
    a = A(f = 5)

b = B()
to_json_serializable_dict(b)  # will give {'a': {'f': 5, ...}, ...} as expected
b.a.f = 7
to_json_serializable_dict(b)  # will give {'a': {'f': 5, ...}, ...}, which should actually be
# {'a': {'f': 7, ...}, ...}

+= not blocked from params with units

If a param has a certain unit, += is broken

Ex.

class A:
    b = FloatParam(help='float', unit='ns')

a = A()
a.b = 2
print(a.b)  # prints 2e-9
a.b += 2
print(a.b)  # prints (2e-9 + 2) * 1e-9

There's no way to block the __iadd__ method for this case because what's actually being called is the __iadd__ method on a.b, which is a float. The alternative is to use si_set:

a.si_set('b', a.b + 2e-9)

but this isn't that nice. Alternatives are to remove units from the __get__ and __set__ methods and only use them for parsing from the command line or from a yaml.

Expose merge_param_classes function

This will be a useful function to expose

Allow users to specify param type in cli or yaml for subset of numpy functions

Would be pretty cool if you could do something like:
--delays 0 100 10 --delays_function Linspace
or
--delays 0 100 10 --delays_function Arange

and the corresponding Param would be created. You can imagine having a Param called NumpyFunctionParam that isn't specific and will add an extra x_function argument to the parser or to the json dictionary. That function will specify which function should be executed on those parameters.

This would be a really cool feature and actually wouldn't require that significant of a refactor (if we were to implement it the way I've specified above

Would be nice if flattening a class automatically resolved param name conflicts by appending prefixes

Example:

class A(Params):
    freqs = LinspaceParam(expand=True, help='lin')

class B(Params):
    freqs = LinspaceParam(expand=True, help='lin')

class C(Params):
    a = A()
    b = B()

c = create_parser_and_parse_args(C)
# Should be able to parse: --a_freqs_center --a_freqs_width --a_freqs_num --b_freqs_center --b_freqs_width --b_freqs_num

Additionally, it should handle conflicts within the same class:

class A(Params):
    freqs = LinspaceParam(expand=True, help='freqs')
    powers = LinspaceParam(expand=True, help='pwrs')

a = create_parser_and_parse_args(A)
# Should become --freqs_center --freqs_width --freqs_num --powers_center --powers_width --powers_num

overriding defaults of nested params class broken when parsing from the command line

Ex.

class MultipleFreqSweeps(Params):
    sweep_1 = FrequencySweep(freqs=[0, 1, 100])
    sweep_2 = FrequencySweep(freqs=[1, 2, 100])

When you convert this class to a parser and parse a blank command line, the returned class
will not have sweep_1 freqs = [0,1,100] and sweep_2_freqs = [1,2,100]. Instead, both will be whatever the default for freqs is that's specified within FrequencySweep.

prefixes for nested class parameters should be optional

If the user would like no prefixes, then there should be no prefixes - merge conflicts at your own risk

Find explicit way to expose Params subclass parameters to the user when constructing an instance of the subclass

• Right now, if you construct an instance of a Params subclass, you either need to manually create an __init__ function to override the BaseClass init to explicitly show what the subclass attributes are. Or, you need to read the source code.

• Neither of these is ideal for people scripting in Ipython or using a jupyter notebook.

• Possible avenues:

1. Override the help on a class somehow, such that it prints all the attributes.
2. Autogenerate an __init__ function through a decorator on the Params subclass

nit Tuple[Params]

paranormal/paranormal/parameter_interface.py

Line 762 in 6d17c7e

def from_parsed_args(*cls_list, params_namespace: Namespace) -> Tuple:

Unroll Params subclass when help called on a function that uses it (has it in the type hints)

• This ensures notebook and ipython users don't have to dig an extra layer to find out what they need to pass in to the Params subclass.

Should be able to change just one parameter of the three used to define a numpy function param

Ex.

class P(Params):
    a = LinspaceParam(help='linspace', default=[0.0, 1.0, 100])

p = P()
p.get_numpy_fn_param('a')[1] += 5
print(p.get_numpy_fn_param('a'))  # prints [0.0, 6.0, 100]

SpanArangeParam not handled in parameter_interface

Can't turn Params classes including a SpanArangeParam into an argparse instance

auto append [start, stop, etc.] to param help if expand is False