Loading from dict/config file

Loading parameters from a dict

You can skip all those add_param and add_command calls, and load the parameter/command definition from a python dictionary.

There are two way to do that:

Express dict

The express way is a depth 1 dictionary where all the keys are flattened.

from pyparam import Params

params_def = {
    "i": 1, # default value
    "i.aliases": ["int"],
    "i.type": "int",
    "i.desc": ["An int parameter"],
    # ...
    "f": 1.0,
    "f.aliases": ["float"],
    "f.type": "float",
    "f.desc": ["A float parameter"]
}

params = Params()
params.file_dict(params_def)
# This does the same as
params.add_param("i,int", default=1, type=int, desc=["An int parameter"])
params.add_param("f,float", default=1.0, type=float, desc=["A float parameter"])

Express dict has some limitations:

No sub-command specifications are allowed
No namespace parameters

Full specification dict

One can use the full specification dictionary to have sub-command and namespace parameter definitions enabled:

params_def = {
    "params": {
        "i": {
            "default": 1,
            "aliases": ["int"],
            "type": "int",
            "desc": ["An int parameter"]
        },
        "f": {
            "default": 1.0,
            "aliases": ["float"],
            "type": "float",
            "desc": ["A float parameter"]
        }
    },
    "commands": {
        "cmd": {
            # aliases: [...]
            # prefix: ...
            "params": {
                "c": {
                    "default": "medium",
                    "type": "choice",
                    "choices": ["small", "medium", "large"]
                },
                "config": {
                    "type": "ns",
                    "desc": "A set of configurations"
                },
                "config.ncores": {
                    "default": 4,
                    "desc": "Number of cores"
                }
            }
        }
    }
}
params.from_dict(params_def)
parsed = params.parse()

This:

prog -i 1 -f 2.0 cmd -c large --config.ncores 1

will produce:

Namesapce(__command__='cmd',
          i=1,
          f=2.0,
          cmd=Namesapce(c='large', config=Namesapce(ncores=1)))

Hiding parameters from help page

If you don't want a parameter to be shown in help page, especially when it not required, just pass show=False to it. In the dictionary, you can just do:

params_def = {
    "x": 1,
    "x.show": False
}

Or with full specification:

params_def = {
    "params": {
        "x": {
            "default": 1,
            "show": False
        }
    }
}

Or if you want to hide all the parameters from a dict except those with show=True explictly:

params.from_dict(params_def, show=False)

Loading from a configuration file

Parameters can also be loaded from a configuration file that is supported by python-simpleconf

Those configuration files will be first read and turned into python dictionary, and then parameter definitions will be loaded using params.from_dict.

There is the toml file that are the same as the above dict examples.

Express:

i = 1
"i.aliases" = ["int"]
"i.type" = "int"
"i.desc" = ["An int parameter"]
# ...
f = 1.0
"f.aliases" = ["float"]
"f.type" = "float"
"f.desc" = ["A float parameter"]

Full specification

[params.i]
default = 1
aliases = ["int"]
type = "int"
desc = "An int parameter"

[params.f]
default = 1.0
aliases = ["float"]
type = "float"
desc = ["A float parameter"]

[commands.cmd.params.c]
default = "medium"
type = "choice"
choices = ["small", "medium", "large"]

[commands.cmd.params.config]
type = "ns"
desc = "A set of configurations"

  [commands.cmd.params."config.ncores"]
  default = 4
  desc = "Number of cores"

Loading from a configuration file specified by a command line argument

You may also ask user to specify a configuration file to load parameter/command definitions from.

Something like:

prog --config-file params_def.toml -i 1 -f 2.0 ...

Then the parameter/command definitions will be loaded from params_def.toml.

To do it:

params.from_arg('config-file')

First, pyparam will scan the sys.argv to see if any item matches --config-file. If it does, then the next item (params_def.toml) will be used.

To show the paramter config-file in the help page, you can also specify some desciptions to it:

params.from_arg('config-file', desc=..., group=...)

To use full configuration of a parameter, one can also add it as a parameter first:

configfile = params.add_params('config-file', ...)
# then
params.from_arg(configfile)

Tip

from_dict, from_file and from_arg have a force argument to force adding parameters/commands, if they exist. Basically later ones will overwrite existing ones.

Dumping all params to a dict or file

You can dump all parameters and commands into a file, which later on can be loaded by params. This is useful when the parameters/commands take time to load.

You can just do params.to_dict() or params.to_file(...). You can directly specify the file path, the type will be automatically detected from the extension. Currently supported extensions are .yaml, .yml, .json and .toml. Otherwise you can specify the type by cfgtype argument. Supported cfgtypes are yaml, json and toml.

Skipping command line argument parsing

pyparam can also work as a value holder, which does not parse anything from the command line, but just use the predefined values. This requires all the parameters are optional and no command has been defined.

For example:

params.add_param('i', default=0)
params.values() # Namespace(i=0)

Parameter callbacks are still applicable here.