.. _parameterized_pipelines: # Customizing your own model parameters This page delves into some of the more powerful things you can do with pipelines. It is aimed at people who have already completed the :ref:`advanced_pipelines` tutorial. ## Pipeline model parameters You should be aware that each pipeline _step_ has parameters, but the pipeline _model_ itself can also have its own parameters. Unlike step parameters, you have complete control over defining the model parameters yourself. The model parameters are essentially custom variables that get replaced at run-time. These variables are denoted by a `$` in the pipeline file. The model parameters can customize the behaviour of the model across _all_ input data. Model parameters can encapsulate an assumption you are making about your model, which you may want to vary. As a simple example, say we only wanted to run our model for one particular construction type at a time. But we wanted to be able to _vary_ that construction type easily, without editing the pipeline file each time. We can turn the construction type we filter on into a `construction` parameter, e.g. ```none input('assets.csv', name: 'asset') -> filter(filter: asset.construction = $construction) ``` When we run our model, we can then specify what specific construction type we are interested in, e.g. ```none riskscape model run my_pipeline --param "construction='steel'" ``` You can define the default values for your parameters in your model's INI definition. Just use `param.<name> = <default>`, e.g. ```ini [model my_pipeline] framework = pipeline location = my_pipeline.txt param.construction = 'timber' ``` .. _parameter_properties: ### Parameter properties When you want to share a model with other users, you can restrict what sort of values the user can specify for certain parameters. For example, you could specify that a parameter called `damage_ratio` always had to be a *numeric*, or always had to be within a certain range, such as 0.0 to 1.0. You can specify what *properties* a parameter is expected to have in your `project.ini` file. For example, the `damage_ratio` parameter would look like this: ``` param.damage_ratio.properties = numeric, min: 0.0, max: 1.0 ``` If the user tries to enter anything that is *not* a numerical value between 0.0 and 1.0, then they will get an error when they try to run the model. The parameter properties supported on the RiskScape CLI include: *bookmark*, *file*, *integer*, *list*, *min*, *max*, *numeric*, and *text*. .. tip:: Specifying the ``text``, ``file`` or ``bookmark`` properties means that the user no longer needs to use ``''`` single-quotes when entering a new parameter value. The ``file`` property behaves identically to the ``text`` property, but allows the RiskScape Platform to display a file chooser. ### Parameter Templates In your INI files, you can also define parameter templates. This saves you from having to define the same parameter in several different models. A parameter template will automatically be used if it has the same name as the parameter, but you can also provide the name of a template to use. E.g. `param.buildings.template = city_buildings` For example: ```ini [model one] framework = pipeline pipeline = input($people) -> sort(name) param.people = ./data/people.csv param.people.properties = bookmark [model two] framework = pipeline pipeline = input($people) -> sort(age) param.people = ./data/people.csv param.people.properties = bookmark ``` could be replaced by: ```ini [parameter people] default = ./data/people.csv properties = bookmark [model one] framework = pipeline pipeline = input($people) -> sort(name) [model two] framework = pipeline pipeline = input($people) -> sort(age) ``` This saves repetition if you have several different models that all use the same parameters over and over. Parameter templates are also useful for configuring how model parameters are presented to the user in the RiskScape Platform web interface. More information about parameter templates can be found in the [RiskScape Platform documentation](https://riskscape.nz/docs/project-setup/ui-parameters.html?highlight=template#parameter-templates) .. _function_parameters: ### Function parameters Model parameters also work if the assumption you are making is in your _function_. You simply need to pass the model parameter through to your function. To do this you add an extra argument to your function. The new argument will be a `Struct` and contain the attributes you want to vary. For example, say our Kaijū function was making an assumption on how resilient timber buildings are to Kaijū attacks. We can modify our function to take an additional 'options' argument, which is a `Struct` with a `timber_resilience` attribute. The (abridged) function would look like this: ```python from nz.org.riskscape.engine.types import Types from nz.org.riskscape.engine.types import Struct ID = 'kaiju_stomp' DESCRIPTION = 'Models damage from a Kaiju stomping a building' ARGUMENT_TYPES = ['building', 'kaiju_attack', \ Struct.of('timber_resilience', Types.INTEGER) ] RETURN_TYPE = 'building_attack_outcome' def function(building, stomp, options): if building.get('construction') == 'timber': # the resilience for timber buildings can be passed # into our function, making it easier to vary it resilience = options.get('timber_resilience') elif building.get('construction') == 'concrete': # whereas the resilience for concrete buildings is # still hard-coded in the function itself resilience = 5 # ... ``` Then to call our function from our pipeline code, we create a new `Struct` with a `timber_resilience` attribute. In this case, the value for `timber_resilience` is a model parameter that we can now change on the fly whenever we run our model. ```none select({*, kaiju_stomp(asset, hazard, { timber_resilience: $timber_resilience }) as damage ) as compute consequence ``` The 'options' `Struct` could hold many different attributes, if there are many assumptions in your function that you want to vary. ### Resolving file paths By default, RiskScape will try to resolve a model parameter file path relative to where the model is defined. For example, say we have the following `PopulationModels/project.ini` file: ```ini [model one] framework = pipeline pipeline = input($people) -> sort(name) param.people = data/people.csv param.people.properties = bookmark ``` The `$people` parameter will be resolved relative to the `project.ini` file, so the model will try to use the file path `PopulationModels/data/people.csv`. Sometimes you may want to share the same parameters, types, or bookmarks across multiple projects. For example, say you had a `C:\RiskScape_Projects\Shared\project.ini` file that defined a common dataset that you wanted to use across all your models: ```ini [parameter people] default = ./data/people.csv properties = bookmark ``` You could then _import_ that `project.ini` file, and all the definitions it contains, into your `PopulationModels/project.ini` file: ```ini [project] import = C:\RiskScape_Projects\Shared\project.ini [model one] framework = pipeline pipeline = input($people) -> sort(name) ``` This imports the `$people` parameter template, so you do not have to define it again in your `PopulationModels` project. .. note:: When ``properties = bookmark`` or ``file`` is used, and the default value is a file path that begins with the special sequence ``./``, it will be resolved relative to the file it is *defined* in, not the model where it is *used*. This means your models can use templates (and their default file values) when the template is in a different folder to the main project file. In the above example, because `./` is used, the `$people` parameter is resolved relative to the `Shared` directory `project.ini` file where it is defined (i.e. `C:\RiskScape_Projects\Shared\data\people.csv`). If `./` were *not* used in the default value (i.e. `data/people.csv` instead of `./data/people.csv`), then the file path would be resolved relative to the `PopulationModels` directory `project.ini` file where the parameter is used (i.e. `PopulationModels\data\people.csv`). ## A working example Click [Here](../pipeline-parameters.zip) to download a working example of a parameterized pipeline. Open the `pipeline.txt` and `project.ini` files and familiarize yourself with them. You can see that the model defines 'resilience' parameters for the three construction materials of interest. These parameters then get passed directly to the `kaiju_stomp` function. To run the model with default parameter values, use the command: ```none riskscape model run demo ``` ### Reproducible models Model parameters can make it harder to tell later what actual parameter values were used to produce a certain set of results. Fortunately, RiskScape always saves the _actual_ parameter values it used in the output directory. Have a look at the output directory created by the last 'model run' command. It should contain a `pipeline.txt` file - this contains the pipeline code with the `$` parameters replaced with the actual values used by the 'model run' command. Any piece of raw pipeline code (i.e. without parameters) can be executed using the `riskscape pipeline eval` command. So you can use the `pipeline.txt` file in the output directory to re-run the exact same pipeline again in the future, e.g. ```none riskscape pipeline eval <pipeline.txt> ``` Try this now and check you get the same results. ### Parameter INI files When your model has several different parameters, rather than specifying each parameter on the command line, you can just specify _one_ INI file that contains all the parameter values you want to use. To pass the INI file to the 'model run' command, use the `--parameters` CLI option (note the 's' on the end). For example: ```none riskscape model run demo --parameters stronger-resilience.ini ``` Using INI files can be helpful if you want to vary several different assumptions in a related way. For example, this project contains two different INI files that make different assumptions about a building's resilience: `stronger-resilience.ini` and `weaker-resilience.ini`. Try running the model with each of these INI files and see what difference it makes to the overall damage.