pysd documentation - read the docs · 2021. 1. 28. · pysd documentation, release 1.0.1...

PySD DocumentationRelease 1.7.0

James Houghton

Jul 02, 2021

Contents

1 Contents: 31.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Basic Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Advanced Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.4 Command Line Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.5 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.6 User Functions Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161.7 Developer Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2 Additional Resources 732.1 PySD Cookbook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732.2 Contributing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732.3 Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3 Indices and tables 75

Python Module Index 77

Index 79

i

PySD Documentation, Release 1.7.0

Simulating System Dynamics Models in Python

This project is a simple library for running System Dynamics models in python, with the purpose of improving inte-gration of Big Data and Machine Learning into the SD workflow.

PySD translates Vensim or XMILE model files into python modules, and provides methods to modify, simulate, andobserve those translated models.

Contents 1


2 Contents

CHAPTER 1

Contents:

1.1 Installation

1.1.1 Installing via pip

To install the PySD package from the Python package index into an established Python environment, use the pipcommand:

pip install pysd

1.1.2 Installing from source

To install from the source, clone the project with git:

git clone https://github.com/JamesPHoughton/pysd.git

Or download the latest version from the project webpage: https://github.com/JamesPHoughton/pysd

In the source directory use the command

python setup.py install

1.1.3 Required Dependencies

PySD was originally built on python 2.7. Hoewever, the last version requires at least python 3.7.

PySD calls on the core Python data analytics stack, and a third party parsing library:

• Numpy

• Scipy

• Pandas

3

https://github.com/JamesPHoughton/pysd


• Parsimonious

• xarray

• xlrd

• lxml

• regex

• chardet

• black

• openpyxl

• progressbar

These modules should build automatically if you are installing via pip. If you are building from the source code, or ifpip fails to load them, they can be loaded with the same pip syntax as above.

1.1.4 Optional Dependencies

In order to plot results from the model as shown in basic usage:

• Matplotlib

These Python libraries bring additional data analytics capabilities to the analysis of SD models:

• PyMC: a library for performing Markov chain Monte Carlo analysis

• Scikit-learn: a library for performing machine learning in Python

• NetworkX: a library for constructing networks

• GeoPandas: a library for manipulating geographic data

Additionally, the System Dynamics Translator utility developed by Robert Ward is useful for translating models fromother system dynamics formats into the XMILE standard, to be read by PySD.

These modules can be installed using pip with syntax similar to the above.

1.1.5 Additional Resources

The PySD Cookbook contains recipes that can help you get set up with PySD.

1.2 Basic Usage

1.2.1 Importing a model and getting started

To begin, we must first load the PySD module, and use it to import a supported model file:

import pysdmodel = pysd.read_vensim('Teacup.mdl')

This code creates an instance of the PySD class loaded with an example model that we will use as the system dynamicsequivalent of ‘Hello World’: a cup of tea cooling to room temperature.

4 Chapter 1. Contents:

https://github.com/JamesPHoughton/PySD-Cookbook


To view a synopsis of the model equations and documentation, call the doc() method of the model class. This willgenerate a listing of all the model elements, their documentation, units, equations, and initial values, where appropriate.Here is a sample from the teacup model:

>>> print model.doc()

Note: You can also load an already translated model file, what will be faster as you will load a Python file:

import pysdmodel = pysd.load('Teacup.py')

Note: The functions read_vensim(), read_xmile() and load() have optional arguments for advancedusage, you can check the full description in User Functions Reference or using help() e.g.:

import pysdhelp(pysd.load)

1.2.2 Running the Model

The simplest way to simulate the model is to use the run() command with no options. This runs the model withthe default parameters supplied by the model file, and returns a Pandas dataframe of the values of the stocks at everytimestamp:

>>> stocks = model.run()

t teacup_temperature0.000 180.0000000.125 178.6335560.250 177.2840910.375 175.951387...

Pandas gives us simple plotting capability, so we can see how the cup of tea behaves:

stocks.plot()plt.ylabel('Degrees F')plt.xlabel('Minutes')

1.2. Basic Usage 5

https://docs.python.org/2.7/library/functions.html#help


To show a progressbar during the model integration the progress flag can be passed to the run() command, progress-bar package is needed:

>>> stocks = model.run(progress=True)

1.2.3 Outputting various run information

The run() command has a few options that make it more useful. In many situations we want to access componentsof the model other than merely the stocks – we can specify which components of the model should be included inthe returned dataframe by including them in a list that we pass to the run() command, using the return_columnskeyword argument:

>>> model.run(return_columns=['Teacup Temperature', 'Room Temperature'])

t Teacup Temperature Room Temperature0.000 180.000000 75.00.125 178.633556 75.00.250 177.284091 75.00.375 175.951387 75.0...

If the measured data that we are comparing with our model comes in at irregular timestamps, we may want to samplethe model at timestamps to match. The run() function gives us this ability with the return_timestamps keywordargument:

>>> model.run(return_timestamps=[0,1,3,7,9.5,13.178,21,25,30])

t Teacup Temperature0.0 180.0000001.0 169.5321193.0 151.4900027.0 124.6243859.5 112.541515...

1.2.4 Retrieving totally flat dataframe

The subscripted variables, in general, will be returned as xarray.DataArray*s in the output *pandas.DataFrame. Toget a totally flat dataframe, like Vensim outuput the flatten=True when calling the run function:

>>> model.run(flatten=True)



1.2.5 Setting parameter values

In many cases, we want to modify the parameters of the model to investigate its behavior under different assumptions.There are several ways to do this in PySD, but the run() function gives us a convenient method in the paramskeyword argument.

This argument expects a dictionary whose keys correspond to the components of the model. The associated values caneither be a constant, or a Pandas series whose indices are timestamps and whose values are the values that the modelcomponent should take on at the corresponding time. For instance, in our model we can set the room temperature to aconstant value:

model.run(params={'Room Temperature': 20})

Alternately, if we believe the room temperature is changing over the course of the simulation, we can give the runfunction a set of time-series values in the form of a Pandas series, and PySD will linearly interpolate between thegiven values in the course of its integration:

import pandas as pdtemp = pd.Series(index=range(30), data=range(20, 80, 2))model.run(params={'Room Temperature':temp})

If the parameter value to change is a subscripted variable (vector, matrix. . . ), there are three different options to setnew value. Suposse we have ‘Subscripted var’ with dims ['dim1', 'dim2'] and coordinates {'dim1': [1,2], 'dim2': [1, 2]}. A constant value can be used and all the values will be replaced:

model.run(params={'Subscripted var': 0})

A partial xarray.DataArray can be used, for example a new variable with ‘dim2’ but not ‘dim2’, the result will berepeated in the remaining dimensions:

import xarray as xrnew_value = xr.DataArray([1,5], {'dim2': [1, 2]}, ['dim2'])model.run(params={'Subscripted var': new_value})

Same dimensions xarray.DataArray can be used (recommended):

import xarray as xrnew_value = xr.DataArray([[1,5],[3,4]], {'dim1': [1, 2], 'dim2': [1, 2]}, ['dim1',→˓'dim2'])model.run(params={'Subscripted var': new_value})

In the same way, a Pandas series can be used with constan values, partially defined xarray.DataArrays or same dimen-sions xarray.DataArrays.

Note: That once parameters are set by the run command, they are permanently changed within the model. Wecan also change model parameters without running the model, using PySD’s set_components(params={})method, which takes the same params dictionary as the run function. We might choose to do this in situations wherewe’ll be running the model many times, and only want to spend time setting the parameters once.

Note: If you need to know the dimensions of a variable, you can check them by usingget_coords(variable__name) function:

>>> model.get_coords('Room Temperature')

(continues on next page)

1.2. Basic Usage 7


(continued from previous page)

None

>>> model.get_coords('Subscripted var')

({'dim1': [1, 2], 'dim2': [1, 2]}, ['dim1', 'dim2'])

this will return the coords dictionary and the dimensions list if the variable is subscripted or ‘None’ if the variable isan scalar.

Note: If you change the value of a lookup function by a constant, the constant value will be used always. If apandas.Series is given the index and values will be used for interpolation when the function is called in the model,keeping the arguments that are included in the model file.

If you change the value of any other variable type by a constant, the constant value will be used always. If a pan-das.Series is given the index and values will be used for interpolation when the function is called in the model, usingthe time as argument.

If you need to know if a variable takes arguments, i.e., if it is a lookup variable, you can check it by usingget_args(variable__name) function:

>>> model.get_args('Room Temperature')

[]

>>> model.get_args('Growth lookup')

['x']

1.2.6 Setting simulation initial conditions

Finally, we can set the initial conditions of our model in several ways. So far, we’ve been using the default value forthe initial_condition keyword argument, which is ‘original’. This value runs the model from the initial conditions thatwere specified originally by the model file. We can alternately specify a tuple containing the start time and a dictionaryof values for the system’s stocks. Here we start the model with the tea at just above freezing:

model.run(initial_condition=(0, {'Teacup Temperature':33}))

The new value setted can be a xarray.DataArray as it is explained in the previous section.

Additionally we can run the model forward from its current position, by passing the initial_condition argument thekeyword ‘current’. After having run the model from time zero to thirty, we can ask the model to continue runningforward for another chunk of time:

model.run(initial_condition='current',return_timestamps=range(31,45))

The integration picks up at the last value returned in the previous run condition, and returns values at the requestedtimestamps.

There are times when we may choose to overwrite a stock with a constant value (ie, for testing). To do this, we just usethe params value, as before. Be careful not to use ‘params’ when you really mean to be setting the initial condition!



1.2.7 Querying current values

We can easily access the current value of a model component by calling its associated method (using python safenames) in the components subclass. For instance, to find the temperature of the teacup, we simply call:

model.components.teacup_temperature()

1.2.8 Supported functions

Vensim functions include:

Vensim Python TranslationCOS np.cosEXP np.expMIN min=IF THEN ELSE functions.if_then_elseLN np.logPULSE TRAIN functions.pulse_trainRAMP functions.rampINTEGER intTAN np.tanPI np.pi= ==< <> >MODULO np.modARCSIN np.arcsinABS abs^ **LOGNORMAL np.random.lognormalMAX maxSQRT np.sqrtARCTAN np.arctanARCCOS np.arccosRANDOM NORMAL functions.bounded_normalRANDOM UNIFORM np.random.randDELAY1 functions.DelayDELAY3 functions.DelayDELAY N functions.DelayNDELAY FIXED functions.DelayFixedSAMPLE IF TRUE functions.SampleIfTrueSMOOTH3 functions.SmoothSMOOTH N functions.Smooth

Continued on next page

1.2. Basic Usage 9


Table 1 – continued from previous pageVensim Python TranslationSMOOTH functions.SmoothINITIAL functions.InitialXIDZ functions.XIDZZIDZ functions.XIDZVMIN functions.vminVMAX functions.vmaxSUM functions.sumPROD functions.prodGET XLS DATA external.ExtDataGET DIRECT DATA external.ExtDataGET XLS LOOKUPS external.ExtLookupGET DIRECT LOOKUPS external.ExtLookupGET XLS CONSTANTS external.ExtConstantGET DIRECT CONSTANTS external.ExtConstantGET XLS SUBSCRIPT external.ExtSubscriptGET DIRECT SUBSCRIPT external.ExtSubscript

np corresponds to the numpy package

1.3 Advanced Usage

The power of PySD, and its motivation for existence, is its ability to tie in to other models and analysis packages inthe Python environment. In this section we’ll discuss how those connections happen.

1.3.1 Replacing model components with more complex objects

In the last section we saw that a parameter could take on a single value, or a series of values over time, with PySDlinearly interpolating between the supplied time-series values. Behind the scenes, PySD is translating that constant ortime-series into a function that then goes on to replace the original component in the model. For instance, in the teacupexample, the room temperature was originally a function defined through parsing the model file as something similarto:

def room_temperature():return 75

However, when we made the room temperature something that varied with time, PySD replaced this function withsomething like:

def room_temperature():return np.interp(t, series.index, series.values)

This drew on the internal state of the system, namely the time t, and the time-series data series that that we wanted tovariable to represent. This process of substitution is available to the user, and we can replace functions ourselves, ifwe are careful.

Because PySD assumes that all components in a model are represented as functions taking no arguments, any compo-nent that we wish to modify must be replaced with a function taking no arguments. As the state of the system and allauxiliary or flow methods are public, our replacement function can call these methods as part of its internal structure.

In our teacup example, suppose we didn’t know the functional form for calculating the heat lost to the room, butinstead had a lot of data of teacup temperatures and heat flow rates. We could use a regression model (here a support



vector regression from Scikit-Learn) in place of the analytic function:

from sklearn.svm import SVRregression = SVR()regression.fit(X_training, Y_training)

Once the regression model is fit, we write a wrapper function for its predict method that accesses the input componentsof the model and formats the prediction for PySD:

def new_heatflow_function():""" Replaces the original flowrate equation

with a regression model"""tea_temp = model.components.teacup_temperature()room_temp = model.components.room_temperature()return regression.predict([room_temp, tea_temp])[0]

We can substitute this function directly for the heat_loss_to_room model component:

model.components.heat_loss_to_room = new_heatflow_function

If you want to replace a subscripted variable, you need to ensure that the output from the new function is thesame as the previous one. You can check the current coordinates and dimensions of a component by usingget_coords(variable_name) as it is explained in basic usage.

1.3.2 Starting simulations from an end-state of another simulation

The current state of a model can be saved in a pickle file using the :py:data:‘.export()‘method:

import pysdmodel1 = pysd.read_vensim("my_model.mdl")model1.run(final_time=50)model1.export("final_state.pic")

Then the exported data can be used in another session:

import pysdmodel2 = pysd.load("my_model.py")model2 = run(initial_condition="final_state.pic", return_timestamps=[55, 60])

the new simulation will have initial time equal to 50 with the saved values from the previous one.

Note: You can set the exact final time of the simulation using the final_time argument. If you want to avoid returningthe dataframe of the stocks you can use return_timestamps=[]:

model1.run(final_time=50, return_timestamps=[])

Note: The changes done with params arguments are not ported to the new model (model2) object that you initializewith final_state.pic. If you want to keep them, you need to call run with the same params values as in the originalmodel (model1).

1.3. Advanced Usage 11


Warning: Exported data is saved and loaded using pickle, this data can be not compatible with future versions ofPySD or xarray. In order to prevent data losses save always the source code.

1.4 Command Line Usage

1.4.1 Basic command line usage

Most of the features available in basic usage are also available using command line. Running:

python -m pysd Teacup.mdl

will translate Teacup.mdl to Teacup.py and run it with the default values. The output will be saved inTeacup_output_%Y_%m_%d-%H_%M_%S_%f.tab. The command line accepts several arguments, this can bechecked using the -h/–help argument:

python -m pysd --help

Set output file

In order to set the output file -o/–output-file argument can be used:

python -m pysd -o my_output_file.csv Teacup.mdl

Note: The output file can be a .csv or .tab.

Note: If -o/–output-file is not given the output will be saved in a file that starts with the model file name and has atime stamp to avoid overwritting files.

Activate progress bar

The progress bar can be activated using -p/–progress command:

python -m pysd --progress Teacup.mdl

Only translate model file

To only translate the model file, it does not run the model, -t/–trasnlate command is provided:

python -m pysd --translate Teacup.mdl

1.4.2 Outputting various run information

The output number of variables can be modified bu passing them as arguments separated by commas, using -r/return_columns argument:


https://docs.python.org/3/library/pickle.html


python -m pysd -r 'Teacup Temperature, Room Temperature' Teacup.mdl

Note that the argument passed after -r/return_columns should be inside ‘’ to be properly read. Moreover each variablename must be split with commas.

Sometimes, the variable names have special characteres, such as commas, which can happen when trying to return avariable with subscripts. In this case whe can save a .txt file with one variable name per row and use it as an argument:

python -m pysd -r output_selected_vars.txt Teacup.mdl

-R/–return-timestamps command can be used to set the return timestamps:

python -m pysd -R '0, 1, 3, 7, 9.5, 13.178, 21, 25, 30' Teacup.mdl

Note: Each time stamp should be able to be computed as initial_time + N x time_step, where N is an integer.

Note: The time outputs can be also modified using the model control variables, explained in next section.

1.4.3 Modify model variables

Modify model control variables

The model control variables such as the initial time. final time, time step and saving step can be easily modified usingthe -I/–initial_time, -F/–final-time, -T/–time-step and -S/–saveper commands respectively. For example:

python -m pysd -I 2005 --final-time=2010 --time-step=1 Teacup.mdl

will set the initial time to 2005, the final time to 2010 and the time step to 1.

Note: If -R/–return-timestamps argument is used the final time and saving step will be ignored.

Modify model variables

In order to modify the values of model variables they can be passed after the model file:

python -m pysd Teacup.mdl 'Room Temperature'=5

this will set Room Temperature variable to the constant value 5. A series can be also passed to change a value of avalue to a time dependent series or the interpolation values of a lookup variable two lists of the same length must begiven:

python -m pysd Teacup.mdl 'Temperature Lookup=[[1, 2, 3, 4], [10, 15, 17, 18]]'

The first list will be used for the time or x values and the second for the data. See setting parameter values in basicusage for more information.

Note: If a variable name or the right hand side are defined with whitespaces it is needed to add ‘’ define it, as hasbeen done in the last example.

1.4. Command Line Usage 13


Several variables can be changed at the same time, e.g.:

python -m pysd Teacup.mdl 'Room Temperature'=5 temperature_lookup='[[1, 2, 3, 4], [10,→˓ 15, 17, 18]]' 'Initial Temperature'=5

Modify initial conditions of model variables

Sometimes we do not want to change a variable value to a constant but change its initial value, for example changeinitial value of a stock object, this can be similarly done to the previos case but using ‘:’ instead of ‘=’:

python -m pysd Teacup.mdl 'Teacup Temperature':30

this will set initial Teacup Temperature to 30.

1.4.4 Putting It All Together

Several commands can be used together, first need to add optional arguments, those starting with ‘-’, next the modelfile, and last the variable or variables to change, for example:

python -m pysd -o my_output_file.csv --progress --final-time=2010 --time-step=1→˓Teacup.mdl 'Room Temperature'=5 temperature_lookup='[[1, 2, 3, 4], [10, 15, 17, 18]]→˓' 'Teacup Temperature':30

will save step 1 outputs until 2010 in my_output_file.csv, showing a progressbar during integration and settung foo to5 and temperature_lookup to ((1, 10), (2, 15), (3, 17), (4, 18)) and initial Teacup Temperature to 30.

1.5 Tools

Some tools are given with the library.

1.5.1 Benchmarking

Benchmarking tools for testing and comparing outputs between different files. Some of these functions are also usedfor testing.

pysd.tools.benchmarking.assert_allclose(x, y, rtol=1e-05, atol=1e-05)Asserts if all numeric values from two arrays are close.

Parameters

• x (ndarray) – Expected value.

• y (ndarray) – Actual value.

• rtol (float (optional)) – Relative tolerance on the error. Default is 1.e-5.

• atol (float (optional)) – Absolut tolerance on the error. Default is 1.e-5.

Returns

Return type None


https://docs.python.org/2.7/library/functions.html#floathttps://docs.python.org/2.7/library/functions.html#floathttps://docs.python.org/2.7/library/constants.html#None


pysd.tools.benchmarking.assert_frames_close(actual, expected, assertion=’raise’, preci-sion=2, **kwargs)

Compare DataFrame items by column and raise AssertionError if any column is not equal.

Ordering of columns is unimportant, items are compared only by label. NaN and infinite values are supported.

Parameters

• actual (pandas.DataFrame) – Actual value from the model output.

• expected (pandas.DataFrame) – Expected model output.

• assertion (str (optional)) – “raise” if an error should be raised when not ableto assert that two frames are close. Otherwise, it will show a warning message. Default is“raise”.

• precision (int (optional)) – Precision to print the numerical values of assertionmessage. Default is 2.

• kwargs – Optional rtol and atol values for assert_allclose.

Examples

>>> assert_frames_close(... pd.DataFrame(100, index=range(5), columns=range(3)),... pd.DataFrame(100, index=range(5), columns=range(3)))

>>> assert_frames_close(... pd.DataFrame(100, index=range(5), columns=range(3)),... pd.DataFrame(110, index=range(5), columns=range(3)),... rtol=.2)

>>> assert_frames_close(... pd.DataFrame(100, index=range(5), columns=range(3)),... pd.DataFrame(150, index=range(5), columns=range(3)),... rtol=.2) # doctest: +IGNORE_EXCEPTION_DETAILTraceback (most recent call last):...AssertionError:Column '0' is not close.Expected values:

[150, 150, 150, 150, 150]Actual values:

[100, 100, 100, 100, 100]Difference:

[50, 50, 50, 50, 50]

>>> assert_frames_close(... pd.DataFrame(100, index=range(5), columns=range(3)),... pd.DataFrame(150, index=range(5), columns=range(3)),... rtol=.2, assertion="warn")...UserWarning:Column '0' is not close.Expected values:

[150, 150, 150, 150, 150]Actual values:

[100, 100, 100, 100, 100]

(continues on next page)

1.5. Tools 15

https://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#int


(continued from previous page)

Difference:[50, 50, 50, 50, 50]

References

Derived from: http://nbviewer.jupyter.org/gist/jiffyclub/ac2e7506428d5e1d587b

pysd.tools.benchmarking.load_outputs(file_name, transpose=False)Load outputs file

Parameters

• file_name (str) – Output file to read. Must be csv or tab.

• transpose (bool (optional)) – If True reads transposed outputs file, i.e. one vari-able per row. Default is False.

Returns A pandas.DataFrame with the outputs values.

Return type pandas.DataFrame

pysd.tools.benchmarking.runner(model_file, canonical_file=None, transpose=False)Translates and runs a model and returns its output and the canonical output.

Parameters

• model_file (str) – Name of the original model file. Must be ‘.mdl’ or ‘.xmile’.

• canonical_file (str or None (optional)) – Canonical output file to read. IfNone, will search for ‘output.csv’ and ‘output.tab’ in the model directory. Default is None.

• transpose (bool (optional)) – If True reads transposed canonical file, i.e. onevariable per row. Default is False.

Returns output, canon – pandas.DataFrame of the model output and the canonical output.

Return type (pandas.DataFrame, pandas.DataFrame)

1.6 User Functions Reference

These are the primary functions that control model import and execution.

pysd.read_vensim(mdl_file, initialize=True, missing_values=’warning’)Construct a model from Vensim .mdl file.

Parameters

• mdl_file (str) – The relative path filename for a raw Vensim .mdl file

• initialize (bool (optional)) – If False, the model will not be initialize when itis loaded. Default is True

• missing_values (str ("warning", "error", "ignore", "keep")(optional)) – What to do with missing values. If “warning” (default) shows a warningmessage and interpolates the values. If “raise” raises an error. If “ignore” interpolates thevalues without showing anything. If “keep” it will keep the missing values, this option maycause the integration to fail, but it may be used to check the quality of the data.

Returns


http://nbviewer.jupyter.org/gist/jiffyclub/ac2e7506428d5e1d587bhttps://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#boolhttps://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/constants.html#Nonehttps://docs.python.org/2.7/library/functions.html#boolhttps://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#boolhttps://docs.python.org/2.7/library/functions.html#str


• model (a PySD class object) – Elements from the python model are loaded into the PySDclass and ready to run

• initialize (bool (optional)) – If False, the model will not be initialize when it is loaded.Default is True

Examples

>>> model = read_vensim('../tests/test-models/samples/teacup/teacup.mdl')

pysd.read_xmile(xmile_file, initialize=True, missing_values=’warning’)Construct a model from .xmile file.

Parameters

• xmile_file (str) – The relative path filename for a raw .xmile file



Returns model – Elements from the python model are loaded into the PySD class and ready to run

Return type a PySD class object

Examples

>>> model = read_xmile('../tests/test-models/samples/teacup/teacup.xmile')

pysd.load(py_model_file, initialize=True, missing_values=’warning’)Load a python-converted model file.

Parameters

• py_model_file (str) – Filename of a model which has already been converted into apython format.



Examples

>>> model = load('../tests/test-models/samples/teacup/teacup.py')

1.6. User Functions Reference 17

https://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#boolhttps://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#boolhttps://docs.python.org/2.7/library/functions.html#str


1.7 Developer Documentation

1.7.1 About the Project

Motivation: The (coming of) age of Big Data

The last few years have witnessed a massive growth in the collection of social and business data, and a correspondingboom of interest in learning about the behavior of social and business systems using this data. The field of ‘datascience’ is developing a host of new techniques for dealing with and analyzing data, responding to an increase indemand for insights and the increased power of computing resources.

So far, however, these new techniques are largely confined to variants of statistical summary, categorization, andinference; and if causal models are used, they are generally static in nature, ignoring the dynamic complexity andfeedback structures of the systems in question. As the field of data science matures, there will be increasing demandfor insights beyond those available through analysis unstructured by causal understanding. At that point data scientistsmay seek to add dynamic models of system structure to their toolbox.

The field of system dynamics has always been interested in learning about social systems, and specializes in under-standing dynamic complexity. There is likewise a long tradition of incorporating various forms of data into systemdynamics models.3 While system dynamics practice has much to gain from the emergence of new volumes of socialdata, the community has yet to benefit fully from the data science revolution.

There are a variety of reasons for this, the largest likely being that the two communities have yet to commingle to agreat extent. A further, and ultimately more tractable reason is that the tools of system dynamics and the tools of dataanalytics are not tightly integrated, making joint method analysis unwieldy. There is a rich problem space that dependsupon the ability of these fields to support one another, and so there is a need for tools that help the two methodologieswork together. PySD is designed to meet this need.

General approaches for integrating system dynamic models and data analytics

Before considering how system dynamics techniques can be used in data science applications, we should consider thevariety of ways in which the system dynamics community has traditionally dealt with integration of data and models.

The first paradigm for using numerical data in support of modeling efforts is to import data into system dynamicsmodeling software. Algorithms for comparing models with data are built into the tool itself, and are usable througha graphical front-end interface as with model fitting in Vensim, or through a programming environment unique to thetool. When new techniques such as Markov chain Monte Carlo analysis become relevant to the system dynamicscommunity, they are often brought into the SD tool.

This approach appropriately caters to system dynamics modelers who want to take advantage of well-establisheddata science techniques without needing to learn a programming language, and extends the functionality of systemdynamics to the basics of integrated model analysis.

A second category of tools uses a standard system dynamics tool as a computation engine for analysis performed ina coding environment. This is the approach taken by the Exploratory Modeling Analysis (EMA) Workbench6, orthe Behavior Analysis and Testing Software (BATS)7. This first step towards bringing system dynamics to a moreinclusive analysis environment enables many new types of model understanding, but imposes limits on the depth ofinteraction with models and the ability to scale simulation to support large analysis.

A third category of tools imports the models created by traditional tools to perform analyses independently of theoriginal modeling tool. An example of this is SDM-Doc8, a model documentation tool, or Abdel-Gawad et. al.’seigenvector analysis tool9. It is this third category to which PySD belongs.

The central paradigm of PySD is that it is more efficient to bring the mature capabilities of system dynamics into anenvironment in use for active development in data science, than to attempt to bring each new development in inferenceand machine learning into the system dynamics enclave.



PySD reads a model file – the product of a modeling program such as Vensim10 or Stella/iThink11 – and crosscompiles it into Python, providing a simulation engine that can run these models natively in the Python environment.It is not a substitute for these tools, and cannot be used to replace a visual model construction environment.

1.7.2 The “4+1” Model View of Software Architecture

The 4+1 model view, designed by Philippe Krutchen, presents a way to describe the architecture of software systems,using multiple and concurrent views. This use of multiple views allows to address separately the concerns of thevarious ‘stakeholders’ of the architecture such as end-user, developers, systems engineers, project managers, etc.

The software architecture deals with abstraction, with decomposition and composition, with style and system’s es-thetic. To describe a software architecture, we use a model formed by multiple views or perspectives. That model ismade up of five main views: logical view, development view, process view, physical view and scenarios or user cases.

• The Physical View: describes the mapping of the software onto the hardware and reflects its distributed aspect

• The Development view: describes the static organization of the software in its development environment.

• The logical view: is the object model for the design

• The process view: captures the concurrency and synchronization aspects of the design

• The scenarios: show how the four views work together seamlessly

The “4+1” Model View of PySD

The development view of PySD

A package diagram, represented in the figure PySD development view, is used to represent the architecture of thefragments that make up PySD. This diagram represents the hierarchical architecture of PySD packages, excluding therelationship between their modules. In PySD relationships between modules we can find the relationships betweenPySD modules.

pysd is the main package that includes all the others and all the logic of the system. It is necessary to differentiatebetween the package called pysd and the module called pysd. The latter allows the user to interact with the systemand has the appropiate functions to be able to translate a Vensim or XMILE model and, in addition, to execute thegenerated translation. However, the XMILE translation process is not defined in this document.

The pysd module interacts with the modules of the py_backend package. This package has two sub-packages: vensimand xmile. The vensim package contains all the logic needed to translate Vensim models.

In PySD relationships between modules the relationships between the main modules are represented. For clarity, eachmodule is represented with its name package.

As mentioned above, users interact with pysd module. In turn, pysd is responsible for translating the Vensim modelinto the Python model by interacting with vensim2py, which creates the correct Python translation. In this process,vensim2py interacts with and uses the functions of the modules: external, utils and builder. To carry out the executionprocess, pysd uses the functions module.

The logical view of PySD

The purpose of each PySD module is detailed below, as well as the most important functions of each module.

It should be noted that in diagrams it has been necessary, input parameters have been detailed with in notation, outputparameters with out notation and parameters that are modified in a function with inout notation. In addition, thedifferent types of parameters that could be, are described by notes in convenient diagrams, which is due to Python’sdynamic typing.

1.7. Developer Documentation 19

https://www.cs.ubc.ca/~gregor/teaching/papers/4+1view-architecture.pdf


Fig. 1: PySD development view

Fig. 2: PySD relationships between modules



In Main modules of PySD: pysd, vensim2py and table2py modules are presented in detail. The pysd module is in thepysd package, while the vensim2py and table2py modules are in the vensim package.

The pysd module has the necessary functions to allow the user to create the translation of a Vensim model into Python.The read_vensim() function takes a Vensim model in text format as a parameter and converts it into an instanceof the Model class, which is in the functions module. Also, pysd has the load() function, which can generate froma Python model to an instance of the Model class, which may be able to execute and perform the simulation. Theload() function is used inside the read_ vensim() function.

Fig. 3: Main modules of PySD

The table2py module has only one function, read_tabular(). This function allows to read a Vensim model intable form (csv, tab or xlsx) and convert it into an instance of the Model class.

In addition, vensim2py is represented in that diagram. In vensim2py, the five grammars of PySD are defined, withtheir associated classes that allow parsing and obtaining the information from a Vensim model.

The main function of the vensim2py module, which is also used by the read_vensim() function of pysd module, istranslate_vensim(). This function starts the translation process. The Vensim model is parsed with the first pysdgrammar, file_structure_grammar, found in the get_file_sections() function. The file_structure_grammardivides the model into sections: the main section with the main code of the model and on the other hand, a sectionfor each macro in the model. The obtained sections are passed as parameters to translate_section() functionafterwards.

The functions get_model_elements(), get_equation_components(),parse_general_expression() and parse_lookup_expression() have the four remaining gram-mars of PySD which are: model_structure_grammar, component_structure_grammar, expression_grammar andlookup_grammar, respectively. In addition, after each of these functions, the NodeVisitor classes associated with eachgrammar are defined. These classes allow the parse tree to be performed and parsed.

Noteworthy is the function _include_common_grammar() which has the basic grammar rules used by all othergrammars.

Due to the complexity of vensim2py, as it has the five functions in which PySD grammars and their visitor classes are



defined, in Simplified vensim2py module it is represented without detail. These classes are: FileParser, ModelParser,ComponentParser, ExpressionParser and LookupParser. Note that these classes inherit from the NodeVisitor class,that provides an inversion-of-control framework for walking a tree and returning a new construct based on it.

Fig. 4: Simplified vensim2py module

In Classes of pysd grammars (Part 1) and Classes of pysd grammars (Part 2) are represented the classes associated tothe grammars.

The methods of each class are the visitor methods associated with the different grammar rules. There is no visitormethod for each rule, but there is a visitor method associated with a rule that serves to store certain information aboutthe parsed model. Within the visitor method, that relevant information is stored in the attributes of each class, whichare then returned as a result of the grammar.

Visitor methods always have three parameters: self, n and vc. Self represents the current instance of the class, n isof type Node and is the node being visited, and vc or visit children is a list of all the results of the child nodes ofthe expression being parsed. From that last parameter, vc, the information is taken and stored in the attributes of theclasses.

The functions module is represented in Simplified functions module. It is one of the most important modules in PySD,since it has the classes that will instantiate the Python translation model and also has the logic needed to run thesimulation. That diagram represents the classes it has and the relationships between them.

The functions module in detail can be found in the Functions module (Part 1) diagram as well as the Time class thatis define in this module. In functions, we can find many functions that are used in Vensim but with the relevant logicin Python, for example: PULSE, IF THEN ELSE, RANDOM UNIFORM, etc.

The Time class represents the time throughout the simulation. The t attribute represents the current time, whichchanges as the simulation progresses, and the step attribute represents the time increment that occurs in each iteration.

In the diagram Functions module (Part 2) the classes of the functions module Stateful, Integ, Macro and Model arerepresented in detail. The Stateful class is one of the most relevant classes of that module, since, except Time, all the



Fig. 5: Classes of pysd grammars (Part 1)

Fig. 6: Classes of pysd grammars (Part 2)



Fig. 7: Simplified functions module

classes inherit from it. This class makes it possible to represents the evolution of the state of a certain element models,recreating the simulation process in Vensim. To do so, it has an attributed called state that simulates the state of theelements and changes its value in each iteration of the simulation.

The Integ class simulates the Vensim stocks. It receives and stores an initial value and has the function from which thederivative necessary to perform the integration is obtained.

The Model class stores all the information about the main code of the (translated) model. An instance of this class iscalled a pysd model, it is the Python language representation of the Vensim file. That is, the Model class implements arepresentation of the stateful elements of the model and has most of the methods to access and modify the componentsof the model. In addition, the Model class is in charge of instantiating the time as a function of the model variablesand it is also in charge of performing the simulation through Euler integration.

The initialize() function of that class initialize the model simulation. The run() function allows to simulatethe behaviour of the model by increasing steps. And the _euler_step() function allows to do the Euler integrationin a single step, using the state of the Stateful elements and updating it.

The Model class inherits from Macro class. The logic for rendering Vensim macros is implemented in Macro class.This class obtains the stateful objects that have been created in the translation process and they are initialized to laterobtain their derivates and the results of the execution. Model does the same functions as Macro, but Model is the rootmodel object so it has more methods to facilitate execution.

Next, in Builder module figures the builder module. There is no class defined in this module, but it is in charge ofmaking the text model in Python, using the results obtained in the translation. It has the necessary code to assemble ina pysd model all the elements of both Vensim or XMILE and make, from these, a Python-compatible version.

The main function of the builder module is build(). That function builds and writes the Python representation ofthe model. It is called from the vensim2py module after finishing the whole process of translating the Vensim model.As parameters it is passed the different elements of the model that have been parsed, subscripts, namespace and thename of the file where the result of the Python representation should be written. This function has certain permanent



Fig. 8: Functions module (Part 1)



Fig. 9: Functions module (Part 2)

Fig. 10: Builder module



lines of code that are always write in the models created, but then, there are certain lines of code that are completedwith the translation generated before in the vensim2py module.

In image Utils module is found the utils module. The main purpose of utils is to bring together in a single module all thefunctions that are useful for the project. Many of these functions are used many times during the translation process.So, as already presented in PySD relationships between modules, this module is used by the builder, functions,external and vensim2py modules. In turn, the accesible names of the decorators, external and functions modulesare imported into the utils modules to define a list of the names that have already been used and that have a particularmeaning in the model being translated.

Fig. 11: Utils module

Simplified external module represents the external module and the classes it contains without detail. The main purposeof the classes defined in that module is to read external data. The main objective of the external module is to gather ina single file, all the required functions or tools to read external data files.

The figure External module (Part 1) shows the detailed diagrams of the External and Excels class.

External is the main class of that module, all other classes inherit from it, except the Excels class.

The External class allows storing certain information, such as the name of the file being read and the data it contains.

The Excels class is in charge of reading Excel files and storing information about them, in order to avoid reading thesefiles more than once, implementing the singleton pattern.

In External module (Part 2) all the classes that inherit from the External class are presented.

In Vensim there are different statements that allow to obtain data from external files that are used as variables in aVensim model. Below is the set of these functions that are supported in PySD.

To obtain data from statements like GET XLS DATA and GET DIRECT DATA, there is the ExtData class. In turn,for the GET XLS LOOKUPS and GET DIRECT LOOKUPS statements, the ExtLookup class. For the GET XLSCONSTANT and GET DIRECT CONSTANT functions, the ExtConstant class and, finally, to implement the GETXLS SUBSCRIPT and GET DIRECT SUBSCRIPT function, the ExtSubscript class.

These expressions create a new instance of the External class where the information to represent the necessary datastructures is stored. These instances of the External class are initialized before the stateful objects.

To better understand the functionality and the reason for the next module presented, called decorators, it would beadvisable to know the Decorator pattern.

In PySD, a kind of two-level cache is implemented to speed up model execution as much as possible. The cache isimplemented using decorators. In the translation process, each translated statement or function is tagged with one oftwo types of caches. The @cache.run decorator is used for functions whose value is constant during model execution.In this way, their value is only calculated once throughout the execution of the model. On the other hand, functionswhose values must change with each execution step are labeled with the @cache.step decorator.

In Decorators module figure the decorators module is detailed where the functions to develop and decorate thefunctions of the model in the translation step are located.


https://refactoring.guru/design-patterns/decorator


Fig. 12: Simplified external module



Fig. 13: External module (Part 1)



Fig. 14: External module (Part 2)



Fig. 15: Decorators module



The Cache class allows to define the functionality of these decorators. The run() and step() functions definethe functionality of the two-level cache used in PySD. The reset() function resets the time entered as a parameterand clears the cache of values tagged as step. The clean() function clears the cache whose name is passed as aparameter.

The process view of PySD

Activity diagrams are used to represent the PySD process view. The Main process view is the main activity diagramof PySD, the other diagrams presented in the next figures are a breakdown of this.

Fig. 16: Main process view

The translation process begins when the user indicates the Vensim model (.mdl extension) to be translated, using theread_vensim() function of the pysd module. In this function, the translate_vensim() function is calledinternally, which is passed as a parameter the Vensim model and is found in the vensim2py module. This is when thefile path extension is modified, changing the extension from mdl to py, so the translated model in Python will be savedin the same path as the Vensim model. Then, the sections that make up the model are split and, subsequently, fromthese obtained sections, a list is created with all macros in the model. Also, each section is organized and translatedresulting in translation to complete the Python file. The subsystems that make up the Main process view diagram areexplained in more detail bellow.

The figure Divide into sections shows the first subsystem. Inside the translate_vensim() function, the Vensimmodel is read in text mode and the grammar file_structure_grammar is responsible for separating the macros and themain code. This grammar is defined in the get_file_sections() function, in vensim2py module. In turn, inthis function defines the class that has the visitor methods associated with the grammar rules, called FileParser. Asresult of this function and grammar, the text of the model is divided into a list with the different sections that composeit, and a section is obtained for each macro of the model and other section with the main code.

Once the ‘Divide into sections’ sequence is complete, it continues to create a list of macros, shown in Create macrolist diagram. In this section of the translation all the sections labeled as macro are filtered to store them all in a list. Soall the macros of the Vensim model are centralized in a single list.

Next, each section in which the Vensim model has been divided into before, is organized and translated with thetranslate_section() function of the vensim2py module.

Organize each section shows this sequence in detail, with its sub-activities developed in Create Python namespaceand Parse each component diagrams.

In the figure Organize each section, from the get_model_elements() function (vensim2py module), each sectionis parsed with the grammar model_structure_grammar, which is responsible for organizing and updating the sectionsto elements of the model depending on whether they are equations or comments. In the get_model_elements()function, in addition to this grammar, the NodeVisitor class associated with it is defined which is called ModelParser.The model_structure_grammar grammar results the model divided into elements that, in turn, are organized by: equa-tion, units, limits, doc and the kind of the statement. Later, as the model progresses through the different grammars ofPySD, the new labels into which these elements are divided are update or added to the stored.



Fig. 17: Divide into sections

Fig. 18: Create macro list



Fig. 19: Organize each section

The elements that have been classified as comments do not influence the translation of the Vensim file, they are onlyuseful for model developers. For this reason, a filter of all the model elements has been placed and the equationelements will be updated through the component_structure_grammar grammar, which is shown in Organize eachsection. This grammar adds more information about the name and the kind of equation. In summary, this gram-mar allows updating and detailing the information of the elements of the model that are equations. The compo-nent_structure_grammar grammar is in the get_equation_components() function of the vensim2py moduleas well as the NodeVisitor class, which contains the necessary logic and is called ComponentParser.

Fig. 20: Create Python namespace

The ‘’Create Python namespace” subsystem is presented in the figure Create Python namespace, which is the nextstep in the translation process. The namespace is a dictionary made up of variables names, subscripts, functions andmacros that are contained in the Vensim file. To these names, a safe name in Python is assigned. To create a safe namein Python is necessary to substitute some characters that are allowed in Vensim variables but in Python they are not



valid in variable names, such as spaces, key words, unauthorized symbols, etc. In this dictionary, Vensim names arestored as the dictionary ‘keys’ and the corresponding safe names in Python are stored in the dictionary ‘values’.

To do this, inside translate_section, you can access the list of macros obtained previously and the different sectionsthat have been updated. With each macro name, each macro parameter and other elements of the model, a record isadded to the namespace dictionary with the name that represents it in Vensim and the corresponding name in Python,generated from the make_python_identifier function of the utils module. Later, another dictionary is created to addnames of subscripts that make up the model, as shown in the figure Create Python namespace. The names of thesubscripts are stored in another dictionary because they are not used to create Python functions, they only representthe dimensions of the DataArrays and do not need to have a safe name in Python. So, this subscript dictionary is madeup of all subscripts in the model and it has the subscript name as the dictionary key and the subscripts values associatedwith it as the dictionary values.

Once the namespace is created, the different components continue to be parsed, as shown in the figure Parse eachcomponent (subsystem of Organize each section). At this point in the translation sequence, the elements of the modelare divided by kind, such as regular expressions or lookups definitions.

Fig. 21: Parse each component

If it is an equation, it will be parsed with the expression_grammar grammar and if it is a lookup, the thelookup_grammar grammar will be used. The first grammar commented, expression_grammar, is found in theparse_general_expression() function of the vensim2py module, where the ExpressionParser class is alsodefined, which contains all the logic associated with this grammar.

The lookup_grammar grammar and its associated class, LookupParser, are defined in the



parse_lookup_expression() function of vensim2py module. Both grammars update the stored elementsagain, adding the corresponding Python translation as a new label on each element.

Once this sequence has been completed and returning to the figure Organize each section, the PySD translation processends with the builder. The builder module is in charge of creating the Python file containing the translation of theVensim model, using the build() function of this module. To do this, it used the namespaces created in the processand the different elements of the model previously translated and tagged with the relevant information, which willbecame part of the final Python file.

The physical view of PySD

PySD system is deployed on a single workstation and everything that is needed is in the same component. Therefore,capturing the physical view of PySD in a deployment diagram would not add more information about the system.

Scenarios of PySD

Two main scenarios can be distinguished throughout the PySD library project. The process of translating a modelfrom Vensim to Python is the first scenario. The second scenario found is the execution of that translated modelbefore, which allows the simulation to be carried out and allows the user to obtain the results of the Vensim model.

1.7.3 Contributing to PySD

If you are interested in helping to develop PySD, the PySD Development Pathway lists areas that are ripe for contribu-tion.

To get started, you can fork the repository and make contributions to your own version. When you’re happy with youredits, submit a pull request to the main branch.

Development Tools

There are a number of tools that you might find helpful in development:

Test Suite

PySD uses the common model test suite found on github which are run using integration_test_vensim_pathway.py andintegration_test_xmile_pathway.py. PySD also has own tests for internal funtionality, unit_test_*.py files of the /tests/directory.

In order to run all the tests nose or pytest can be used. A Makefile is given to run easier the tests with pytest,check tests/README for more information.

These tests run quickly and should be executed when any changes are made to ensure that current functionality remainsintact. If any new functionality is added or a bug is corrected, the tests should be updated with new models in test suiteor complementary tests in the corresponding unit_test_*.py file.

Speed Tests

The speed tests may be developed in the future. Any contribution is welcome.


https://github.com/SDXorg/test-modelshttps://github.com/JamesPHoughton/pysd/tree/master/tests/README.md


Profiler

Profiling the code can help to identify bottlenecks in operation. To understand how changes to the code influence itsspeed, we should construct a profiling test that executes the PySD components in question.

The profiler depends on cProfile and cprofilev

Python Linter

Pylint is a module that checks that your code meets proper python coding practices. It is helpful for making sure thatthe code will be easy for other people to read, and also is good fast feedback for improving your coding practice. Thelint checker can be run for the entire packages, and for individual python modules or classes. It should be run at a locallevel (ie, on specific files) whenever changes are made, and globally before the package is committed. It doesn’t needto be perfect, but we should aspire always to move in a positive direction.’

PySD Design Philosophy

Understanding that a focussed project is both more robust and maintainable, PySD aspires to the following philosophy:

• Do as little as possible.

• Anything that is not endemic to System Dynamics (such as plotting, integration, fitting, etc) should either beimplemented using external tools, or omitted.

• Stick to SD. Let other disciplines (ABM, Discrete Event Simulation, etc) create their own tools.

• Use external model creation tools

• Use the core language of system dynamics.

• Limit implementation to the basic XMILE standard.

• Resist the urge to include everything that shows up in all vendors’ tools.

• Emphasize ease of use. Let SD practitioners who haven’t used python before understand the basics.

• Take advantage of general python constructions and best practices.

• Develop and use strong testing and profiling components. Share your work early. Find bugs early.

• Avoid firefighting or rushing to add features quickly. SD knows enough about short term thinking in softwaredevelopment to know where that path leads.

1.7.4 PySD Development Pathway

High priority features, bugs, and other elements of active effort are listed on the github issue tracker. To get involvedsee Contributing to PySD.

High Priority

• Subscripts/arrays Github Issue Track

• Refactor delays to take advantage of array architecture

• Improve translation of model documentation strings and units into python function docstrings


https://docs.python.org/2.7/library/profile.html#module-cProfilehttps://github.com/ymichael/cprofilevhttp://docs.pylint.org/https://github.com/JamesPHoughton/pysd/issueshttps://github.com/JamesPHoughton/pysd/issues/21


Medium Priority

• Outsource model translation to SDXchange model translation toolset

• Improve model exexution speed using cython, theano, numba, or another package

• Improve performance when returning non-stock model elements

Low Priority

• Import model component documentation in a way that enables doctest, to enable writing unit tests within themodeling environment.

• Handle simulating over timeseries

• Implement run memoization to improve speed of larger analyses

• Implement an interface for running the model over a range of conditions, build in intelligent parallelization.

Not Planned

• Model Construction

• Display of Model Diagrams

• Outputting models to XMILE or other formats

Ideas for Other Projects

• SD-lint checker (units, modeling conventions, bounds/limits, etc)

• Contribution to external Data Science tools to make them more appropriate for dynamic assistant

Current Features

• Basic XMILE and Vensim parser

• Established library structure and data formats

• Simulation using existing python integration tools

• Integration with basic python Data Science functionality

• Run-at-a-time parameter modification

• Time-variant exogenous inputs

• Extended backends for storing parameters and output values

• Demonstration of integration with Machine Learning/Monte Carlo/Statistical Methods

• Python methods for programmatically manipulating SD model structure

• Turn off and on ‘traces’ or records of the values of variables


https://github.com/SDXchange


1.7.5 Structure of the PySD module

PySD provides a set of translators that interpret a Vensim or XMILE format model into a Python native class. Themodel components object represents the state of the system, and contains methods that compute auxiliary and flowvariables based upon the current state.

The components object is wrapped within a Python class that provides methods for modifying and executing the model.These three pieces constitute the core functionality of the PySD module, and allow it to interact with the Python dataanalytics stack.

Translation

The internal functions of the model translation components can be seen in the following documents

Vensim Translation

PySD parses a vensim ‘.mdl’ file and translates the result into python, creating a new file in the same directory as theoriginal. For example, the Vensim file Teacup.mdl becomes Teacup.py .

This allows model execution independent of the Vensim environment, which can be handy for deploying models asbackends to other products, or for performing massively parallel distributed computation.

These translated model files are read by PySD, which provides methods for modifying or running the model andconveniently accessing simulation results.

Translated Functions

Ongoing development of the translator will support the full subset of Vensim functionality that has an equivalent inXMILE. The current release supports the following functionality:

Vensim Python TranslationCOS np.cosEXP np.expMIN min=IF THEN ELSE functions.if_then_elseLN np.logPULSE TRAIN functions.pulse_trainRAMP functions.rampINTEGER intTAN np.tanPI np.pi= ==< <> >

Continued on next page


https://github.com/JamesPHoughton/PySD-Cookbook/blob/master/source/models/Teacup/Teacup.mdlhttps://github.com/JamesPHoughton/PySD-Cookbook/blob/master/source/models/Teacup/Teacup.py


Table 2 – continued from previous pageVensim Python TranslationMODULO np.modARCSIN np.arcsinABS abs^ **LOGNORMAL np.random.lognormalMAX maxSQRT np.sqrtARCTAN np.arctanARCCOS np.arccosRANDOM NORMAL functions.bounded_normalRANDOM UNIFORM np.random.randDELAY1 functions.DelayDELAY3 functions.DelayDELAY N functions.DelayNDELAY FIXED functions.DelayFixedSAMPLE IF TRUE functions.SampleIfTrueSMOOTH3 functions.SmoothSMOOTH N functions.SmoothSMOOTH functions.SmoothINITIAL functions.InitialXIDZ functions.XIDZZIDZ functions.XIDZVMIN functions.vminVMAX functions.vmaxSUM functions.sumPROD functions.prodGET XLS DATA external.ExtDataGET DIRECT DATA external.ExtDataGET XLS LOOKUPS external.ExtLookupGET DIRECT LOOKUPS external.ExtLookupGET XLS CONSTANTS external.ExtConstantGET DIRECT CONSTANTS external.ExtConstantGET XLS SUBSCRIPT external.ExtSubscriptGET DIRECT SUBSCRIPT external.ExtSubscript

np corresponds to the numpy package

Additionally, identifiers are currently limited to alphanumeric characters and the dollar sign $.

Future releases will include support for:

• subscripts

• arrays

• arbitrary identifiers

There are some constructs (such as tagging variables as ‘suplementary’) which are not currently parsed, and may throwan error. Future releases will handle this with more grace.



Used Functions for Translation

These functions translate vensim .mdl file to pieces needed by the builder module to write a python version of themodel. Everything that requires knowledge of vensim syntax should be here.

pysd.py_backend.vensim.vensim2py._include_common_grammar(source_grammar)

pysd.py_backend.vensim.vensim2py.get_equation_components(equation_str,root_path=None)

Breaks down a string representing only the equation part of a model element. Recognizes the various types ofmodel elements that may exist, and identifies them.

Parameters

• equation_str (basestring) – the first section in each model element - the full equa-tion.

• root_path (basestring) – the root path of the vensim file (necessary to resolve ex-ternal data file paths)

Returns

• Returns a dictionary containing the following

• real_name (basestring) – The name of the element as given in the original vensim file

• subs (list of strings) – list of subscripts or subscript elements

• expr (basestring)

• kind (basestring) – What type of equation have we found? - component - normal modelexpression or constant - lookup - a lookup table - subdef - a subscript definition - data - adata variable

• keyword (basestring or None)

Examples

>>> get_equation_components(r'constant = 25'){'expr': '25', 'kind': 'component', 'subs': [], 'real_name': 'constant'}

Notes

in this function we don’t create python identifiers, we use real names. This is so that when everything comesback together, we can manage any potential namespace conflicts properly

pysd.py_backend.vensim.vensim2py.get_file_sections(file_str)This is where we separate out the macros from the rest of the model file. Working based upon documentationat: https://www.vensim.com/documentation/index.html?macros.htm

Macros will probably wind up in their own python modules eventually.

Parameters file_str –

Returns

entries – Each dictionary represents a different section of the model file, either a macro, or themain body of the model file. The dictionaries contain various elements: - returns: list of strings

represents what is returned from a macro (for macros) or empty for main model


https://www.vensim.com/documentation/index.html?macros.htm


• params: list of strings represents what is passed into a macro (for macros) or empty formain model

• name: string the name of the macro, or ‘main’ for main body of model

• string: string string representing the model section

Return type list of dictionaries

Examples

>>> get_file_sections(r'a~b~c| d~e~f| g~h~i|')[{'returns': [], 'params': [], 'name': 'main', 'string': 'a~b~c| d~e~f| g~h~i|'}]

pysd.py_backend.vensim.vensim2py.get_model_elements(model_str)Takes in a string representing model text and splits it into elements

I think we’re making the assumption that all newline characters are removed. . .

Parameters model_str (string) –

Returns entries – Each dictionary contains the components of a different model element, separatedinto the equation, units, and docstring.

Return type array of dictionaries

Examples

# Basic Parsing: >>> get_model_elements(r’a~b~c| d~e~f| g~h~i|’) [{‘doc’: ‘c’, ‘unit’: ‘b’, ‘eqn’: ‘a’}, {‘doc’:‘f’, ‘unit’: ‘e’, ‘eqn’: ‘d’}, {‘doc’: ‘i’, ‘unit’: ‘h’, ‘eqn’: ‘g’}]

# Special characters are escaped within double-quotes: >>> get_model_elements(r’a~b~c| d~e”~”~f| g~h~i|’)[{‘doc’: ‘c’, ‘unit’: ‘b’, ‘eqn’: ‘a’}, {‘doc’: ‘f’, ‘unit’: ‘e”~”’, ‘eqn’: ‘d’}, {‘doc’: ‘i’, ‘unit’: ‘h’, ‘eqn’: ‘g’}]>>> get_model_elements(r’a~b~c| d~e~”|”f| g~h~i|’) [{‘doc’: ‘c’, ‘unit’: ‘b’, ‘eqn’: ‘a’}, {‘doc’: ‘”|”f’, ‘unit’:‘e’, ‘eqn’: ‘d’}, {‘doc’: ‘i’, ‘unit’: ‘h’, ‘eqn’: ‘g’}]

# Double-quotes within escape groups are themselves escaped with backslashes: >>>get_model_elements(r’a~b~c| d~e””~”~f| g~h~i|’) [{‘doc’: ‘c’, ‘unit’: ‘b’, ‘eqn’: ‘a’}, {‘doc’: ‘f’, ‘unit’:‘e”"~”’, ‘eqn’: ‘d’}, {‘doc’: ‘i’, ‘unit’: ‘h’, ‘eqn’: ‘g’}] >>> get_model_elements(r’a~b~c| d~e~””|”f| g~h~i|’)[{‘doc’: ‘c’, ‘unit’: ‘b’, ‘eqn’: ‘a’}, {‘doc’: ‘”"|”f’, ‘unit’: ‘e’, ‘eqn’: ‘d’}, {‘doc’: ‘i’, ‘unit’: ‘h’, ‘eqn’: ‘g’}]>>> get_model_elements(r’a~b~c| d~e”xnx”~f| g~h~|’) [{‘doc’: ‘c’, ‘unit’: ‘b’, ‘eqn’: ‘a’}, {‘doc’: ‘f’, ‘unit’:‘e”x\nx”’, ‘eqn’: ‘d’}, {‘doc’: ‘’, ‘unit’: ‘h’, ‘eqn’: ‘g’}]

# Todo: Handle model-level or section-level documentation >>> get_model_elements(r’*** .model doc ***~Docstring!| d~e~f| g~h~i|’) [{‘doc’: ‘Docstring!’, ‘unit’: ‘’, ‘eqn’: ‘’}, {‘doc’: ‘f’, ‘unit’: ‘e’, ‘eqn’: ‘d’},{‘doc’: ‘i’, ‘unit’: ‘h’, ‘eqn’: ‘g’}]

# Handle control sections, returning appropriate docstring pieces >>> get_model_elements(r’a~b~c| ****.Con-trol***~ Simulation Control Parameters | g~h~i|’) [{‘doc’: ‘c’, ‘unit’: ‘b’, ‘eqn’: ‘a’}, {‘doc’: ‘i’, ‘unit’: ‘h’,‘eqn’: ‘g’}]

# Handle the model display elements (ignore them) >>> get_model_elements(r’a~b~c| d~e~f| -–///junk|junk~junk’) [{‘doc’: ‘c’, ‘unit’: ‘b’, ‘eqn’: ‘a’}, {‘doc’: ‘f’, ‘unit’: ‘e’, ‘eqn’: ‘d’}]

Notes

• Tildes and pipes are not allowed in element docstrings, but we should still handle them there



pysd.py_backend.vensim.vensim2py.parse_general_expression(element, names-pace={}, sub-script_dict={},macro_list=None,elements_subs_dict={},subs_compatibility={})

Parses a normal expression # its annoying that we have to construct and compile the grammar every time. . .

Parameters

• element (dictionary) –

• namespace (dictionary) –

• subscript_dict (dictionary) –

• macro_list (list of dictionaries) – [{‘name’: ‘M’, ‘py_name’:’m’, ‘file-name’:’path/to/file’, ‘args’:[‘arg1’, ‘arg2’]}]

• elements_subs_dict (dictionary) – The dictionary with element python namesas keys and their merged subscripts as values.

• subs_compatibility (dictionary) – The dictionary storing the mapped subscripts

Returns

• translation

• new_elements (list of dictionaries) – If the expression contains builder functions, thosebuilders will create new elements to add to our running list (that will eventually be output toa file) such as stock initialization and derivative funcs, etc.

Examples

>>> parse_general_expression({'expr': 'INTEG (FlowA, -10)',... 'py_name':'test_stock',... 'subs':None},... {'FlowA': 'flowa'}),({'kind': 'component', 'py_expr': "_state['test_stock']"},[{'kind': 'implicit',

'subs': None,'doc': 'Provides initial conditions for test_stock function','py_name': 'init_test_stock','real_name': None,'unit': 'See docs for test_stock','py_expr': '-10'},

{'py_name': 'dtest_stock_dt','kind': 'implicit','py_expr': 'flowa','real_name': None}])

pysd.py_backend.vensim.vensim2py.parse_lookup_expression(element, subscript_dict)This syntax parses lookups that are defined with their own element

pysd.py_backend.vensim.vensim2py.parse_units(units_str)Extract and parse the units Extract the bounds over which the expression is assumed to apply.

Parameters units_str –



Examples

>>> parse_units('Widgets/Month [-10,10,1]')('Widgets/Month', (-10,10,1))

>>> parse_units('Month [0,?]')('Month', [-10, None])

>>> parse_units('Widgets [0,100]')('Widgets', (0, 100))

>>> parse_units('Widgets')('Widgets', (None, None))

>>> parse_units('[0, 100]')('', (0, 100))

pysd.py_backend.vensim.vensim2py.translate_section(section, macro_list, root_path)

pysd.py_backend.vensim.vensim2py.translate_vensim(mdl_file)

Parameters mdl_file (basestring) – file path of a vensim model file to translate to python

Examples

>>> translate_vensim('../tests/test-models/tests/subscript_3d_arrays/test_→˓subscript_3d_arrays.mdl')

XMILE Translation

The XMILE reference documentation is located at:

• XMILE: http://www.iseesystems.com/community/support/XMILEv4.pdf

• SMILE: http://www.iseesystems.com/community/support/SMILEv4.pdf

The PySD module is capable of importing models from a Vensim model file (*.mdl) or an XMILE format xml file.Translation makes use of a Parsing Expression Grammar parser, using the third party Python library Parsimonious13to construct an abstract syntax tree based upon the full model file (in the case of Vensim) or individual expressions (inthe case of XMILE).

The translators then crawl the tree, using a dictionary to translate Vensim or Xmile syntax into its appropriate Pythonequivalent. The use of a translation dictionary for all syntactic and programmatic components prevents execution ofarbitrary code from unverified model files, and ensures that we only translate commands that PySD is equipped tohandle. Any unsupported model functionality should therefore be discovered at import, instead of at runtime.

The use of a one-to-one dictionary in translation means that the breadth of functionality is inherently limited. In thecase where no direct Python equivalent is available, PySD provides a library of functions such as pulse, step, etc. thatare specific to dynamic model behavior.

In addition to translating individual commands between Vensim/XMILE and Python, PySD reworks component iden-tifiers to be Python-safe by replacing spaces with underscores. The translator allows source identifiers to make use ofalphanumeric characters, spaces, or the $ symbol.


http://www.iseesystems.com/community/support/XMILEv4.pdfhttp://www.iseesystems.com/community/support/SMILEv4.pdf


The model class

The translator constructs a Python class that represents the system dynamics model. The class maintains a dictionaryrepresenting the current values of each of the system stocks, and the current simulation time, making it a ‘statefull’model in much the same way that the system itself has a specific state at any point in time.

The model class also contains a function for each of the model components, representing the essential model equations.The docstring for each function contains the model documentation and units as translated from the original model file.A query to any of the model functions will calculate and return its value according to the stored state of the system.

The model class maintains only a single state of the system in memory, meaning that all functions must obey theMarkov property - that the future state of the system can be calculated entirely based upon its current state. In additionto simplifying integration, this requirement enables analyses that interact with the model at a step-by-step level. Thedownside to this design choice is that several components of Vensim or XMILE functionality – the most significantbeing the infinite order delay – are intentionally not supported. In many cases similar behavior can be approximatedthrough other constructs.

Lastly, the model class provides a set of methods that are used to facilitate simulation. PySD uses the standardordinary differential equations solver provided in the well-established Python library Scipy, which expects the stateand its derivative to be represented as an ordered list. The model class provides the function .d_dt() that takes astate vector from the integrator and uses it to update the model state, and then calculates the derivative of each stock,returning them in a corresponding vector. A complementary function .state_vector() creates an ordered vector of statesfor use in initializing the integrator.

The PySD class

Internal Functions

This section documents the functions that are going on behaind the scenes, for the benefit of developers.

Special functions needed for model execution

These functions have no direct analog in the standard python data analytics stack, or require information about theinternal state of the system beyond what is present in the function call. We provide them in a structure that makes iteasy for the model elements to call.

class pysd.py_backend.functions.Delay(delay_input, delay_time, initial_value, or-der, tstep=,py_name=’Delay object’)

Implements DELAY function

ddt()

export()

initialize(init_val=None)

class pysd.py_backend.functions.DelayFixed(delay_input, delay_time, initial_value, tstep,py_name)

Implements DELAY FIXED function

ddt()

export()


update(state)



class pysd.py_backend.functions.DelayN(delay_input, delay_time, initial_value, order, tstep,py_name)

Implements DELAY N function

ddt()

export()


class pysd.py_backend.functions.DynamicStateful

update(state)

class pysd.py_backend.functions.Initial(initial_value, py_name=’Initial object’)Implements INITIAL function

export()


class pysd.py_backend.functions.Integ(ddt, initial_value, py_name=’Integ object’)Implements INTEG function

export()


class pysd.py_backend.functions.Macro(py_model_file, params=None, return_func=None,time=None, time_initialization=None,py_name=None)

The Model class implements a stateful representation of the system, and contains the majority of methods foraccessing and modifying model components.

When the instance in question also serves as the root model object (as opposed to a macro or submodel withinanother model) it will have added methods to facilitate execution.

_constant_component(value, dims, args=[])Internal function for creating a constant model element

_timeseries_component(series, dims, args=[])Internal function for creating a timeseries model element

ddt()

doc()Formats a table of documentation strings to help users remember variable names, and understand how theyare translated into python safe names.

Returns

docs_df –

Dataframe with columns for the model components:

• Real names

• Python safe identifiers (as used in model.components)

• Units string

• Documentation strings from the original model file

Return type pandas dataframe



export(file_name)Export stateful values to pickle file.

Parameters file_name (str) – Name of the file to export the values.

get_args(param)Returns the arguments of a model element.

Parameters param (str or func) – The model element name or function.

Returns

• args (list) – List of arguments of the function.

• >>> model.get_args(‘birth_rate’)

• >>> model.get_args(‘Birth Rate’)

get_coords(param)Returns the coordinates and dims of a model element.

Parameters param (str or func) – The model element name or function.

Returns

• (coords, dims) or None ((dict, list) or None) – The coords and the dimensions of theelement if it has. Otherwise, returns None.

• >>> model.get_coords(‘birth_rate’)

• >>> model.get_coords(‘Birth Rate’)

get_pysd_compiler_version()Returns the version of pysd complier that used for generating this model

import_pickle(file_name)Import stateful values from pickle file.

Parameters file_name (str) – Name of the file to import the values from.

initialize(initialization_order=None)This function tries to initialize the stateful objects.

In the case where an initialization function for Stock A depends on the value of Stock B, if we try to initializeStock A before Stock B then we will get an error, as the value will not yet exist.

In this case, just skip initializing Stock A for now, and go on to the other state initializations. Then comeback to it and try again.

set_components(params)Set the value of exogenous model elements. Element values can be passed as keyword=value pairs in thefunction call. Values can be numeric type or pandas Series. Series will be interpolated by integrator.

Examples

>>> model.set_components({'birth_rate': 10})>>> model.set_components({'Birth Rate': 10})

>>> br = pandas.Series(index=range(30), values=np.sin(range(30))>>> model.set_components({'birth_rate': br})

set_initial_value(t, initial_value)Set the system initial value.


https://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.html#strhttps://docs.python.org/2.7/library/functions.ht

pysd documentation - read the docs · 2021. 1. 28. · pysd documentation, release 1.0.1...

Documents