poliosim.base module

Base classes for Poliosim. These classes handle a lot of the boilerplate of the People and Sim classes (e.g. loading, saving, key lookups, etc.), so those classes can be focused on the disease-specific functionality.

class ParsObj(pars)

Bases: sciris.sc_utils.prettyobj

A class based around performing operations on a self.pars dict.

update_pars(pars=None, create=False)

Update internal dict with new pars.

Parameters

• pars (dict) – the parameters to update (if None, do nothing)

• create (bool) – if create is False, then raise a KeyNotFoundError if the key does not already exist

class Result(name=None, npts=None, scale='dynamic', color=None)

Bases: object

Stores a single result – by default, acts like an array.

Parameters

• name (str) – name of this result, e.g. new_infections

• npts (int) – if values is None, precreate it to be of this length

• scale (str) – whether or not the value scales by population size; options are “dynamic”, “static”, or False

• color (str/arr) – default color for plotting (hex or RGB notation)

• n_variants (int) – the number of variants the result is for (0 for results not by variant)

Example:

import poliosim as ps
r1 = ps.Result(name='test1', npts=10)
r1[:5] = 20
print(r1.values)

property npts

class BaseSim(*args, **kwargs)

Bases: poliosim.base.ParsObj

The BaseSim class handles the running of the simulation: the number of people, number of time points, and the parameters of the simulation.

set_metadata(simfile)

Set the metadata for the simulation – creation time and filename

set_seed(seed=- 1)

Set the seed for the random number stream from the stored or supplied value

Parameters

seed (None or int) – if no argument, use current seed; if None, randomize; otherwise, use and store supplied seed

Returns

None

property n

Count the number of people – if it fails, assume none

property scaled_pop_size

Get the total population size, i.e. the number of agents times the scale factor – if it fails, assume none

property npts

Count the number of time points

property tvec

Create a time vector

property datevec

Create a vector of dates

Returns

Array of datetime instances containing the date associated with each simulation time step

day(day, *args)

Convert a string, date/datetime object, or int to a day (int).

Parameters

day (str, date, int, or list) – convert any of these objects to a day relative to the simulation’s start day

Returns

the day(s) in simulation time

Return type

days (int or str)

Example:

sim.day('2020-04-05') # Returns 35

date(ind, *args, dateformat=None, as_date=False)

Convert one or more integer days of simulation time to a date/list of dates – by default returns a string, or returns a datetime Date object if as_date is True. See also ps.date(), which provides a partly overlapping set of date conversion features.

Parameters

• ind (int, list, or array) – the day(s) in simulation time

• args (list) – additional day(s)

• dateformat (str) – the format to return the date in

• as_date (bool) – whether to return as a datetime date instead of a string

Returns

the date(s) corresponding to the simulation day(s)

Return type

dates (str, Date, or list)

Examples:

sim.date(34) # Returns '2020-04-04'
sim.date([34, 54]) # Returns ['2020-04-04', '2020-04-24']
sim.date(34, 54, as_dt=True) # Returns [datetime.date(2020, 4, 4), datetime.date(2020, 4, 24)]

result_keys()

Get the actual results objects, not other things stored in sim.results

copy()

Returns a deep copy of the sim

export_results(for_json=True, filename=None, indent=2, *args, **kwargs)

Convert results to dict – see also to_json().

The results written to Excel must have a regular table shape, whereas for the JSON output, arbitrary data shapes are supported.

Parameters

• for_json (bool) – if False, only data associated with Result objects will be included in the converted output

• filename (str) – filename to save to; if None, do not save

• indent (int) – indent (int): if writing to file, how many indents to use per nested level

• args (list) – passed to savejson()

• kwargs (dict) – passed to savejson()

Returns

dictionary representation of the results

Return type

resdict (dict)

export_pars(filename=None, indent=2, *args, **kwargs)

Return parameters for JSON export – see also to_json().

This method is required so that interventions can specify their JSON-friendly representation.

Parameters

• filename (str) – filename to save to; if None, do not save

• indent (int) – indent (int): if writing to file, how many indents to use per nested level

• args (list) – passed to savejson()

• kwargs (dict) – passed to savejson()

Returns

a dictionary containing all the parameter values

Return type

pardict (dict)

to_json(filename=None, keys=None, tostring=False, indent=2, verbose=False, *args, **kwargs)

Export results and parameters as JSON.

Parameters

• filename (str) – if None, return string; else, write to file

• keys (str or list) – attributes to write to json (default: results, parameters, and summary)

• tostring (bool) – if not writing to file, whether to write to string (alternative is sanitized dictionary)

• indent (int) – if writing to file, how many indents to use per nested level

• verbose (bool) – detail to print

• args (list) – passed to savejson()

• kwargs (dict) – passed to savejson()

Returns

A unicode string containing a JSON representation of the results, or writes the JSON file to disk

Examples:

json = sim.to_json()
sim.to_json('results.json')
sim.to_json('summary.json', keys='summary')

to_df(date_index=False)

Export results to a pandas dataframe

Parameters

date_index (bool) – if True, use the date as the index

to_pandas(key=None, to_file=False, filename='output.ftr')

Return a part of the sim as a pandas dataframe with optional export

Parameters

• key (str) – attribute to return as pandas dataframe (options are: contacts, infection_log, or states (from save_states analyzer))

• to_file (bool) – if True, export as a feather file

• filename (str) – filename for write to file (should end in .ftr)

Returns

A pd.dataframe, or writes the file to disk

Examples:

sim = ps.create_sim(rand_seed=100, pop_size=10e3, n_days=30,
            pop_infected = 100,
            vx_coverage=0.05,
            trace_prob=1.0, test_delay=0,
            rel_symp_prob=1.0,
            analyzers=ps.save_states(full=True),
        )
sim.run()
sim.to_pandas('contacts')
sim.to_pandas('contacts', to_file=True, filename='df_contacts.ftr')
sim.to_pandas('infection_log', to_file=True, filename='df_infection_log.ftr')
sim.to_pandas('states', to_file=True, filename='df_states.ftr')

to_excel(filename=None, skip_pars=None)

Export parameters and results as Excel format

Parameters

• filename (str) – if None, return string; else, write to file

• skip_pars (list) – if provided, a custom list parameters to exclude

Returns

An sc.Spreadsheet with an Excel file, or writes the file to disk

shrink(skip_attrs=None, in_place=True)

“Shrinks” the simulation by removing the people, and returns a copy of the “shrunken” simulation. Used to reduce the memory required for saved files.

Parameters

skip_attrs (list) – a list of attributes to skip in order to perform the shrinking; default “people”

Returns

a Sim object with the listed attributes removed

Return type

shrunken (Sim)

save(filename=None, keep_people=None, skip_attrs=None, **kwargs)

Save to disk as a gzipped pickle.

Parameters

• filename (str or None) – the name or path of the file to save to; if None, uses stored

• kwargs – passed to sc.makefilepath()

Returns

the validated absolute path to the saved file

Return type

filename (str)

Example:

sim.save() # Saves to a .sim file with the date and time of creation by default

static load(filename, *args, **kwargs)

Load from disk from a gzipped pickle.

Parameters

• filename (str) – the name or path of the file to load from

• kwargs – passed to ps.load()

Returns

the loaded simulation object

Return type

sim (Sim)

Example:

sim = ps.Sim.load('my-simulation.sim')

update_pars(pars=None, create=False, **kwargs)

Ensure that metaparameters get used properly before being updated

load_data(datafile=None, datacols=None, verbose=None, **kwargs)

Load the data to calibrate against, if provided

layer_keys()

Attempt to retrieve the current layer keys, in the following order: from the people object (for an initialized sim), from the popdict (for one in the process of being initialized), from the beta_layer parameter (for an uninitialized sim), or by assuming a default (if none of the above are available).

validate_layer_pars()

Handle layer parameters, since they need to be validated after the population creation, rather than before.

validate_pars(validate_layers=True)

Some parameters can take multiple types; this makes them consistent.

Parameters

validate_layers (bool) – whether to validate layer parameters as well via validate_layer_pars() – usually yes, except during initialization

load_population(popfile=None, **kwargs)

Load the population dictionary from file – typically done automatically as part of sim.initialize(). Supports loading either saved population dictionaries (popdicts, file ending .pop by convention), or ready-to-go People objects (file ending .ppl by convention). Either object an also be supplied directly. Once a population file is loaded, it is removed from the Sim object.

Parameters

• popfile (str or obj) – if a string, name of the file; otherwise, the popdict or People object to load

• kwargs (dict) – passed to sc.makefilepath()

init_people(save_pop=False, load_pop=False, popfile=None, verbose=None, seed_infections=None, **kwargs)

Create the people.

Parameters

• save_pop (bool) – if true, save the population dictionary to popfile

• load_pop (bool) – if true, load the population dictionary from popfile

• popfile (str) – filename to load/save the population

• verbose (int) – detail to print

• kwargs (dict) – passed to ps.make_people()

init_interventions()

Initialize and validate the interventions

finalize_interventions()

init_analyzers()

Initialize the analyzers

finalize_analyzers()

restore_pars(orig_pars)

Restore the original parameter values, except for the analyzers

compute_summary(verbose=None)

Compute the summary statistics to display at the end of a run

summarize(output=False)

Print a brief summary of the simulation

brief(output=False)

Return a one-line description of a sim

compute_fit(*args, **kwargs)

Compute the fit between the model and the data. See cv.Fit() for more information.

Parameters

• args (list) – passed to cv.Fit()

• kwargs (dict) – passed to cv.Fit()

Returns

A Fit object

Example:

sim = cv.Sim(datafile='data.csv')
sim.run()
fit = sim.compute_fit()
fit.plot()

calibrate(calib_pars, **kwargs)

Automatically calibrate the simulation, returning a Calibration object (a type of analyzer). See the documentation on that class for more information.

Parameters

• calib_pars (dict) – a dictionary of the parameters to calibrate of the format dict(key1=[best, low, high])

• kwargs (dict) – passed to cv.Calibration()

Returns

A Calibration object

Example:

sim = cv.Sim(datafile='data.csv')
calib_pars = dict(beta=[0.015, 0.010, 0.020])
calib = sim.calibrate(calib_pars, n_trials=50)
calib.plot()

make_age_histogram(output=True, *args, **kwargs)

Calculate the age histograms of infections, diagnoses, etc. See ps.age_histogram() for more information. This can be used alternatively to supplying the age histogram as an analyzer to the sim. If used this way, it can only record the final time point since the states of each person are not saved during the sim.

Parameters

• output (bool) – whether or not to return the age histogram; if not, store in sim.results

• args (list) – passed to ps.age_histogram()

• kwargs (dict) – passed to ps.age_histogram()

Example:

sim = ps.Sim()
sim.run()
agehist = sim.make_age_histogram()
agehist.plot()

make_transtree(output=True, *args, **kwargs)

Create a TransTree (transmission tree) object, for analyzing the pattern of transmissions in the simulation. See ps.TransTree() for more information.

Parameters

• output (bool) – whether or not to return the TransTree; if not, store in sim.results

• args (list) – passed to ps.TransTree()

• kwargs (dict) – passed to ps.TransTree()

Example:

sim = ps.Sim()
sim.run()
tt = sim.make_transtree()

plot(*args, **kwargs)

Plot the results of a single simulation.

plot_result(key, *args, **kwargs)

Simple method to plot a single result. Useful for results that aren’t standard outputs. See sim.plot() for explanation of other arguments.

Parameters

key (str) – the key of the result to plot

Examples:

sim.plot_result('r_eff')

get_interventions(label=None, partial=False, as_inds=False)

Find the matching intervention(s) by label, index, or type. If None, return all interventions. If the label provided is “summary”, then print a summary of the interventions (index, label, type).

Parameters

• label (str, int, Intervention, list) – the label, index, or type of intervention to get; if a list, iterate over one of those types

• partial (bool) – if true, return partial matches (e.g. ‘beta’ will match all beta interventions)

• as_inds (bool) – if true, return matching indices instead of the actual interventions

Examples:

tp = ps.test_prob(symp_prob=0.1)
cb1 = ps.change_beta(days=5, changes=0.3, label='NPI')
cb2 = ps.change_beta(days=10, changes=0.3, label='Masks')
sim = ps.Sim(interventions=[tp, cb1, cb2])
cb1, cb2 = sim.get_interventions(ps.change_beta)
tp, cb2 = sim.get_interventions([0,2])
ind = sim.get_interventions(ps.change_beta, as_inds=True) # Returns [1,2]
sim.get_interventions('summary') # Prints a summary

get_intervention(label=None, partial=False, first=False, die=True)

Like get_interventions(), find the matching intervention(s) by label, index, or type. If more than one intervention matches, return the last by default. If no label is provided, return the last intervention in the list.

Parameters

• label (str, int, Intervention, list) – the label, index, or type of intervention to get; if a list, iterate over one of those types

• partial (bool) – if true, return partial matches (e.g. ‘beta’ will match all beta interventions)

• first (bool) – if true, return first matching intervention (otherwise, return last)

• die (bool) – whether to raise an exception if no intervention is found

Examples:

tp = ps.test_prob(symp_prob=0.1)
cb = ps.change_beta(days=5, changes=0.3, label='NPI')
sim = ps.Sim(interventions=[tp, cb])
cb = sim.get_intervention('NPI')
cb = sim.get_intervention('NP', partial=True)
cb = sim.get_intervention(ps.change_beta)
cb = sim.get_intervention(1)
cb = sim.get_intervention()
tp = sim.get_intervention(first=True)

get_analyzers(label=None, partial=False, as_inds=False)

Same as get_interventions(), but for analyzers.

get_analyzer(label=None, partial=False, first=False, die=True)

Same as get_intervention(), but for analyzers.

class BasePeople

Bases: sciris.sc_utils.prettyobj

A class to handle all the boilerplate for people – note that everything interesting happens in the People class.

Parameters

pars (dict) – a dictionary with, at minimum, keys ‘pop_size’ and ‘n_days’

Initialize essential attributes used for filtering

property len_inds

Alias (almost) to len(self)

property len_people

Full length of People array, ignoring filtering

property pop_size

Alias to len_people

property inds

Alias to self._inds to prevent accidental overwrite & increase speed by allowing “_” shortcircuit

property subinds

Alias to self._subinds

set(key, value, die=True)

Ensure sizes and dtypes match

get(key)

Convenience method – key can be string or list of strings

filter(criteria=None, inds=None, reset=False)

Store indices to allow for easy filtering of the People object.

Parameters

• criteria (array) – a boolean array for the filtering critria

• inds (array) – alternatively, explicitly filter by these indices

• reset (bool) – reset the indices rather than use existing ones

Returns

A filtered People object, which works just like a normal People object except only operates on a subset of indices.

unfilter()

An easy way of unfiltering the People object, returning the original.

binomial(prob, as_inds=False, as_filter=True)

Return indices either by a single probability or by an array of probabilities. By default just return the boolean array, but can also return the indices, or the filtered People object.

Parameters

• prob (float/array) – either a scalar probability, or an array of probabilities of the same length as People

• as_inds (bool) – return as list of indices instead of a boolean array

• as_filter (bool) – return as filter instead than boolean array

filter_tf(arr)

Split the People object into True and False sets

true(key)

Return indices matching the condition

false(key)

Return indices not matching the condition

defined(key)

Return indices of people who are not-nan

not_defined(key)

Return indices of people who are nan

count(key)

Count the number of people for a given key

count_not(key)

Count the number of people who do not have a property for a given key

set_pars(pars=None)

Re-link the parameters stored in the people object to the sim containing it, and perform some basic validation.

keys()

Returns keys for all properties of the people object

person_keys()

Returns keys specific to a person (e.g., their age)

state_keys()

Returns keys for different states of a person (e.g., symptomatic)

date_keys()

Returns keys for different event dates (e.g., date a person became symptomatic)

dur_keys()

Returns keys for different durations (e.g., the duration from exposed to infectious)

layer_keys()

Get the available contact keys – try contacts first, then beta_layer

indices()

The indices of each people array

validate(die=True, verbose=False)

to_df()

Convert to a Pandas dataframe

to_arr()

Return as numpy array

person(ind)

Method to create person from the people

to_people()

Return all people as a list

from_people(people, resize=True)

Convert a list of people back into a People object

to_graph()

Convert all people to a networkx MultiDiGraph, including all properties of the people (nodes) and contacts (edges).

Example:

import poliosim as ps
import networkx as nx
sim = ps.Sim(pop_size=50, pop_type='hybrid', contacts=dict(h=3, s=10, w=10, c=5)).run()
G = sim.people.to_graph()
nodes = G.nodes(data=True)
edges = G.edges(keys=True)
node_colors = [n['age'] for i,n in nodes]
layer_map = dict(h='#37b', s='#e11', w='#4a4', c='#a49')
edge_colors = [layer_map[G[i][j][k]['layer']] for i,j,k in edges]
edge_weights = [G[i][j][k]['beta']*5 for i,j,k in edges]
nx.draw(G, node_color=node_colors, edge_color=edge_colors, width=edge_weights, alpha=0.5)

init_contacts(reset=False)

Initialize the contacts dataframe with the correct columns and data types

add_contacts(contacts, lkey=None, beta=None)

Add new contacts to the array

update_contacts()

Refresh dynamic contacts, e.g. community

make_edgelist(contacts)

Parse a list of people with a list of contacts per person and turn it into an edge list.

static remove_duplicates(df)

Sort the dataframe and remove duplicates – note, not extensively tested

class Person(pars=None, uid=None, age=- 1, sex=- 1, contacts=None)

Bases: sciris.sc_utils.prettyobj

Class for a single person. Note: this is largely deprecated since sim.people is now based on arrays rather than being a list of people.

class FlexDict

Bases: dict

A dict that allows more flexible element access: in addition to obj[‘a’], also allow obj[0]. Lightweight implementation of the Sciris odict class.

keys()

values()

items()

class Contacts(layer_keys=None)

Bases: poliosim.base.FlexDict

A simple (for now) class for storing different contact layers.

add_layer(**kwargs)

Small method to add one or more layers to the contacts. Layers should be provided as keyword arguments.

Example:

hospitals_layer = cv.Layer(label='hosp')
sim.people.contacts.add_layer(hospitals=hospitals_layer)

pop_layer(*args)

Remove the layer(s) from the contacts.

Example:

sim.people.contacts.pop_layer('hospitals')

Note: while included here for convenience, this operation is equivalent to simply popping the key from the contacts dictionary.

to_graph()

Convert all layers to a networkx MultiDiGraph

Example:

import networkx as nx
sim = cv.Sim(pop_size=50, pop_type='hybrid').run()
G = sim.people.contacts.to_graph()
nx.draw(G)

class Layer(label=None, **kwargs)

Bases: poliosim.base.FlexDict

A small class holding a single layer of contact edges (connections) between people.

The input is typically three arrays: person 1 of the connection, person 2 of the connection, and the weight of the connection. Connections are undirected; each person is both a source and sink.

This class is usually not invoked directly by the user, but instead is called as part of the population creation.

Parameters

• p1 (array) – an array of N connections, representing people on one side of the connection

• p2 (array) – an array of people on the other side of the connection

• beta (array) – an array of weights for each connection

• label (str) – the name of the layer (optional)

• kwargs (dict) – other keys copied directly into the layer

Note that all arguments (except for label) must be arrays of the same length, although not all have to be supplied at the time of creation (they must all be the same at the time of initialization, though, or else validation will fail).

Examples:

# Generate an average of 10 contacts for 1000 people
n = 10_000
n_people = 1000
p1 = np.random.randint(n_people, size=n)
p2 = np.random.randint(n_people, size=n)
beta = np.ones(n)
layer = cv.Layer(p1=p1, p2=p2, beta=beta, label='rand')

# Convert one layer to another with extra columns
index = np.arange(n)
self_conn = p1 == p2
layer2 = cv.Layer(**layer, index=index, self_conn=self_conn, label=layer.label)

property members

Return sorted array of all members

meta_keys()

Return the keys for the layer’s meta information – i.e., p1, p2, beta

validate()

Check the integrity of the layer: right types, right lengths

pop_inds(inds)

“Pop” the specified indices from the edgelist and return them as a dict. Returns in the right format to be used with layer.append().

Parameters

inds (int, array, slice) – the indices to be removed

append(contacts)

Append contacts to the current layer.

Parameters

contacts (dict) – a dictionary of arrays with keys p1,p2,beta, as returned from layer.pop_inds()

to_df()

Convert to dataframe

from_df(df, keys=None)

Convert from a dataframe

to_graph()

Convert to a networkx DiGraph

Example:

import networkx as nx
sim = ps.Sim(pop_size=20, pop_type='hybrid').run()
G = sim.people.contacts['h'].to_graph()
nx.draw(G)

find_contacts(inds, as_array=True)

Find all contacts of the specified people

For some purposes (e.g. contact tracing) it’s necessary to find all of the contacts associated with a subset of the people in this layer. Since contacts are bidirectional it’s necessary to check both P1 and P2 for the target indices. The return type is a Set so that there is no duplication of indices (otherwise if the Layer has explicit symmetric interactions, they could appear multiple times). This is also for performance so that the calling code doesn’t need to perform its own unique() operation. Note that this cannot be used for cases where multiple connections count differently than a single infection, e.g. exposure risk.

Parameters

• inds (array) – indices of people whose contacts to return

• as_array (bool) – if true, return as sorted array (otherwise, return as unsorted set)

Returns

a set of indices for pairing partners

Return type

contact_inds (array)

Example: If there were a layer with - P1 = [1,2,3,4] - P2 = [2,3,1,4] Then find_contacts([1,3]) would return {1,2,3}

update(people, frac=1.0)

Regenerate contacts on each timestep.

This method gets called if the layer appears in sim.pars['dynam_layer']. The Layer implements the update procedure so that derived classes can customize the update e.g. implementing over-dispersion/other distributions, random clusters, etc.

Typically, this method also takes in the people object so that the update can depend on person attributes that may change over time (e.g. changing contacts for people that are severe/critical).

Parameters

• people (People) – the Poliosim People object, which is usually used to make new contacts

• frac (float) – the fraction of contacts to update on each timestep