T2 - Plotting, printing, and saving¶
This tutorial provides a brief overview of options for plotting results, printing objects, and saving results.
Click here to open an interactive version of this notebook.
Global plotting configuration¶
HPVsim allows the user to set various options that apply to all plots. You can change the font size, default DPI, whether plots should be shown by default, etc. (for the full list, see hpv.options.help()). For example, we might want higher resolution, to turn off automatic figure display, close figures after they’re rendered, and to turn off the messages that print when a simulation is running. We can do this using built-in defaults for Jupyter notebooks (and then run a sim) with:
[1]:
import hpvsim as hpv
hpv.options(jupyter=True, verbose=0) # Standard options for Jupyter notebook
sim = hpv.Sim()
sim.run()
HPVsim 1.2.0 (2023-05-31) — © 2023 by IDM
Loading location-specific demographic data for "nigeria"
/home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/envs/v1.2.0/lib/python3.9/site-packages/sciris/sc_fileio.py:172: UserWarning: Fixing known unpickling deprecation "No module named 'pandas.core.indexes.numeric'"
obj = _unpickler(filestr, **kw, **kwargs) # Unpickle the data
/home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/envs/v1.2.0/lib/python3.9/site-packages/sciris/sc_fileio.py:172: UserWarning: Fixing known unpickling deprecation "No module named 'pandas.core.indexes.numeric'"
obj = _unpickler(filestr, **kw, **kwargs) # Unpickle the data
[1]:
Sim(<no label>; 1995.0 to 2030.0; pop: 20000 default; epi: 4.66012e+08⚙, 327403♋︎)
Printing objects¶
There are three levels of detail available for most objects (sims, multisims, scenarios, and people). The shortest is brief():
[2]:
sim.brief()
Sim(<no label>; 1995.0 to 2030.0; pop: 20000 default; epi: 4.66012e+08⚙, 327403♋︎)
You can get more detail with summarize():
[3]:
sim.summarize()
Simulation summary:
13,395,204 infections
0 dysplasias
0 pre-cins
3,230,231 cin1s
173,048 cin2s
81,717 cin3s
5,793,372 cins
20,296 cancers
0 cancer detections
17,625 cancer deaths
0 detected cancer deaths
9,683,216 reinfections
0 reactivations
774,427,776 number susceptible
16,136,736 number infectious
120,172 number with inactive infection
260,436,336 number with no cellular changes
39,733,240 number with episomal infection
2,670 number with transformation
120,172 number with cancer
16,256,907 number infected
39,853,408 number with abnormal cells
0 number with latent infection
3,388,858 number with precin
5,666,257 number with cin1
1,453,284 number with cin2
1,126,949 number with cin3
8,122,044 number with detectable dysplasia
0 number with detected cancer
0 number screened
0 number treated for precancerous lesions
0 number treated for cancer
0 number vaccinated
0 number given therapeutic vaccine
0.58 hpv incidence (/100)
0 cin1 incidence (/100,000)
0 cin2 incidence (/100,000)
0 cin3 incidence (/100,000)
0 dysplasia incidence (/100,000)
16 cancer incidence (/100,000)
9,416,166 births
2,604,267 other deaths
-1,089,562 migration
24 age-adjusted cervical cancer incidence (/100,000)
0 age-adjusted cervical cancer mortality
0 newly vaccinated
0 cumulative number vaccinated
0 new doses
0 cumulative doses
0 new therapeutic vaccine doses
0 newly received therapeutic vaccine
0 cumulative therapeutic vaccine doses
0 total received therapeutic vaccine
0 new screens
0 newly screened
0 new cin treatments
0 newly treated for cins
0 new cancer treatments
0 newly treated for cancer
0 cumulative screens
0 cumulative number screened
0 cumulative cin treatments
0 cumulative number treated for cins
0 cumulative cancer treatments
0 cumulative number treated for cancer
0 detected cancer incidence (/100,000)
14 cancer mortality
260,436,336 number alive
0 crude death rate
0 crude birth rate
2.07 hpv prevalence (/100)
0 pre-cin prevalence (/100,000)
0 cin1 prevalence (/100,000)
0 cin2 prevalence (/100,000)
0 cin3 prevalence (/100,000)
Finally, to show the full object, including all methods and attributes, use disp():
[4]:
sim.disp()
<hpvsim.sim.Sim at 0x7f270c0bcf40>
————————————————————————————————————————————————————————————
Methods:
_brief() get_intervention() reset_layer_pars()
_disp() get_interventio... result_keys()
_get_ia() get_t() result_types()
brief() init_analyzers() run()
compute_fit() init_genotypes() save()
compute_results() init_hiv() set_metadata()
compute_states() init_immunity() shrink()
compute_summary() init_interventi... step()
copy() init_people() summarize()
disp() init_results() to_df()
export_pars() init_states() to_excel()
export_results() initialize() to_json()
finalize() layer_keys() update_pars()
finalize_analyz... load() validate_init_c...
get_analyzer() load_data() validate_layer_...
get_analyzers() plot() validate_pars()
————————————————————————————————————————————————————————————
Properties:
n
————————————————————————————————————————————————————————————
_default_ver: None
_orig_pars: {'n_agents': 20000, 'total_pop': None, 'pop_scale':
5340.99025, 'ms [...]
analyzers: []
art_datafile: None
complete: True
created: None
data: None
datafile: None
hiv_datafile: None
hivsim: <hpvsim.hiv.HIVsim at 0x7f26d0e055e0>
initialized: True
interventions: []
label: None
npts: 144
pars: {'n_agents': 20000, 'total_pop': None, 'pop_scale':
5340.99025, 'ms [...]
people: People(n=69683; layers: m, c, o)
popdict: None
popfile: None
res_npts: 36
res_tvec: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, [...]
res_yearvec: array([1995., 1996., 1997., 1998., 1999., 2000., 2001.,
2002., 2003 [...]
resfreq: 4
results: #0. 'infections':
<hpvsim.base.Result at 0x7f26d0e05640>
—————————— [...]
results_ready: True
summary: #0. 'infections': 13395203.80078125
#1. 'dysplasias' [...]
t: 143
tvec: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8,
9, 10, 11, [...]
years: array([1995., 1996., 1997., 1998., 1999., 2000., 2001.,
2002., 2003 [...]
yearvec: array([1995. , 1995.25, 1995.5 , 1995.75, 1996. ,
1996.25, 1996.5 [...]
————————————————————————————————————————————————————————————
Plotting options¶
While a sim can be plotted using default settings simply by sim.plot(), this is just a small fraction of what’s available. First, note that results can be plotted directly using e.g. Matplotlib. You can see what quantities are available for plotting with sim.results.keys() (remember, it’s just a dict). A simple example of plotting using Matplotlib is:
[5]:
import pylab as plt # Shortcut for import matplotlib.pyplot as plt
plt.plot(sim.results['year'], sim.results['infections']);
However, as you can see, this isn’t ideal since the default formatting is…not great. (Also, note that each result is a Result object, not a simple Numpy array; like a pandas dataframe, you can get the array of values directly via e.g. sim.results.infections.values.)
An alternative, you can also select one or more quantities to plot with the first (to_plot) argument, e.g.
[6]:
sim.plot(to_plot=['infections', 'hpv_incidence']);
While we can save this figure using Matplotlib’s built-in savefig(), if we use HPVsim’s hpv.savefig() we get a couple of advantages:
[7]:
hpv.savefig('my-fig.png')
[7]:
'my-fig.png'
<Figure size 614.4x460.8 with 0 Axes>
First, it saves the figure at higher resolution by default (which you can adjust with the dpi argument). But second, it stores information about the code that was used to generate the figure as metadata, which can be loaded later. Made an awesome plot but can’t remember even what script you ran to generate it, much less what version of the code? You’ll never have to worry about that again.
[8]:
hpv.get_png_metadata('my-fig.png')
HPVsim version: 1.2.0
HPVsim branch: Branch N/A
HPVsim hash: Hash N/A
HPVsim date: Date N/A
HPVsim caller branch: Branch N/A
HPVsim caller hash: Hash N/A
HPVsim caller date: Date N/A
HPVsim caller filename: /home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/envs/v1.2.0/lib/python3.9/site-packages/hpvsim/misc.py
HPVsim current time: 2023-Aug-10 17:40:46
HPVsim calling file: /tmp/ipykernel_1239/1797766398.py
Customizing plots¶
We saw above how to set default plot configuration options for Jupyter. HPVsim provides a lot of flexibility in customizing the appearance of plots as well. There are three different levels at which you can set plotting options: global, just for HPVsim, or just for the current plot. To give an example with changing the figure DPI: - Change the setting globally (for both HPVsim and Matplotlib): sc.options(dpi=150) or pl.rc('figure', dpi=150) (where sc is import sciris as sc) -
Change for HPVsim plots, but not for Matplotlib plots: hpv.options(dpi=150) - Change for the current HPVsim plot, but not other HPVsim plots: sim.plot(dpi=150)
The easiest way to change the style of HPVsim plots is with the style argument. For example, to plot using a built-in Matplotlib style would simply be:
[9]:
sim.plot(style='ggplot');
In addition to the default style ('hpvsim'), there is also a “simple” style. You can combine built-in styles with additional overrides, including any valid Matplotlib commands:
[10]:
sim.plot(style='simple', legend_args={'frameon':True}, style_args={'ytick.direction':'in'});
Although most style handling is done automatically, you can also use it yourself in a with block, e.g.:
[11]:
import numpy as np
with hpv.options.with_style(fontsize=6):
sim.plot() # This will have 6 point font
plt.figure(); plt.plot(np.random.rand(20), 'o') # So will this
Saving options¶
Saving sims is also pretty simple. The simplest way to save is simply
[12]:
sim.save('my-sim.sim')
[12]:
'/home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/checkouts/v1.2.0/docs/tutorials/my-sim.sim'
Technically, this saves as a gzipped pickle file (via sc.saveobj() using the Sciris library). By default this does not save the people in the sim since they are very large (and since, if the random seed is saved, they can usually be regenerated). If you want to save the people as well, you can use the keep_people argument. For example, here’s what it would look like to create a sim, run it halfway, save it, load it, change the overall transmissibility (beta), and finish running it:
[13]:
sim_orig = hpv.Sim(start=2000, end=2030, label='Load & save example')
sim_orig.run(until='2015')
sim_orig.save('my-half-finished-sim.sim') # Note: HPVsim always saves the people if the sim isn't finished running yet
sim = hpv.load('my-half-finished-sim.sim')
sim['beta'] *= 0.3
sim.run()
sim.plot(['infections', 'hpv_incidence', 'cancer_incidence']);
Loading location-specific demographic data for "nigeria"
Aside from saving the entire simulation, there are other export options available. You can export the results and parameters to a JSON file (using sim.to_json()), but probably the most useful is to export the results to an Excel workbook, where they can easily be stored and processed with e.g. Pandas:
[14]:
import pandas as pd
sim.to_excel('my-sim.xlsx')
df = pd.read_excel('my-sim.xlsx')
print(df)
Object saved to /home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/checkouts/v1.2.0/docs/tutorials/my-sim.xlsx.
year t infections dysplasias precins cin1s cin2s \
0 2000 0 2.295594e+07 0 0 8.879287e+06 0.000000
1 2001 1 1.867026e+07 0 0 5.408315e+06 0.000000
2 2002 2 1.953710e+07 0 0 4.696056e+06 20003.856140
3 2003 3 1.934918e+07 0 0 4.570578e+06 250957.430359
4 2004 4 1.966440e+07 0 0 4.715454e+06 432810.640625
5 2005 5 1.954316e+07 0 0 4.780315e+06 367949.682617
6 2006 6 1.968864e+07 0 0 4.535420e+06 378860.876953
7 2007 7 1.918551e+07 0 0 4.587551e+06 238227.733887
8 2008 8 1.877938e+07 0 0 4.086242e+06 332185.219238
9 2009 9 1.806409e+07 0 0 4.088667e+06 280053.951660
10 2010 10 1.756702e+07 0 0 3.921968e+06 242470.966919
11 2011 11 1.740335e+07 0 0 3.700107e+06 310969.018555
12 2012 12 1.663957e+07 0 0 3.803763e+06 220648.569519
13 2013 13 1.693053e+07 0 0 3.708593e+06 304907.236816
14 2014 14 1.645772e+07 0 0 3.544319e+06 180640.882019
15 2015 15 1.134764e+07 0 0 3.071501e+06 149725.827515
16 2016 16 9.917063e+06 0 0 2.238007e+06 282478.691406
17 2017 17 8.613782e+06 0 0 2.022814e+06 200644.731445
18 2018 18 8.292508e+06 0 0 1.772463e+06 153969.075439
19 2019 19 7.068029e+06 0 0 1.676081e+06 199432.377136
20 2020 20 6.358802e+06 0 0 1.333590e+06 129721.971069
21 2021 21 5.576833e+06 0 0 1.242058e+06 141845.521545
22 2022 22 4.897914e+06 0 0 9.741272e+05 77590.713867
23 2023 23 4.570578e+06 0 0 9.704901e+05 123660.201050
24 2024 24 4.152316e+06 0 0 8.777449e+05 96988.391479
25 2025 25 4.031080e+06 0 0 8.595596e+05 55768.325195
26 2026 26 3.606756e+06 0 0 7.698453e+05 60617.744507
27 2027 27 3.352161e+06 0 0 6.874052e+05 56374.501099
28 2028 28 3.291544e+06 0 0 6.983164e+05 92745.150635
29 2029 29 2.758107e+06 0 0 6.225442e+05 29702.694153
30 2030 30 2.248918e+06 0 0 4.315983e+05 53949.791626
cin3s cins cancers ... detected_cancer_incidence \
0 0.000000 1.756096e+07 0.000000 ... 0
1 0.000000 1.065842e+07 0.000000 ... 0
2 0.000000 9.079325e+06 0.000000 ... 0
3 0.000000 9.804920e+06 0.000000 ... 0
4 0.000000 9.995866e+06 0.000000 ... 0
5 0.000000 9.946765e+06 0.000000 ... 0
6 7880.306702 9.490920e+06 0.000000 ... 0
7 54555.970398 9.230264e+06 0.000000 ... 0
8 83046.311035 8.780480e+06 0.000000 ... 0
9 135783.746765 8.812608e+06 1818.532288 ... 0
10 145482.576111 8.370098e+06 1818.532288 ... 0
11 101231.633545 7.783318e+06 4243.241943 ... 0
12 115779.889526 7.976082e+06 6667.951599 ... 0
13 155181.423035 8.309480e+06 10911.193787 ... 0
14 132146.678101 7.325048e+06 11517.371216 ... 0
15 106687.227295 6.489130e+06 12729.726074 ... 0
16 138814.628479 5.126443e+06 13942.080872 ... 0
17 194582.939331 4.615435e+06 15760.613220 ... 0
18 131540.500549 4.024412e+06 16972.968262 ... 0
19 104868.698364 3.921362e+06 19397.677917 ... 0
20 142451.701477 3.249111e+06 19397.677856 ... 0
21 101231.635010 2.929050e+06 19397.677979 ... 0
22 139420.808228 2.502907e+06 16366.790405 ... 0
23 110930.477600 2.348938e+06 16366.790527 ... 0
24 96988.392151 2.054335e+06 18791.500244 ... 0
25 107293.406128 2.062216e+06 19397.677856 ... 0
26 92745.153076 1.752459e+06 23034.742310 ... 0
27 98200.750854 1.704571e+06 23034.742615 ... 0
28 88501.904907 1.862177e+06 24247.097412 ... 0
29 79409.246887 1.490590e+06 22428.565063 ... 0
30 87289.556030 1.119004e+06 27277.984253 ... 0
cancer_mortality n_alive cdr cbr hpv_prevalence \
0 0.000000 121329440 0.018086 0.042367 0.054116
1 0.000000 124610064 0.012701 0.041349 0.051365
2 0.000000 128046488 0.011935 0.041281 0.051405
3 0.000000 131650816 0.012446 0.041072 0.049731
4 0.000000 135229696 0.011852 0.040971 0.048750
5 0.000000 139002528 0.011696 0.040775 0.047766
6 0.000000 142874800 0.011349 0.040603 0.046342
7 0.000000 146773136 0.011671 0.040350 0.045154
8 0.000000 150846640 0.012285 0.040105 0.043532
9 0.000000 155065648 0.011610 0.039873 0.041145
10 0.000000 159339184 0.010694 0.039641 0.040203
11 0.000000 163809120 0.011720 0.039262 0.038165
12 0.000000 168427600 0.010585 0.038978 0.036913
13 0.000000 173154576 0.010558 0.038614 0.036185
14 1.375809 177821536 0.010741 0.038214 0.034670
15 0.671736 182446048 0.007432 0.037943 0.028363
16 2.618504 187029344 0.009989 0.037370 0.024505
17 1.914853 191921824 0.009561 0.036891 0.021795
18 5.594198 196771840 0.010345 0.036444 0.019928
19 6.676378 201663088 0.010322 0.036011 0.017581
20 5.931726 206600432 0.010815 0.036001 0.015151
21 11.585338 211642000 0.010824 0.036003 0.013346
22 9.607343 216767840 0.009765 0.036074 0.011911
23 8.269795 221971872 0.009566 0.036102 0.011097
24 10.226381 227252288 0.009797 0.036090 0.009827
25 11.057172 232609664 0.009632 0.036119 0.009011
26 14.389526 238055552 0.009360 0.036133 0.008233
27 9.565140 243512384 0.009634 0.036145 0.007232
28 11.812885 249046752 0.009753 0.036169 0.006563
29 12.027443 254641776 0.009648 0.036160 0.005866
30 11.780783 260377440 0.008900 0.036202 0.005150
precin_prevalence cin1_prevalence cin2_prevalence cin3_prevalence
0 0.039634 0.027113 0.000000 0.000000
1 0.026444 0.037956 0.000000 0.000000
2 0.026467 0.037946 0.000067 0.000000
3 0.024254 0.037543 0.001464 0.000000
4 0.023353 0.035061 0.004453 0.000000
5 0.023000 0.032789 0.006474 0.000064
6 0.021476 0.030628 0.008433 0.000165
7 0.019930 0.029945 0.008616 0.000440
8 0.019588 0.027805 0.008613 0.000914
9 0.018713 0.025648 0.008716 0.001597
10 0.017022 0.025076 0.008304 0.001888
11 0.016480 0.023497 0.008205 0.002259
12 0.016243 0.022171 0.008262 0.002605
13 0.015786 0.021827 0.007662 0.003279
14 0.014414 0.021057 0.007106 0.003536
15 0.009595 0.019906 0.006668 0.003818
16 0.008325 0.017079 0.006479 0.004039
17 0.007093 0.014900 0.005857 0.004379
18 0.006752 0.013271 0.005588 0.004330
19 0.004878 0.012030 0.005248 0.004514
20 0.004577 0.010543 0.004815 0.004289
21 0.003586 0.009836 0.004244 0.004215
22 0.003257 0.008876 0.003677 0.004244
23 0.003457 0.007943 0.003424 0.004218
24 0.002851 0.007424 0.003073 0.004121
25 0.002399 0.006926 0.002829 0.004132
26 0.002441 0.006676 0.002431 0.003916
27 0.002077 0.006476 0.002317 0.003643
28 0.002113 0.005803 0.002148 0.003436
29 0.001676 0.005699 0.001973 0.003175
30 0.001282 0.005490 0.001976 0.002980
[31 rows x 77 columns]