T2 - Plotting, printing, and saving¶
This tutorial provides a brief overview of options for plotting results, printing objects, and saving results.
Click here to open an interactive version of this notebook.
Global plotting configuration¶
HPVsim allows the user to set various options that apply to all plots. You can change the font size, default DPI, whether plots should be shown by default, etc. (for the full list, see hpv.options.help()). For example, we might want higher resolution, to turn off automatic figure display, close figures after they’re rendered, and to turn off the messages that print when a simulation is running. We can do this using built-in defaults for Jupyter notebooks (and then run a sim) with:
[1]:
import hpvsim as hpv
hpv.options(jupyter=True, verbose=0) # Standard options for Jupyter notebook
sim = hpv.Sim()
sim.run()
HPVsim 1.2.7 (2023-09-22) — © 2023 by IDM
Loading location-specific demographic data for "nigeria"
[1]:
Sim(<no label>; 1995.0 to 2030.0; pop: 20000 default; epi: 4.78163e+08⚙, 361051♋︎)
Printing objects¶
There are three levels of detail available for most objects (sims, multisims, scenarios, and people). The shortest is brief():
[2]:
sim.brief()
Sim(<no label>; 1995.0 to 2030.0; pop: 20000 default; epi: 4.78163e+08⚙, 361051♋︎)
You can get more detail with summarize():
[3]:
sim.summarize()
Simulation summary:
478,162,867 total HPV infections
361,051 total cancers
193,878 total cancer deaths
3.09 mean HPV prevalence (%)
9.96 mean cancer incidence (per 100k)
35.01 mean age of infection (years)
49.41 mean age of cancer (years)
Finally, to show the full object, including all methods and attributes, use disp():
[4]:
sim.disp()
<hpvsim.sim.Sim at 0x7f5a34489850>
[<class 'hpvsim.sim.Sim'>, <class 'hpvsim.base.BaseSim'>, <class 'hpvsim.base.ParsObj'>, <class 'hpvsim.base.FlexPretty'>, <class 'sciris.sc_utils.prettyobj'>, <class 'object'>]
————————————————————————————————————————————————————————————
Methods:
_brief() get_intervention() reset_layer_pars()
_disp() get_interventio... result_keys()
_get_ia() get_t() result_types()
brief() init_analyzers() run()
compute_age_mean() init_genotypes() save()
compute_fit() init_hiv() set_metadata()
compute_results() init_immunity() shrink()
compute_states() init_interventi... step()
compute_summary() init_people() summarize()
copy() init_results() to_df()
disp() init_states() to_excel()
export_pars() init_time_vecs() to_json()
export_results() initialize() update_pars()
finalize() layer_keys() validate_dt()
finalize_analyz... load() validate_init_c...
get_analyzer() load_data() validate_layer_...
get_analyzers() plot() validate_pars()
————————————————————————————————————————————————————————————
Properties:
n
————————————————————————————————————————————————————————————
_default_ver: None
_orig_pars: {'n_agents': 20000, 'total_pop': None, 'pop_scale':
5340.99025, 'ms [...]
analyzers: []
art_datafile: None
complete: True
created: None
data: None
datafile: None
hiv_datafile: None
hivsim: <hpvsim.hiv.HIVsim at 0x7f59f90be610>
initialized: True
interventions: []
label: None
npts: 144
pars: {'n_agents': 20000, 'total_pop': None, 'pop_scale':
5340.99025, 'ms [...]
people: People(n=70208; layers: m, c, o)
popdict: None
popfile: None
res_npts: 36
res_tvec: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, [...]
res_yearvec: array([1995., 1996., 1997., 1998., 1999., 2000., 2001.,
2002., 2003 [...]
resfreq: 4
results: #0. 'infections':
<hpvsim.base.Result at 0x7f59f90bebb0>
[<class 'h [...]
results_ready: True
short_summary: #0: 'total HPV infections':
478162867.1484375
#1: 'tota [...]
summary: #0. 'infections': 12396439.044921875
#1. 'dysplasias [...]
t: 143
tvec: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8,
9, 10, 11, [...]
years: array([1995., 1996., 1997., 1998., 1999., 2000., 2001.,
2002., 2003 [...]
yearvec: array([1995. , 1995.25, 1995.5 , 1995.75, 1996. ,
1996.25, 1996.5 [...]
————————————————————————————————————————————————————————————
Plotting options¶
While a sim can be plotted using default settings simply by sim.plot(), this is just a small fraction of what’s available. First, note that results can be plotted directly using e.g. Matplotlib. You can see what quantities are available for plotting with sim.results.keys() (remember, it’s just a dict). A simple example of plotting using Matplotlib is:
[5]:
import pylab as plt # Shortcut for import matplotlib.pyplot as plt
plt.plot(sim.results['year'], sim.results['infections']);
However, as you can see, this isn’t ideal since the default formatting is…not great. (Also, note that each result is a Result object, not a simple Numpy array; like a pandas dataframe, you can get the array of values directly via e.g. sim.results.infections.values.)
An alternative, you can also select one or more quantities to plot with the first (to_plot) argument, e.g.
[6]:
sim.plot(to_plot=['infections', 'hpv_incidence']);
While we can save this figure using Matplotlib’s built-in savefig(), if we use HPVsim’s hpv.savefig() we get a couple of advantages:
[7]:
hpv.savefig('my-fig.png')
[7]:
'my-fig.png'
<Figure size 614.4x460.8 with 0 Axes>
First, it saves the figure at higher resolution by default (which you can adjust with the dpi argument). But second, it stores information about the code that was used to generate the figure as metadata, which can be loaded later. Made an awesome plot but can’t remember even what script you ran to generate it, much less what version of the code? You’ll never have to worry about that again.
[8]:
hpv.get_png_metadata('my-fig.png')
HPVsim version: 1.2.7
HPVsim branch: Branch N/A
HPVsim hash: Hash N/A
HPVsim date: Date N/A
HPVsim caller branch: Branch N/A
HPVsim caller hash: Hash N/A
HPVsim caller date: Date N/A
HPVsim caller filename: /home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/envs/latest/lib/python3.9/site-packages/hpvsim/misc.py
HPVsim current time: 2023-Sep-27 12:18:11
HPVsim calling file: /tmp/ipykernel_1219/1797766398.py
Customizing plots¶
We saw above how to set default plot configuration options for Jupyter. HPVsim provides a lot of flexibility in customizing the appearance of plots as well. There are three different levels at which you can set plotting options: global, just for HPVsim, or just for the current plot. To give an example with changing the figure DPI: - Change the setting globally (for both HPVsim and Matplotlib): sc.options(dpi=150) or pl.rc('figure', dpi=150) (where sc is import sciris as sc) -
Change for HPVsim plots, but not for Matplotlib plots: hpv.options(dpi=150) - Change for the current HPVsim plot, but not other HPVsim plots: sim.plot(dpi=150)
The easiest way to change the style of HPVsim plots is with the style argument. For example, to plot using a built-in Matplotlib style would simply be:
[9]:
sim.plot(style='ggplot');
In addition to the default style ('hpvsim'), there is also a “simple” style. You can combine built-in styles with additional overrides, including any valid Matplotlib commands:
[10]:
sim.plot(style='simple', legend_args={'frameon':True}, style_args={'ytick.direction':'in'});
Although most style handling is done automatically, you can also use it yourself in a with block, e.g.:
[11]:
import numpy as np
with hpv.options.with_style(fontsize=6):
sim.plot() # This will have 6 point font
plt.figure(); plt.plot(np.random.rand(20), 'o') # So will this
Saving options¶
Saving sims is also pretty simple. The simplest way to save is simply
[12]:
sim.save('my-sim.sim')
[12]:
'/home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/checkouts/latest/docs/tutorials/my-sim.sim'
Technically, this saves as a gzipped pickle file (via sc.saveobj() using the Sciris library). By default this does not save the people in the sim since they are very large (and since, if the random seed is saved, they can usually be regenerated). If you want to save the people as well, you can use the keep_people argument. For example, here’s what it would look like to create a sim, run it halfway, save it, load it, change the overall transmissibility (beta), and finish running it:
[13]:
sim_orig = hpv.Sim(start=2000, end=2030, label='Load & save example')
sim_orig.run(until='2015')
sim_orig.save('my-half-finished-sim.sim') # Note: HPVsim always saves the people if the sim isn't finished running yet
sim = hpv.load('my-half-finished-sim.sim')
sim['beta'] *= 0.3
sim.run()
sim.plot(['infections', 'hpv_incidence', 'cancer_incidence']);
Loading location-specific demographic data for "nigeria"
Aside from saving the entire simulation, there are other export options available. You can export the results and parameters to a JSON file (using sim.to_json()), but probably the most useful is to export the results to an Excel workbook, where they can easily be stored and processed with e.g. Pandas:
[14]:
import pandas as pd
sim.to_excel('my-sim.xlsx')
df = pd.read_excel('my-sim.xlsx')
print(df)
Object saved to /home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/checkouts/latest/docs/tutorials/my-sim.xlsx.
year t infections dysplasias precins cin1s cin2s \
0 2000 0 2.224065e+07 0 0 8.846553e+06 0.000000
1 2001 1 1.883393e+07 0 0 5.732620e+06 0.000000
2 2002 2 1.960378e+07 0 0 4.706968e+06 19397.678711
3 2003 3 1.920976e+07 0 0 4.765161e+06 199432.370789
4 2004 4 1.934918e+07 0 0 4.603312e+06 389165.888672
5 2005 5 1.927038e+07 0 0 4.509960e+06 376436.175781
6 2006 6 1.862783e+07 0 0 4.457223e+06 404926.494141
7 2007 7 1.820957e+07 0 0 4.146254e+06 335822.283203
8 2008 8 1.798528e+07 0 0 4.030474e+06 334003.756836
9 2009 9 1.751246e+07 0 0 3.861350e+06 275810.728027
10 2010 10 1.788829e+07 0 0 4.017138e+06 341277.880859
11 2011 11 1.825200e+07 0 0 4.114732e+06 317636.978516
12 2012 12 1.752459e+07 0 0 3.864987e+06 339459.350586
13 2013 13 1.676687e+07 0 0 3.820130e+06 281872.501465
14 2014 14 1.693054e+07 0 0 3.644945e+06 238833.908203
15 2015 15 1.074753e+07 0 0 3.096354e+06 314606.080078
16 2016 16 9.680654e+06 0 0 2.327721e+06 299451.646484
17 2017 17 8.898685e+06 0 0 1.787617e+06 289146.622742
18 2018 18 7.704515e+06 0 0 1.827019e+06 307938.132324
19 2019 19 6.898299e+06 0 0 1.505745e+06 262474.818726
20 2020 20 6.292122e+06 0 0 1.466343e+06 217011.529785
21 2021 21 5.855674e+06 0 0 1.310556e+06 230953.598633
22 2022 22 5.376794e+06 0 0 1.168710e+06 149725.828735
23 2023 23 4.340231e+06 0 0 1.052930e+06 173366.753296
24 2024 24 3.709806e+06 0 0 8.807758e+05 168517.324219
25 2025 25 3.782548e+06 0 0 6.771002e+05 134571.397217
26 2026 26 3.467335e+06 0 0 7.074091e+05 177003.815918
27 2027 27 3.418841e+06 0 0 7.146832e+05 136996.106201
28 2028 28 3.097567e+06 0 0 6.898299e+05 133359.039795
29 2029 29 2.673243e+06 0 0 6.183010e+05 118204.603149
30 2030 30 2.770231e+06 0 0 5.564709e+05 112142.828491
cin3s cins cancers ... detected_cancer_incidence \
0 0.000000 1.763916e+07 0.000000 ... 0
1 0.000000 1.116700e+07 0.000000 ... 0
2 0.000000 9.238144e+06 0.000000 ... 0
3 0.000000 1.015468e+07 0.000000 ... 0
4 0.000000 1.009952e+07 0.000000 ... 0
5 0.000000 9.513954e+06 0.000000 ... 0
6 1818.532288 9.563055e+06 0.000000 ... 0
7 32127.403259 8.538010e+06 0.000000 ... 0
8 99413.098877 8.363430e+06 0.000000 ... 0
9 77590.710693 8.006998e+06 1212.354858 ... 0
10 91532.790710 8.521642e+06 3030.887146 ... 0
11 95169.857605 8.642272e+06 5455.596863 ... 0
12 118204.599487 7.979720e+06 5455.596802 ... 0
13 89108.081604 8.033063e+06 7880.306458 ... 0
14 76378.357666 7.302013e+06 12123.548584 ... 0
15 113961.353760 6.733419e+06 11517.371155 ... 0
16 116992.246643 5.267682e+06 9698.838806 ... 0
17 69710.401978 4.302647e+06 11517.371216 ... 0
18 81227.773621 4.205053e+06 13942.080750 ... 0
19 66073.338928 3.566142e+06 16972.967957 ... 0
20 103050.165100 3.521891e+06 20610.032532 ... 0
21 55768.326904 3.092717e+06 21216.210083 ... 0
22 69104.226074 2.668999e+06 22428.564453 ... 0
23 53949.789917 2.405918e+06 20610.032349 ... 0
24 49706.550476 2.261042e+06 16972.968079 ... 0
25 63042.455322 1.616069e+06 20610.032776 ... 0
26 28490.338379 1.854297e+06 18185.323181 ... 0
27 55768.322388 1.916733e+06 16972.968018 ... 0
28 37582.999756 1.701540e+06 18791.500000 ... 0
29 26065.628967 1.574243e+06 23640.919128 ... 0
30 35158.289551 1.365718e+06 15154.435669 ... 0
cancer_mortality n_alive cdr cbr hpv_prevalence \
0 0.000000 121323984 0.018087 0.042669 0.053737
1 0.000000 124639776 0.011784 0.041826 0.050849
2 0.000000 128044672 0.012654 0.041755 0.050786
3 0.000000 131551408 0.013667 0.041563 0.049655
4 0.000000 135173920 0.012839 0.041391 0.048826
5 0.000000 138992224 0.010781 0.041344 0.047033
6 0.000000 142788112 0.011993 0.041137 0.045432
7 0.000000 146754320 0.011970 0.040934 0.044291
8 0.000000 150778736 0.011595 0.040927 0.042383
9 0.000000 155018352 0.011223 0.040668 0.042165
10 0.000000 159311904 0.010935 0.040371 0.041407
11 0.000000 163753360 0.011020 0.040090 0.040664
12 0.000000 168415456 0.009981 0.039556 0.039001
13 0.000000 173101232 0.010173 0.038941 0.036681
14 0.677415 177803936 0.010200 0.038456 0.035289
15 1.322174 182359984 0.007153 0.038094 0.027769
16 4.516595 186971792 0.010660 0.037608 0.024068
17 0.000000 191807856 0.009974 0.037197 0.021545
18 5.532742 196654848 0.010767 0.036743 0.018541
19 5.998116 201577632 0.010269 0.036417 0.016631
20 7.035070 206539200 0.011282 0.036364 0.014826
21 4.004600 211591664 0.010987 0.036384 0.013547
22 4.477028 216684176 0.010242 0.036452 0.012003
23 7.100682 221916096 0.009779 0.036466 0.010156
24 12.263659 227223168 0.008740 0.036522 0.009112
25 13.546464 232530848 0.010245 0.036496 0.008358
26 14.765185 237947680 0.010149 0.036481 0.007476
27 10.472459 243462672 0.009675 0.036526 0.007060
28 10.225796 249037680 0.009702 0.036536 0.006520
29 14.753054 254622384 0.009666 0.036567 0.005861
30 15.343770 260305312 0.009136 0.036584 0.005357
precin_prevalence cin1_prevalence cin2_prevalence cin3_prevalence
0 0.039198 0.026112 0.000000 0.000000
1 0.026283 0.038388 0.000000 0.000000
2 0.026586 0.038531 0.000035 0.000000
3 0.023367 0.039241 0.001498 0.000000
4 0.023493 0.034741 0.004157 0.000000
5 0.022160 0.031991 0.006238 0.000029
6 0.021275 0.029410 0.007678 0.000099
7 0.020150 0.027882 0.008472 0.000324
8 0.018231 0.027349 0.008450 0.000801
9 0.018451 0.025965 0.008431 0.001221
10 0.018216 0.025331 0.008790 0.001536
11 0.018077 0.024881 0.008325 0.002041
12 0.015984 0.024436 0.008071 0.002427
13 0.015091 0.022571 0.007694 0.002774
14 0.016170 0.021578 0.006679 0.003234
15 0.009733 0.020183 0.006205 0.003281
16 0.007433 0.018015 0.005904 0.003628
17 0.007286 0.015625 0.005667 0.003670
18 0.005748 0.013869 0.005397 0.003701
19 0.005852 0.012138 0.005176 0.003549
20 0.004657 0.011162 0.004592 0.003830
21 0.004178 0.010399 0.004298 0.003709
22 0.004037 0.009363 0.003970 0.003587
23 0.002950 0.008814 0.003359 0.003443
24 0.002344 0.008332 0.003123 0.003475
25 0.002635 0.007433 0.002907 0.003411
26 0.001994 0.007169 0.002721 0.003381
27 0.002327 0.006661 0.002385 0.003321
28 0.001889 0.006569 0.002209 0.003129
29 0.001858 0.006206 0.001878 0.002966
30 0.001669 0.006086 0.001846 0.002881
[31 rows x 77 columns]