T2 - Plotting, printing, and saving

This tutorial provides a brief overview of options for plotting results, printing objects, and saving results.

Click here to open an interactive version of this notebook.

Global plotting configuration

HPVsim allows the user to set various options that apply to all plots. You can change the font size, default DPI, whether plots should be shown by default, etc. (for the full list, see hpv.options.help()). For example, we might want higher resolution, to turn off automatic figure display, close figures after they’re rendered, and to turn off the messages that print when a simulation is running. We can do this using built-in defaults for Jupyter notebooks (and then run a sim) with:

[1]:
import hpvsim as hpv

hpv.options(jupyter=True, verbose=0) # Standard options for Jupyter notebook

sim = hpv.Sim()
sim.run()
HPVsim 1.1.1 (2023-03-01) — © 2023 by IDM
Loading location-specific demographic data for "nigeria"
[1]:
Sim(<no label>; 1995.0 to 2030.0; pop: 20000 default; epi: 1.68343e+09⚙, 202958♋︎)

Printing objects

There are three levels of detail available for most objects (sims, multisims, scenarios, and people). The shortest is brief():

[2]:
sim.brief()
Sim(<no label>; 1995.0 to 2030.0; pop: 20000 default; epi: 1.68343e+09⚙, 202958♋︎)

You can get more detail with summarize():

[3]:
sim.summarize()
Simulation summary:
   74,687,882 infections
            0 dysplasias
            0 pre-cins
    2,660,347 cin1s
      553,327 cin2s
      303,902 cin3s
    6,916,582 cins
       14,421 cancers
            0 cancer detections
        6,943 cancer deaths
            0 detected cancer deaths
   62,028,132 reinfections
            0 reactivations
   665,669,184 number susceptible
   83,682,120 number infectious
      129,252 number with inactive infection
   259,850,416 number with no cellular changes
   83,682,120 number with episomal infection
      128,718 number with transformation
      129,252 number with cancer
   83,811,360 number infected
   83,811,360 number with abnormal cells
            0 number with latent infection
   43,536,556 number with precin
   18,460,068 number with cin1
    3,408,620 number with cin2
    2,663,552 number with cin3
    1,426,044 number with carcinoma in situ
   24,525,296 number with detectable dysplasia
            0 number with detected cancer
            0 number screened
            0 number treated for precancerous lesions
            0 number treated for cancer
            0 number vaccinated
            0 number given therapeutic vaccine
         3.74 hpv incidence (/100)
            0 cin1 incidence (/100,000)
            0 cin2 incidence (/100,000)
            0 cin3 incidence (/100,000)
            0 dysplasia incidence (/100,000)
           11 cancer incidence (/100,000)
    9,410,825 births
    2,651,268 other deaths
   -1,036,152 migration
           17 age-adjusted cervical cancer incidence (/100,000)
            0 age-adjusted cervical cancer mortality
            0 newly vaccinated
            0 cumulative number vaccinated
            0 new doses
            0 cumulative doses
            0 new therapeutic vaccine doses
            0 newly received therapeutic vaccine
            0 cumulative therapeutic vaccine doses
            0 total received therapeutic vaccine
            0 new screens
            0 newly screened
            0 new cin treatments
            0 newly treated for cins
            0 new cancer treatments
            0 newly treated for cancer
            0 cumulative screens
            0 cumulative number screened
            0 cumulative cin treatments
            0 cumulative number treated for cins
            0 cumulative cancer treatments
            0 cumulative number treated for cancer
            0 detected cancer incidence (/100,000)
            5 cancer mortality
   259,850,416 number alive
            0 crude death rate
            0 crude birth rate
        10.73 hpv prevalence (/100)
            0 pre-cin prevalence (/100,000)
            0 cin1 prevalence (/100,000)
            0 cin2 prevalence (/100,000)
            0 cin3 prevalence (/100,000)

Finally, to show the full object, including all methods and attributes, use disp():

[4]:
sim.disp()
<hpvsim.sim.Sim at 0x7f88bc05ff10>
————————————————————————————————————————————————————————————
Methods:
  _brief()            get_intervention()  reset_layer_pars()
  _disp()             get_interventio...  result_keys()
  _get_ia()           get_t()             result_types()
  brief()             init_analyzers()    run()
  compute_fit()       init_genotypes()    save()
  compute_results()   init_hiv()          set_metadata()
  compute_states()    init_immunity()     shrink()
  compute_summary()   init_interventi...  step()
  copy()              init_people()       summarize()
  disp()              init_results()      to_df()
  export_pars()       init_states()       to_excel()
  export_results()    initialize()        to_json()
  finalize()          layer_keys()        update_pars()
  finalize_analyz...  load()              validate_init_c...
  get_analyzer()      load_data()         validate_layer_...
  get_analyzers()     plot()              validate_pars()
————————————————————————————————————————————————————————————
Properties:
  n
————————————————————————————————————————————————————————————
 _default_ver: None
   _orig_pars: {'n_agents': 20000, 'total_pop': None, 'pop_scale':
               5340.99025, 'ms [...]
    analyzers: []
 art_datafile: None
     complete: True
      created: None
         data: None
     datafile: None
 hiv_datafile: None
       hivsim: <hpvsim.hiv.HIVsim at 0x7f8882ca2e20>

  initialized: True
interventions: []
        label: None
         npts: 144
         pars: {'n_agents': 20000, 'total_pop': None, 'pop_scale':
               5340.99025, 'ms [...]
       people: People(n=69135; layers: m, c, o)
      popdict: None
      popfile: None
     res_npts: 36
     res_tvec: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11,
               12, 13, 14,  [...]
  res_yearvec: array([1995., 1996., 1997., 1998., 1999., 2000., 2001.,
               2002., 2003 [...]
      resfreq: 4
      results: #0. 'infections':
               <hpvsim.base.Result at 0x7f8882ca2dc0>
               —————————— [...]
results_ready: True
      summary: #0. 'infections':                74687882.30273438
               #1. 'dysplasias' [...]
            t: 143
         tvec: array([  0,   1,   2,   3,   4,   5,   6,   7,   8,
               9,  10,  11,  [...]
        years: array([1995., 1996., 1997., 1998., 1999., 2000., 2001.,
               2002., 2003 [...]
      yearvec: array([1995.  , 1995.25, 1995.5 , 1995.75, 1996.  ,
               1996.25, 1996.5 [...]
————————————————————————————————————————————————————————————

Plotting options

While a sim can be plotted using default settings simply by sim.plot(), this is just a small fraction of what’s available. First, note that results can be plotted directly using e.g. Matplotlib. You can see what quantities are available for plotting with sim.results.keys() (remember, it’s just a dict). A simple example of plotting using Matplotlib is:

[5]:
import pylab as plt # Shortcut for import matplotlib.pyplot as plt
plt.plot(sim.results['year'], sim.results['infections']);
../_images/tutorials_tut_plotting_11_0.svg

However, as you can see, this isn’t ideal since the default formatting is…not great. (Also, note that each result is a Result object, not a simple Numpy array; like a pandas dataframe, you can get the array of values directly via e.g. sim.results.infections.values.)

An alternative, you can also select one or more quantities to plot with the first (to_plot) argument, e.g.

[6]:
sim.plot(to_plot=['infections', 'hpv_incidence']);
../_images/tutorials_tut_plotting_13_0.svg

While we can save this figure using Matplotlib’s built-in savefig(), if we use HPVsim’s hpv.savefig() we get a couple of advantages:

[7]:
hpv.savefig('my-fig.png')
[7]:
'my-fig.png'
<Figure size 614.4x460.8 with 0 Axes>

First, it saves the figure at higher resolution by default (which you can adjust with the dpi argument). But second, it stores information about the code that was used to generate the figure as metadata, which can be loaded later. Made an awesome plot but can’t remember even what script you ran to generate it, much less what version of the code? You’ll never have to worry about that again.

[8]:
hpv.get_png_metadata('my-fig.png')
HPVsim version: 1.1.1
HPVsim branch: Branch N/A
HPVsim hash: Hash N/A
HPVsim date: Date N/A
HPVsim caller branch: Branch N/A
HPVsim caller hash: Hash N/A
HPVsim caller date: Date N/A
HPVsim caller filename: /home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/envs/latest/lib/python3.9/site-packages/hpvsim/misc.py
HPVsim current time: 2023-Mar-10 03:25:25
HPVsim calling file: /tmp/ipykernel_1209/1797766398.py

Customizing plots

We saw above how to set default plot configuration options for Jupyter. HPVsim provides a lot of flexibility in customizing the appearance of plots as well. There are three different levels at which you can set plotting options: global, just for HPVsim, or just for the current plot. To give an example with changing the figure DPI: - Change the setting globally (for both HPVsim and Matplotlib): sc.options(dpi=150) or pl.rc('figure', dpi=150) (where sc is import sciris as sc) - Change for HPVsim plots, but not for Matplotlib plots: hpv.options(dpi=150) - Change for the current HPVsim plot, but not other HPVsim plots: sim.plot(dpi=150)

The easiest way to change the style of HPVsim plots is with the style argument. For example, to plot using a built-in Matplotlib style would simply be:

[9]:
sim.plot(style='ggplot');
../_images/tutorials_tut_plotting_19_0.svg

In addition to the default style ('hpvsim'), there is also a “simple” style. You can combine built-in styles with additional overrides, including any valid Matplotlib commands:

[10]:
sim.plot(style='simple', legend_args={'frameon':True}, style_args={'ytick.direction':'in'});
../_images/tutorials_tut_plotting_21_0.svg

Although most style handling is done automatically, you can also use it yourself in a with block, e.g.:

[11]:
import numpy as np
with hpv.options.with_style(fontsize=6):
    sim.plot() # This will have 6 point font
    plt.figure(); plt.plot(np.random.rand(20), 'o') # So will this
../_images/tutorials_tut_plotting_23_0.svg
../_images/tutorials_tut_plotting_23_1.svg

Saving options

Saving sims is also pretty simple. The simplest way to save is simply

[12]:
sim.save('my-sim.sim')
[12]:
'/home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/checkouts/latest/docs/tutorials/my-sim.sim'

Technically, this saves as a gzipped pickle file (via sc.saveobj() using the Sciris library). By default this does not save the people in the sim since they are very large (and since, if the random seed is saved, they can usually be regenerated). If you want to save the people as well, you can use the keep_people argument. For example, here’s what it would look like to create a sim, run it halfway, save it, load it, change the overall transmissibility (beta), and finish running it:

[13]:
sim_orig = hpv.Sim(start=2000, end=2030, label='Load & save example')
sim_orig.run(until='2015')
sim_orig.save('my-half-finished-sim.sim') # Note: HPVsim always saves the people if the sim isn't finished running yet

sim = hpv.load('my-half-finished-sim.sim')
sim['beta'] *= 0.3
sim.run()
sim.plot(['infections', 'hpv_incidence', 'cancer_incidence']);
Loading location-specific demographic data for "nigeria"
Loading location-specific demographic data for "nigeria"
../_images/tutorials_tut_plotting_27_1.svg

Aside from saving the entire simulation, there are other export options available. You can export the results and parameters to a JSON file (using sim.to_json()), but probably the most useful is to export the results to an Excel workbook, where they can easily be stored and processed with e.g. Pandas:

[14]:
import pandas as pd

sim.to_excel('my-sim.xlsx')
df = pd.read_excel('my-sim.xlsx')
print(df)
Object saved to /home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/checkouts/latest/docs/tutorials/my-sim.xlsx.
    year   t    infections  dysplasias  precins         cin1s          cin2s  \
0   2000   0  2.232612e+07           0        0  0.000000e+00       0.000000
1   2001   1  2.166114e+07           0        0  0.000000e+00       0.000000
2   2002   2  2.500664e+07           0        0  9.329071e+05       0.000000
3   2003   3  2.770534e+07           0        0  1.519687e+06       0.000000
4   2004   4  3.086716e+07           0        0  9.510924e+05       0.000000
5   2005   5  3.284633e+07           0        0  1.665776e+06       0.000000
6   2006   6  3.583236e+07           0        0  1.498471e+06       0.000000
7   2007   7  3.820130e+07           0        0  1.423911e+06       0.000000
8   2008   8  4.023745e+07           0        0  1.525749e+06       0.000000
9   2009   9  4.338533e+07           0        0  1.615463e+06    6061.774414
10  2010  10  4.492138e+07           0        0  1.577880e+06  113961.356445
11  2011  11  4.578397e+07           0        0  2.013722e+06  217617.699219
12  2012  12  4.798985e+07           0        0  2.142837e+06  291571.343445
13  2013  13  4.960228e+07           0        0  2.236795e+06  346127.310547
14  2014  14  5.193243e+07           0        0  2.341057e+06  355219.972656
15  2015  15  3.716413e+07           0        0  2.385308e+06  334609.934570
16  2016  16  3.709078e+07           0        0  2.092525e+06  394015.323730
17  2017  17  3.615545e+07           0        0  2.433802e+06  343096.419922
18  2018  18  3.481883e+07           0        0  1.834293e+06  326123.454102
19  2019  19  3.371195e+07           0        0  1.836718e+06  386741.195312
20  2020  20  3.246565e+07           0        0  1.840961e+06  344308.777344
21  2021  21  3.231471e+07           0        0  1.668200e+06  356432.320312
22  2022  22  3.109387e+07           0        0  1.598490e+06  442509.502930
23  2023  23  3.092475e+07           0        0  1.559088e+06  417050.068359
24  2024  24  2.896558e+07           0        0  1.538478e+06  455239.239258
25  2025  25  2.931595e+07           0        0  1.381478e+06  465544.261719
26  2026  26  2.853641e+07           0        0  1.474224e+06  389772.089355
27  2027  27  2.807996e+07           0        0  1.349957e+06  375223.830078
28  2028  28  2.750530e+07           0        0  1.319648e+06  317636.974609
29  2029  29  2.689973e+07           0        0  1.214173e+06  273386.015808
30  2030  30  2.632750e+07           0        0  1.243270e+06  323698.748047

            cin3s          cins       cancers  ...  detected_cancer_incidence  \
0        0.000000  0.000000e+00      0.000000  ...                          0
1        0.000000  0.000000e+00      0.000000  ...                          0
2        0.000000  2.013115e+06      0.000000  ...                          0
3        0.000000  4.090485e+06   1212.354858  ...                          0
4        0.000000  2.396220e+06      0.000000  ...                          0
5        0.000000  3.326702e+06    606.177429  ...                          0
6        0.000000  3.273358e+06   1212.354858  ...                          0
7        0.000000  3.058165e+06   2424.709656  ...                          0
8        0.000000  3.312760e+06    606.177429  ...                          0
9        0.000000  3.507949e+06   4243.242004  ...                          0
10       0.000000  3.749207e+06   1818.532288  ...                          0
11       0.000000  5.023393e+06   2424.709717  ...                          0
12       0.000000  5.321025e+06   6061.774292  ...                          0
13    6061.774414  5.567134e+06   3637.064575  ...                          0
14   33945.937683  5.859917e+06   3637.064575  ...                          0
15  139420.811035  5.990852e+06   5455.596863  ...                          0
16  183671.760254  5.720497e+06  12729.726135  ...                          0
17  201857.093262  6.209681e+06  10911.193726  ...                          0
18  215799.171875  4.860331e+06   4849.419434  ...                          0
19  259443.941895  4.777891e+06   7274.129089  ...                          0
20  201857.084717  4.795470e+06   7880.306519  ...                          0
21  229741.248535  4.483895e+06   9092.661438  ...                          0
22  240046.267090  4.429338e+06   5455.596924  ...                          0
23  275810.730957  4.704543e+06   6667.951599  ...                          0
24  216405.354492  4.594825e+06   9698.838745  ...                          0
25  260050.123047  4.168682e+06  12123.548645  ...                          0
26  220648.585938  3.996528e+06   8486.483887  ...                          0
27  412806.816406  4.254759e+06   9092.661377  ...                          0
28  281266.337891  3.764968e+06  12123.548462  ...                          0
29  267930.419922  3.447331e+06  13335.903320  ...                          0
30  280053.980469  3.494613e+06  13335.903320  ...                          0

    cancer_mortality    n_alive       cdr       cbr  hpv_prevalence  \
0           0.000000  121197904  0.018106  0.042413        0.061782
1           0.000000  124478528  0.012126  0.041441        0.064703
2           0.000000  127821600  0.012615  0.041353        0.070638
3           0.000000  131375616  0.012509  0.041158        0.075019
4           0.000000  134938112  0.011999  0.041059        0.079922
5           0.000000  138672176  0.011650  0.040915        0.083203
6           0.000000  142415920  0.012437  0.040691        0.086385
7           0.000000  146311824  0.012019  0.040478        0.089433
8           0.000000  150286528  0.013012  0.040214        0.092575
9           0.000000  154511584  0.010769  0.040016        0.094350
10          0.000000  158756048  0.011810  0.039748        0.095963
11          0.000000  163216896  0.010195  0.039479        0.097667
12          0.728624  167750496  0.011556  0.039063        0.098352
13          0.000000  172441712  0.010591  0.038738        0.099816
14          0.000000  177068032  0.011527  0.038308        0.100480
15          0.000000  181708352  0.007119  0.038064        0.092049
16          0.000000  186360160  0.010936  0.037471        0.088128
17          0.638696  191102880  0.011127  0.036985        0.084585
18          1.870686  196006864  0.009831  0.036555        0.082062
19          3.041085  200914464  0.010880  0.036115        0.079061
20          4.155240  205913616  0.010342  0.036150        0.076368
21          0.578041  210912160  0.010237  0.036185        0.074120
22          3.947354  216025856  0.009681  0.036170        0.070958
23          4.413452  221232912  0.010313  0.036168        0.067623
24          2.153513  226554544  0.010127  0.036175        0.065206
25          5.780772  231927104  0.009822  0.036199        0.063797
26          3.080399  237457856  0.009320  0.036198        0.062002
27          3.508192  242963792  0.009234  0.036226        0.060260
28          7.828785  248571520  0.009128  0.036263        0.058623
29          6.694209  254137440  0.009570  0.036256        0.056160
30          4.680027  259864608  0.009069  0.036250        0.054948

    precin_prevalence  cin1_prevalence  cin2_prevalence  cin3_prevalence
0            0.074211         0.000000         0.000000         0.000000
1            0.078607         0.000000         0.000000         0.000000
2            0.081980         0.006277         0.000000         0.000000
3            0.081126         0.017538         0.000000         0.000000
4            0.085411         0.021119         0.000000         0.000000
5            0.086708         0.028698         0.000000         0.000000
6            0.088285         0.033803         0.000000         0.000000
7            0.089896         0.037688         0.000000         0.000000
8            0.092979         0.040875         0.000000         0.000000
9            0.093202         0.043422         0.000000         0.000000
10           0.095516         0.044561         0.000581         0.000000
11           0.096209         0.045688         0.002204         0.000000
12           0.097767         0.046702         0.003937         0.000000
13           0.098470         0.046802         0.005704         0.000024
14           0.099520         0.046273         0.007060         0.000172
15           0.093459         0.047341         0.007580         0.000871
16           0.086242         0.047449         0.007673         0.001740
17           0.081410         0.046821         0.008001         0.002757
18           0.078010         0.045711         0.007846         0.003901
19           0.075137         0.043161         0.008235         0.004702
20           0.072109         0.041833         0.008487         0.005465
21           0.070423         0.039605         0.008661         0.006058
22           0.067175         0.037460         0.008855         0.006594
23           0.065965         0.035216         0.009029         0.006793
24           0.063085         0.033101         0.008842         0.007159
25           0.062341         0.031181         0.009092         0.007212
26           0.060730         0.030058         0.008707         0.007174
27           0.059218         0.028298         0.008010         0.007499
28           0.058067         0.027658         0.007592         0.007568
29           0.056223         0.026944         0.007102         0.007751
30           0.055089         0.026445         0.006530         0.007802

[31 rows x 78 columns]