T2 - Plotting, printing, and saving

This tutorial provides a brief overview of options for plotting results, printing objects, and saving results.

Click here to open an interactive version of this notebook.

Global plotting configuration

HPVsim allows the user to set various options that apply to all plots. You can change the font size, default DPI, whether plots should be shown by default, etc. (for the full list, see hpv.options.help()). For example, we might want higher resolution, to turn off automatic figure display, close figures after they’re rendered, and to turn off the messages that print when a simulation is running. We can do this using built-in defaults for Jupyter notebooks (and then run a sim) with:

[1]:
import hpvsim as hpv

hpv.options(jupyter=True, verbose=0) # Standard options for Jupyter notebook

sim = hpv.Sim()
sim.run()
HPVsim 0.4.0 (2022-11-16) — © 2022 by IDM
No genotypes provided, will assume only simulating HPV16 by default
[1]:
Sim(<no label>; 2015.0 to 2030.0; pop: 20000 default; epi: 1034⚙, 16♋︎)

Printing objects

There are three levels of detail available for most objects (sims, multisims, scenarios, and people). The shortest is brief():

[2]:
sim.brief()
Sim(<no label>; 2015.0 to 2030.0; pop: 20000 default; epi: 1034⚙, 16♋︎)

You can get more detail with summarize():

[3]:
sim.summarize()
Simulation summary:
        1,034 total infections
          240 total cin1s
          267 total cin2s
          130 total cin3s
          637 total cins
           16 total cancers
            0 total cancer detections
            0 total cancer deaths
            0 total detected cancer deaths
          411 total reinfections
          131 total reactivations
            0 total hiv infections
         5.05 total hpv incidence (/100)
        1,736 total cin1 incidence (/100,000)
        1,931 total cin2 incidence (/100,000)
          940 total cin3 incidence (/100,000)
        4,607 total cin incidence (/100,000)
          116 total cancer incidence (/100,000)
         9.13 total hpv prevalence (/100)

Finally, to show the full object, including all methods and attributes, use disp():

[4]:
sim.disp()
<hpvsim.sim.Sim at 0x7f7a543e98e0>
————————————————————————————————————————————————————————————
Methods:
  _brief()            get_intervention()  reset_layer_pars()
  _disp()             get_interventio...  result_keys()
  _get_ia()           get_t()             run()
  brief()             init_analyzers()    save()
  compute_fit()       init_genotypes()    set_metadata()
  compute_results()   init_immunity()     set_seed()
  compute_states()    init_interventi...  shrink()
  compute_summary()   init_people()       step()
  copy()              init_results()      summarize()
  disp()              init_states()       to_df()
  export_pars()       initialize()        to_excel()
  export_results()    layer_keys()        to_json()
  finalize()          load()              update_pars()
  finalize_analyz...  load_data()         validate_init_c...
  get_analyzer()      load_hiv_data()     validate_layer_...
  get_analyzers()     plot()              validate_pars()
————————————————————————————————————————————————————————————
Properties:
  n
————————————————————————————————————————————————————————————
 _default_ver: None
   _orig_pars: {'n_agents': 20000, 'total_pop': None, 'pop_scale':
               1.0, 'use_multi [...]
 art_datafile: None
     complete: True
      created: None
         data: None
     datafile: None
     hiv_data: #0. 'infection_rates': None
               #1. 'art_adherence':   None
 hiv_datafile: None
  initialized: True
        label: None
         npts: 80
         pars: {'n_agents': 20000, 'total_pop': None, 'pop_scale':
               1.0, 'use_multi [...]
       people: People(n=30682; layers: m, c, o)
      popdict: None
      popfile: None
     res_npts: 16
     res_tvec: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11,
               12, 13, 14,  [...]
  res_yearvec: array([2015., 2016., 2017., 2018., 2019., 2020., 2021.,
               2022., 2023 [...]
      resfreq: 5
      results: #0. 'total_infections':
               <hpvsim.base.Result at 0x7f7a26942d30>
               ———— [...]
results_ready: True
      summary: #0. 'total_infections':             1034.0
               #1. 'total_cin1s':       [...]
            t: 79
         tvec: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11,
               12, 13, 14,  [...]
        years: array([2015., 2016., 2017., 2018., 2019., 2020., 2021.,
               2022., 2023 [...]
      yearvec: array([2015. , 2015.2, 2015.4, 2015.6, 2015.8, 2016. ,
               2016.2, 2016 [...]
————————————————————————————————————————————————————————————

Plotting options

While a sim can be plotted using default settings simply by sim.plot(), this is just a small fraction of what’s available. First, note that results can be plotted directly using e.g. Matplotlib. You can see what quantities are available for plotting with sim.results.keys() (remember, it’s just a dict). A simple example of plotting using Matplotlib is:

[5]:
import pylab as plt # Shortcut for import matplotlib.pyplot as plt
plt.plot(sim.results['year'], sim.results['total_infections']);
../_images/tutorials_tut_plotting_11_0.svg

However, as you can see, this isn’t ideal since the default formatting is…not great. (Also, note that each result is a Result object, not a simple Numpy array; like a pandas dataframe, you can get the array of values directly via e.g. sim.results.total_infections.values.)

An alternative, you can also select one or more quantities to plot with the first (to_plot) argument, e.g.

[6]:
sim.plot(to_plot=['total_infections', 'total_hpv_incidence']);
../_images/tutorials_tut_plotting_13_0.svg

While we can save this figure using Matplotlib’s built-in savefig(), if we use HPVsim’s hpv.savefig() we get a couple of advantages:

[7]:
hpv.savefig('my-fig.png')
[7]:
'my-fig.png'
<Figure size 614.4x460.8 with 0 Axes>

First, it saves the figure at higher resolution by default (which you can adjust with the dpi argument). But second, it stores information about the code that was used to generate the figure as metadata, which can be loaded later. Made an awesome plot but can’t remember even what script you ran to generate it, much less what version of the code? You’ll never have to worry about that again.

[8]:
hpv.get_png_metadata('my-fig.png')
HPVsim version: 0.4.0
HPVsim branch: Branch N/A
HPVsim hash: Hash N/A
HPVsim date: Date N/A
HPVsim caller branch: Branch N/A
HPVsim caller hash: Hash N/A
HPVsim caller date: Date N/A
HPVsim caller filename: /home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/envs/latest/lib/python3.9/site-packages/hpvsim/misc.py
HPVsim current time: 2022-Nov-16 06:41:38
HPVsim calling file: /tmp/ipykernel_1079/1797766398.py

Customizing plots

We saw above how to set default plot configuration options for Jupyter. HPVsim provides a lot of flexibility in customizing the appearance of plots as well. There are three different levels at which you can set plotting options: global, just for HPVsim, or just for the current plot. To give an example with changing the figure DPI: - Change the setting globally (for both HPVsim and Matplotlib): sc.options(dpi=150) or pl.rc('figure', dpi=150) (where sc is import sciris as sc) - Change for HPVsim plots, but not for Matplotlib plots: hpv.options(dpi=150) - Change for the current HPVsim plot, but not other HPVsim plots: sim.plot(dpi=150)

The easiest way to change the style of HPVsim plots is with the style argument. For example, to plot using a built-in Matplotlib style would simply be:

[9]:
sim.plot(style='ggplot');
../_images/tutorials_tut_plotting_19_0.svg

In addition to the default style ('hpvsim'), there is also a “simple” style. You can combine built-in styles with additional overrides, including any valid Matplotlib commands:

[10]:
sim.plot(style='simple', legend_args={'frameon':True}, style_args={'ytick.direction':'in'});
../_images/tutorials_tut_plotting_21_0.svg

Although most style handling is done automatically, you can also use it yourself in a with block, e.g.:

[11]:
import numpy as np
with hpv.options.with_style(fontsize=6):
    sim.plot() # This will have 6 point font
    plt.figure(); plt.plot(np.random.rand(20), 'o') # So will this
../_images/tutorials_tut_plotting_23_0.svg
../_images/tutorials_tut_plotting_23_1.svg

Saving options

Saving sims is also pretty simple. The simplest way to save is simply

[12]:
sim.save('my-sim.sim')
[12]:
'/home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/checkouts/latest/docs/tutorials/my-sim.sim'

Technically, this saves as a gzipped pickle file (via sc.saveobj() using the Sciris library). By default this does not save the people in the sim since they are very large (and since, if the random seed is saved, they can usually be regenerated). If you want to save the people as well, you can use the keep_people argument. For example, here’s what it would look like to create a sim, run it halfway, save it, load it, change the overall transmissibility (beta), and finish running it:

[13]:
sim_orig = hpv.Sim(start=2000, end=2030, label='Load & save example')
sim_orig.run(until='2015')
sim_orig.save('my-half-finished-sim.sim') # Note: HPVsim always saves the people if the sim isn't finished running yet

sim = hpv.load('my-half-finished-sim.sim')
sim['beta'] *= 0.3
sim.run()
sim.plot(['total_infections', 'total_hpv_incidence', 'total_cin_incidence']);
No genotypes provided, will assume only simulating HPV16 by default
../_images/tutorials_tut_plotting_27_1.svg

Aside from saving the entire simulation, there are other export options available. You can export the results and parameters to a JSON file (using sim.to_json()), but probably the most useful is to export the results to an Excel workbook, where they can easily be stored and processed with e.g. Pandas:

[14]:
import pandas as pd

sim.to_excel('my-sim.xlsx')
df = pd.read_excel('my-sim.xlsx')
print(df)
WARNING: skipping infections from export since not 1D array
WARNING: skipping cin1s from export since not 1D array
WARNING: skipping cin2s from export since not 1D array
WARNING: skipping cin3s from export since not 1D array
WARNING: skipping cins from export since not 1D array
WARNING: skipping cancers from export since not 1D array
WARNING: skipping reinfections from export since not 1D array
WARNING: skipping reactivations from export since not 1D array
WARNING: skipping n_susceptible from export since not 1D array
WARNING: skipping n_infectious from export since not 1D array
WARNING: skipping n_inactive from export since not 1D array
WARNING: skipping n_no_dysp from export since not 1D array
WARNING: skipping n_cin1 from export since not 1D array
WARNING: skipping n_cin2 from export since not 1D array
WARNING: skipping n_cin3 from export since not 1D array
WARNING: skipping n_cancerous from export since not 1D array
WARNING: skipping n_infected from export since not 1D array
WARNING: skipping n_cin from export since not 1D array
WARNING: skipping n_precin from export since not 1D array
WARNING: skipping n_latent from export since not 1D array
WARNING: skipping hpv_incidence from export since not 1D array
WARNING: skipping cin1_incidence from export since not 1D array
WARNING: skipping cin2_incidence from export since not 1D array
WARNING: skipping cin3_incidence from export since not 1D array
WARNING: skipping cin_incidence from export since not 1D array
WARNING: skipping cancer_incidence from export since not 1D array
WARNING: skipping total_infections_by_sex from export since not 1D array
WARNING: skipping other_deaths_by_sex from export since not 1D array
WARNING: skipping cancers_by_age from export since not 1D array
WARNING: skipping no_dysp_types from export since not 1D array
WARNING: skipping cin1_types from export since not 1D array
WARNING: skipping cin2_types from export since not 1D array
WARNING: skipping cin3_types from export since not 1D array
WARNING: skipping cancer_types from export since not 1D array
WARNING: skipping new_vaccinated from export since not 1D array
WARNING: skipping cum_vaccinated from export since not 1D array
WARNING: skipping n_alive_by_sex from export since not 1D array
WARNING: skipping hpv_prevalence from export since not 1D array
Object saved to /home/docs/checkouts/readthedocs.org/user_builds/institute-for-disease-modeling-hpvsim/checkouts/latest/docs/tutorials/my-sim.xlsx.
    year   t  total_infections  total_cin1s  total_cin2s  total_cin3s  \
0   2000   0               542           80            0            0
1   2001   1               613          136           45            0
2   2002   2               777          184          113            0
3   2003   3               992          207          160           43
4   2004   4              1135          246          203           64
5   2005   5              1211          301          225          110
6   2006   6              1202          286          266          118
7   2007   7              1188          286          304          143
8   2008   8              1197          284          291          165
9   2009   9              1180          290          275          157
10  2010  10              1125          290          286          172
11  2011  11              1043          249          287          166
12  2012  12              1029          243          264          168
13  2013  13              1042          285          237          146
14  2014  14              1134          244          267          165
15  2015  15               666          257          261          131
16  2016  16               653          180          249          153
17  2017  17               645          159          223          153
18  2018  18               606          146          157          156
19  2019  19               555          158          138          116
20  2020  20               632          143          146           87
21  2021  21               599          154          158           80
22  2022  22               660          169          153           84
23  2023  23               670          163          159           89
24  2024  24               696          166          156           74
25  2025  25               700          178          158          109
26  2026  26               659          192          183           83
27  2027  27               739          182          191           90
28  2028  28               742          186          173          109
29  2029  29               702          181          181          113
30  2030  30               785          169          176           88

    total_cins  total_cancers  total_detected_cancers  total_cancer_deaths  \
0           80              0                       0                    0
1          181              0                       0                    0
2          297              0                       0                    0
3          410              0                       0                    0
4          513              0                       0                    0
5          636              0                       0                    0
6          670              0                       0                    0
7          733              0                       0                    0
8          740              0                       0                    0
9          722              1                       0                    0
10         748              7                       0                    0
11         702              6                       0                    0
12         675             10                       0                    0
13         668             11                       0                    0
14         676             18                       0                    0
15         649             18                       0                    0
16         582             22                       0                    0
17         535             19                       0                    0
18         459             17                       0                    0
19         412             27                       0                    1
20         376             14                       0                    7
21         392             13                       0                    2
22         406             20                       0                    5
23         411             18                       0                   11
24         396             22                       0                   13
25         445             15                       0                    9
26         458             21                       0                    9
27         463             12                       0                   14
28         468              3                       0                   12
29         475              9                       0                   19
30         433             16                       0                   18

    ...  cum_cancer_treatments  cum_cancer_treated  detected_cancer_incidence  \
0   ...                      0                   0                          0
1   ...                      0                   0                          0
2   ...                      0                   0                          0
3   ...                      0                   0                          0
4   ...                      0                   0                          0
5   ...                      0                   0                          0
6   ...                      0                   0                          0
7   ...                      0                   0                          0
8   ...                      0                   0                          0
9   ...                      0                   0                          0
10  ...                      0                   0                          0
11  ...                      0                   0                          0
12  ...                      0                   0                          0
13  ...                      0                   0                          0
14  ...                      0                   0                          0
15  ...                      0                   0                          0
16  ...                      0                   0                          0
17  ...                      0                   0                          0
18  ...                      0                   0                          0
19  ...                      0                   0                          0
20  ...                      0                   0                          0
21  ...                      0                   0                          0
22  ...                      0                   0                          0
23  ...                      0                   0                          0
24  ...                      0                   0                          0
25  ...                      0                   0                          0
26  ...                      0                   0                          0
27  ...                      0                   0                          0
28  ...                      0                   0                          0
29  ...                      0                   0                          0
30  ...                      0                   0                          0

    cancer_mortality  n_alive       cdr       cbr  hiv_incidence  \
0           0.000000    20621  0.008390  0.038504              0
1           0.000000    21277  0.007426  0.038257              0
2           0.000000    21968  0.006509  0.037964              0
3           0.000000    22642  0.007906  0.037673              0
4           0.000000    23339  0.007455  0.037320              0
5           0.000000    24070  0.006564  0.036934              0
6           0.000000    24787  0.007544  0.036471              0
7           0.000000    25498  0.008079  0.035964              0
8           0.000000    26247  0.006820  0.035356              0
9           0.000000    26977  0.007636  0.034696              0
10          0.000000    27729  0.006816  0.033936              0
11          0.000000    28433  0.008335  0.033095              0
12          0.000000    29157  0.007442  0.032274              0
13          0.000000    29890  0.006892  0.031415              0
14          0.000000    30602  0.007287  0.030554              0
15          0.000000    31537  0.004186  0.029648              0
16          0.000000    32257  0.006727  0.029048              0
17          0.000000    32970  0.006855  0.028480              0
18          0.000000    33644  0.007906  0.027940              0
19          5.838393    34308  0.008132  0.027515              0
20         40.002286    34995  0.007687  0.027518              0
21         11.165699    35727  0.006970  0.027514              0
22         27.340332    36474  0.006909  0.027526              0
23         58.848705    37209  0.007498  0.027547              0
24         68.212824    37951  0.007641  0.027536              0
25         46.255846    38735  0.007074  0.027546              0
26         45.257970    39525  0.007312  0.027527              0
27         69.081220    40325  0.007340  0.027526              0
28         58.060770    41131  0.007658  0.027546              0
29         90.068737    41965  0.007196  0.027523              0
30         83.561580    42779  0.008088  0.027537              0

    hiv_prevalence  total_hpv_prevalence
0                0              0.031085
1                0              0.044696
2                0              0.059587
3                0              0.076274
4                0              0.089978
5                0              0.098297
6                0              0.102271
7                0              0.102165
8                0              0.102945
9                0              0.100567
10               0              0.097804
11               0              0.092393
12               0              0.088418
13               0              0.086149
14               0              0.086105
15               0              0.071630
16               0              0.062932
17               0              0.057901
18               0              0.052907
19               0              0.049726
20               0              0.049579
21               0              0.047723
22               0              0.047979
23               0              0.047327
24               0              0.047930
25               0              0.047631
26               0              0.046022
27               0              0.047216
28               0              0.047069
29               0              0.045300
30               0              0.045817

[31 rows x 71 columns]