idmtools.analysis.download_analyzer module

idmtools Download analyzer.

Download Analyzer.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

class idmtools.analysis.download_analyzer.DownloadAnalyzer(filenames=None, output_path='output', **kwargs)

Bases: idmtools.entities.ianalyzer.IAnalyzer

A simple base class that will download the files specified in filenames without further treatment.

Can be used by creating a child class:

class InsetDownloader(DownloadAnalyzer):
    filenames = ['output/InsetChart.json']

Or by directly calling it:

analyzer = DownloadAnalyzer(filenames=['output/InsetChart.json'])

Examples

# Example DownloadAnalyzer for EMOD Experiment
# In this example, we will demonstrate how to create an DownloadAnalyzer to download simulation output files locally

# First, import some necessary system and idmtools packages.
from idmtools.analysis.analyze_manager import AnalyzeManager
from idmtools.analysis.download_analyzer import DownloadAnalyzer
from idmtools.core import ItemType
from idmtools.core.platform_factory import Platform

if __name__ == '__main__':

    # Set the platform where you want to run your analysis
    # In this case we are running in BELEGOST, but this can be changed to run 'Local'
    with Platform('BELEGOST') as platform:

        # Arg option for analyzer init are uid, working_dir, data in the method map (aka select_simulation_data),
        # and filenames
        # In this case, we want to provide a filename to analyze
        filenames = ['StdOut.txt']
        # Initialize the analyser class with the path of the output files to download
        analyzers = [DownloadAnalyzer(filenames=filenames, output_path='download')]

        # Set the experiment you want to analyze
        experiment_id = '40c1b14d-0a04-eb11-a2c7-c4346bcb1553'  # comps exp id

        # Specify the id Type, in this case an Experiment
        manager = AnalyzeManager(ids=[(experiment_id, ItemType.EXPERIMENT)],
                                 analyzers=analyzers)
        manager.analyze()
reduce(all_data: Dict[Union[IWorkflowItem, Simulation], Any])

Combine the map() data for a set of items into an aggregate result. In this case, for downloading, we just ignore it because there is no reduction.

Parameters:

all_data – Dictionary in form item->map result where item is Simulations or WorkItems

Returns:

None

__init__(filenames=None, output_path='output', **kwargs)

Constructor of the analyzer.

initialize()

Initialize our sim. In this case, we create our output directory.

Returns:

None

get_item_folder(item: Union[IWorkflowItem, Simulation])

Concatenate the specified top-level output folder with the item ID.

Parameters:

item – A simulation output parsing thread.

Returns:

The name of the folder to download this simulation’s output to.

map(data: Dict[str, Any], item: Union[IWorkflowItem, Simulation])

Provide a map of filenames->data for each item. We then download each of these files to our output folder.

Parameters:
  • data – Map filenames->data

  • item – Item we are mapping.

Returns:

None