idmtools.analysis.download_analyzer module¶

idmtools Download analyzer.

Download Analyzer.

class idmtools.analysis.download_analyzer.DownloadAnalyzer(filenames=None, output_path='output', **kwargs)¶

Bases: IAnalyzer

A simple base class that will download the files specified in filenames without further treatment.

Can be used by creating a child class:

class InsetDownloader(DownloadAnalyzer):
    filenames = ['output/InsetChart.json']

Or by directly calling it:

analyzer = DownloadAnalyzer(filenames=['output/InsetChart.json'])

Examples

# Example DownloadAnalyzer for EMOD Experiment
# In this example, we will demonstrate how to create an DownloadAnalyzer to download simulation output files locally

# First, import some necessary system and idmtools packages.
from idmtools.analysis.analyze_manager import AnalyzeManager
from idmtools.analysis.download_analyzer import DownloadAnalyzer
from idmtools.core import ItemType
from idmtools.core.platform_factory import Platform

if __name__ == '__main__':

    # Set the platform where you want to run your analysis
    # In this case we are running in BELEGOST, but this can be changed to run 'Local'
    with Platform('BELEGOST') as platform:

        # Arg option for analyzer init are uid, working_dir, data in the method map (aka select_simulation_data),
        # and filenames
        # In this case, we want to provide a filename to analyze
        filenames = ['StdOut.txt']
        # Initialize the analyser class with the path of the output files to download
        analyzers = [DownloadAnalyzer(filenames=filenames, output_path='download')]

        # Set the experiment you want to analyze
        experiment_id = '40c1b14d-0a04-eb11-a2c7-c4346bcb1553'  # comps exp id

        # Specify the id Type, in this case an Experiment
        manager = AnalyzeManager(ids=[(experiment_id, ItemType.EXPERIMENT)],
                                 analyzers=analyzers)
        manager.analyze()

reduce(all_data: Dict[Union[IWorkflowItem, Simulation], Any])¶

Combine the map() data for a set of items into an aggregate result. In this case, for downloading, we just ignore it because there is no reduction.

Parameters:: all_data – Dictionary in form item->map result where item is Simulations or WorkItems
Returns:: None

__init__(filenames=None, output_path='output', **kwargs)¶: Constructor of the analyzer.

initialize()¶

Initialize our sim. In this case, we create our output directory.

Returns:: None

get_item_folder(item: Union[IWorkflowItem, Simulation])¶

Concatenate the specified top-level output folder with the item ID.

Parameters:: item – A simulation output parsing thread.
Returns:: The name of the folder to download this simulation’s output to.

map(data: Dict[str, Any], item: Union[IWorkflowItem, Simulation])¶

Provide a map of filenames->data for each item. We then download each of these files to our output folder.

Parameters:

data – Map filenames->data
item – Item we are mapping.

Returns:

None