1.0.0#

Analyzers#

  • #0034 - Create the Plotting step

  • #0057 - Output files retrieval

  • #0196 - Filtering

  • #0197 - Select_simulation_data

  • #0198 - Finalize

  • #0279 - Port dtk-tools analyze system to idmtools

  • #0283 - Fix up all platform-based test due to analyzer/platform refactor/genericization

  • #0337 - Change AnalyzeManager to support passing ids (Experiment, Simulation, Suite)

  • #0338 - Two AnalyzeManager files - one incorrect and needs to be removed

  • #0340 - Cleanup DownloadAnalyzer

  • #0344 - AnalyzeManager configuration should be option parameter

  • #0589 - Rename suggestion: example_analysis_multiple_cases => example_analysis_MultipleCases

  • #0592 - analyzers error on platform.get_files for COMPS: argument of type ‘NoneType’ is not iterable

  • #0594 - analyzer error multiprocessing pool StopIteration error in finalize_results

  • #0614 - Convenience function to exclude items in analyze manager

  • #0619 - Ability to get exp sim object ids in analyzers

Bugs#

  • #0124 - Can not run teststest_python_simulation.py from console

  • #0125 - relative_path for AssetCollection does not work

  • #0129 - new python model root node changed from “config” to “parameters”

  • #0142 - experiment.batch_simulations seems not to be batching

  • #0143 - COMPSPlatform’s refresh_experiment_status() get called too much from ExperimentManager’s wait_till_done() mathod

  • #0150 - missing pandas package

  • #0184 - Missing ‘data’ dir for test_experiment_manager test. (TestPlatform)

  • #0223 - UnicodeDecodeError for testcases in test_dtk.py when run with LocalPlatform

  • #0236 - LocalRunner: ExperimentsClient get_all method should have parameter ‘tags’ not ‘tag’

  • #0265 - load_files for DTKExperiment create nested ‘parameters’ in config.json

  • #0266 - load_files for demographics.json does not work

  • #0272 - diskcache objects cause cleanup failure if used in failing processes

  • #0294 - Docker containers failed to start if they are created but stopped

  • #0299 - Sometime in Windows command line, local docker runner stuck and no way to stop from command line

  • #0302 - Local Platform delete is broken

  • #0318 - Postgres Connection error on Local Platform

  • #0320 - COMPSPlatform Asset handling - currently DuplicatedAssetError content is not same

  • #0323 - idmtools is not retro-compatible with pre-idmtools experiments

  • #0332 - with large number of simulations, local platform either timeout on dramatiq or stuck on persistamceService save method

  • #0339 - Analyzer tests fails on AnalyzeManager analyze len(self.potential_items) == 0

  • #0341 - AnalyzeManager Runtime error on worker_pool

  • #0346 - UnknownItemException for analyzers on COMPSPlatform PythonExperiments

  • #0350 - RunTask in local platform should catch exception

  • #0351 - AnalyzeManager finalize_results Process cannot access the cache.db because it is being used by another process

  • #0352 - Current structure of code leads to circular dependencies or classes as modules

  • #0367 - Analyzer does not work with reduce method with no hashable object

  • #0375 - AnalyzerManager does not work for case to add experiment to analyzermanager

  • #0376 - AnalyzerManager does not work for simulation

  • #0378 - experiment/simulation display and print are messed up in latest dev

  • #0386 - Local platform cannot create more than 20 simulations in a given experiment

  • #0398 - Ensure that redis and postgres ports work as expected

  • #0399 - PopulaionAnalyzer does not return all items in reduce mathod in centos platform

  • #0424 - ExperimentBuilder’s add_sweep_definition is not flexible enough to take more parameters

  • #0427 - Access to the experiment object in analyzers

  • #0453 - cli: “idmtools local down –delete-data” not really delete any .local_data in user default dir

  • #0458 - There is no way to add custom tags to simulations

  • #0465 - BuilderExperiment for sweep “string” is wrong

  • #0545 - pymake docker-local always fail in centos

  • #0553 - BLOCKING: idmtools_model_r does not get built with make setup-dev

  • #0560 - docker-compose build does not work for r-model example

  • #0562 - workflow_item_operations get workitem querycriteria fails

  • #0564 - typing is missing in asset_collection.py which almost break every tests

  • #0565 - missing ‘copy’ in local_platform.py

  • #0566 - test_tasks.py fail for case test_command_is_required

  • #0567 - ‘platform_supports’ is missing for test_comps_plugin.py in idmtools_platform_comps/tests

  • #0570 - webui for localhost:5000 got 403 error

  • #0572 - python 3.7.3 less version will fail for task type changing

  • #0585 - print(platform) throws exception for Python 3.6

  • #0588 - Running the dev installation in a virtualenv “installs” it globally

  • #0598 - CSVAnalyzer pass wrong value to parse in super().__init__ call

  • #0602 - Analyzer doesn’t work for my Python SEIR model

  • #0605 - When running multiple analyzers together, ‘data’ in one analyzer should not contains data from other analyzer

  • #0606 - can not import cached_property

  • #0608 - Cannot add custom tag to AssetCollection in idmtools

  • #0613 - idmtools webui does not working anymore

  • #0616 - AssetCollection pre_creation failed if no tag

  • #0617 - AssetCollection’s find_index_of_asset is wrong

  • #0618 - analyzer-manager should fail if map status return False

  • #0641 - Remove unused code in the python_requirements_ac

  • #0644 - Platform cannot run workitem directly

  • #0646 - platform.get_items(ac) not return tags

  • #0667 - analyzer_manager could stuck on _run_and_wait_for_reducing

CLI#

  • #0009 - Boilerplate command

  • #0118 - Add the printing of children in the EntityContainer

  • #0187 - Move the CLI package to idmtools/cli

  • #0190 - Add a platform attribute to the CLI commands

  • #0191 - Create a PlatformFactory

  • #0241 - CLI should be distinct package and implement as plugins

  • #0251 - Setup for the CLI package should provide a entrypoint for easy use of commands

  • #0252 - Add –debug to cli main level

Configuration#

  • #0248 - Logging needs to support user configuration through the idmtools.ini

  • #0392 - Improve IdmConfigParser: make decorator for ensure_ini() method…

  • #0597 - Platform should not be case sensitive.

Core#

  • #0032 - Create NextPointAlgorithm Step

  • #0042 - Stabilize the IStep object

  • #0043 - Create the generic Workflow object

  • #0044 - Implement validation for the Steps of a workflow based on Marshmallow

  • #0058 - Filtering system for simulations

  • #0081 - Allows the sweeps to be created in arms

  • #0091 - Refactor the Experiment/Simulation objects to not persist the simulations

  • #0141 - Standard Logging throughout tools

  • #0169 - Handle 3.6 requirements automatically

  • #0172 - Decide what state to store for tasks

  • #0173 - workflows: Decide on state storage scheme

  • #0174 - workflows: Reimplement state storage

  • #0175 - workflows: Create unit tests of core classes and behaviors

  • #0176 - workflows: reorganize files into appropriate repo/directory

  • #0180 - switch prettytable for tabulate

  • #0200 - Platforms should be plugins

  • #0238 - Simulations of Experiment should be made pickle ignored

  • #0244 - Inputs values needs to be validated when creating a Platform

  • #0257 - CsvExperimentBuilder does not handle csv field with empty space

  • #0268 - demographics filenames should be loaded to asset collection

  • #0274 - Unify id attribute naming scheme

  • #0281 - Improve Platform to display selected Block info when creating a platform

  • #0297 - Fix issues with platform factory

  • #0308 - idmtools: Module names should be consistent

  • #0315 - Basic support of suite in the tools

  • #0357 - ExperimentPersistService.save are not consistent

  • #0359 - SimulationPersistService is not used in Idmtools

  • #0361 - assets in Experiment should be made “pickle-ignore”

  • #0362 - base_simulation in Experiment should be made “pickle-ignore”

  • #0368 - PersistService should support clear() method

  • #0369 - The method create_simulations of Experiment should consider pre-defined max_workers and batch_size in idmtools.ini

  • #0370 - Add unit test for deepcopy on simulations

  • #0371 - Wrong type for platform_id in IEntity definition

  • #0391 - Improve Asset and AssetCollection classes by using @dataclass (field) for clear comparison

  • #0394 - Remove the ExperimentPersistService

  • #0438 - Support pulling Eradication from URLs and bamboo

  • #0518 - Add a task class.

  • #0520 - Rename current experiment builders to sweep builders

  • #0526 - Create New Generic Experiment Class

  • #0527 - Create new Generic Simulation Class

  • #0528 - Remove old Experiments/Simulations

  • #0529 - Create New Task API

  • #0530 - Rename current model api to simulation/experiment API.

  • #0538 - Refactor platform interface into subinterfaces

  • #0681 - idmtools should have way to query comps with filter

Developer/Test#

  • #0631 - Ensure setup.py is consistent throughout

Documentation#

  • #0100 - Installation steps documented for users

  • #0312 - idmtools: there is a typo in README

  • #0360 - The tools should refer to “EMOD” not “DTK”

  • #0474 - Stand alone builder

  • #0486 - Overview of the analysis in idmtools

  • #0510 - Local platform options

  • #0512 - SSMT platform options

  • #0578 - Add installation for users

  • #0593 - Simple Python SEIR model demo example

  • #0632 - Update idmtools_core setup.py to remove model emod from idm install

Feature Request#

  • #0061 - Built-in DownloadAnalyzer

  • #0064 - Support of CSV files

  • #0070 - [Local Runner] Output files serving

  • #0233 - Add local runner timeout

  • #0437 - Prompt users for docker credentials when not available

  • #0603 - Implement install custom requirement libs to asset collection with WorkItem

Models#

  • #0021 - Python model

  • #0024 - R Model support

  • #0053 - Support of demographics files

  • #0212 - Models should be plugins

  • #0287 - Add info about support models/docker support to platform

  • #0288 - Create DockerExperiment and subclasses

  • #0519 - Move experiment building to ExperimentBuilder

  • #0521 - Create Generic Dictionary Config Task

  • #0522 - Create PythonTask

  • #0523 - Create PythonDictionaryTask

  • #0524 - Create RTask

  • #0525 - Create EModTask

  • #0535 - Create DockerTask

Platforms#

  • #0025 - LOCAL Platform

  • #0027 - SSMT Platform

  • #0094 - Batch and parallelize simulation creation in the COMPSPlatform

  • #0122 - Ability to create an AssetCollection based on a COMPS asset collection id

  • #0130 - User configuration and data storage location

  • #0186 - The local_runner client should move to the idmtools package

  • #0194 - COMPS Files retrieval system

  • #0195 - LOCAL Files retrieval system

  • #0221 - Local runner for experiment/simulations have different file hierarchy than COMPS

  • #0254 - Local Platform Asset should be implemented via API or Docker socket

  • #0264 - idmtools_local_runner’s tasks/run.py should have better handle for unhandled exception

  • #0276 - Docker services should be started for end-users without needing to use docker-compose

  • #0280 - Generalize sim/exp/suite format of ISimulation, IExperiment, IPlatform

  • #0286 - Add special GPU queue to Local Platform

  • #0305 - Create a website for local platform

  • #0306 - AssetCollection’s assets_from_directory logic wrong if set flatten and relative path at same time

  • #0313 - idmtools: MAX_SUBDIRECTORY_LENGTH = 35 should be made Global in COMPSPlatform definition

  • #0314 - Fix local platform to work with latest analyze/platform updates

  • #0316 - Integrate website with Local Runner Container

  • #0321 - COMPSPlatform _retrieve_experiment errors on experiments with and without suites

  • #0329 - Experiment level status

  • #0330 - Paging on simulation/experiment APIs for better UI experience

  • #0333 - ensure pyComps allows compatible releases

  • #0364 - Local platform should use production artfactory for docker images

  • #0381 - Support Work Items in COMPS Platform

  • #0387 - Local platform webUI only show simulations up to 20

  • #0393 - local platform tests keep getting EOFError while logger is in DEBUG and console is on

  • #0405 - Support analysis of data from Work Items in Analyze Manager

  • #0407 - Support Service Side Analysis through SSMT

  • #0447 - Set limitation for docker container’s access to memory

  • #0532 - Make updates to ExperimentManager/Platform to support tasks

  • #0540 - Create initial SSMT Plaform from COMPS Platform

  • #0596 - COMPSPlatform.get_files(item,..) not working for Experiment or Suite

  • #0635 - Update SSMT base image

  • #0639 - Add a way for the python_requirements_ac to use additional wheel file

  • #0676 - ssmt missing QueryCriteria support

  • #0677 - ssmt: refresh_status returns None

User Experience#

  • #0457 - Option to analyze failed simulations