1.0.0#
Analyzers#
#0034 - Create the Plotting step
#0057 - Output files retrieval
#0196 - Filtering
#0197 - Select_simulation_data
#0198 - Finalize
#0279 - Port dtk-tools analyze system to idmtools
#0283 - Fix up all platform-based test due to analyzer/platform refactor/genericization
#0337 - Change AnalyzeManager to support passing ids (Experiment, Simulation, Suite)
#0338 - Two AnalyzeManager files - one incorrect and needs to be removed
#0340 - Cleanup DownloadAnalyzer
#0344 - AnalyzeManager configuration should be option parameter
#0589 - Rename suggestion: example_analysis_multiple_cases => example_analysis_MultipleCases
#0592 - analyzers error on platform.get_files for COMPS: argument of type ‘NoneType’ is not iterable
#0594 - analyzer error multiprocessing pool StopIteration error in finalize_results
#0614 - Convenience function to exclude items in analyze manager
#0619 - Ability to get exp sim object ids in analyzers
Bugs#
#0124 - Can not run teststest_python_simulation.py from console
#0125 - relative_path for AssetCollection does not work
#0129 - new python model root node changed from “config” to “parameters”
#0142 - experiment.batch_simulations seems not to be batching
#0143 - COMPSPlatform’s refresh_experiment_status() get called too much from ExperimentManager’s wait_till_done() mathod
#0150 - missing pandas package
#0184 - Missing ‘data’ dir for test_experiment_manager test. (TestPlatform)
#0223 - UnicodeDecodeError for testcases in test_dtk.py when run with LocalPlatform
#0236 - LocalRunner: ExperimentsClient get_all method should have parameter ‘tags’ not ‘tag’
#0265 - load_files for DTKExperiment create nested ‘parameters’ in config.json
#0266 - load_files for demographics.json does not work
#0272 - diskcache objects cause cleanup failure if used in failing processes
#0294 - Docker containers failed to start if they are created but stopped
#0299 - Sometime in Windows command line, local docker runner stuck and no way to stop from command line
#0302 - Local Platform delete is broken
#0318 - Postgres Connection error on Local Platform
#0320 - COMPSPlatform Asset handling - currently DuplicatedAssetError content is not same
#0323 - idmtools is not retro-compatible with pre-idmtools experiments
#0332 - with large number of simulations, local platform either timeout on dramatiq or stuck on persistamceService save method
#0339 - Analyzer tests fails on AnalyzeManager analyze len(self.potential_items) == 0
#0341 - AnalyzeManager Runtime error on worker_pool
#0346 - UnknownItemException for analyzers on COMPSPlatform PythonExperiments
#0350 - RunTask in local platform should catch exception
#0351 - AnalyzeManager finalize_results Process cannot access the cache.db because it is being used by another process
#0352 - Current structure of code leads to circular dependencies or classes as modules
#0367 - Analyzer does not work with reduce method with no hashable object
#0375 - AnalyzerManager does not work for case to add experiment to analyzermanager
#0376 - AnalyzerManager does not work for simulation
#0378 - experiment/simulation display and print are messed up in latest dev
#0386 - Local platform cannot create more than 20 simulations in a given experiment
#0398 - Ensure that redis and postgres ports work as expected
#0399 - PopulaionAnalyzer does not return all items in reduce mathod in centos platform
#0424 - ExperimentBuilder’s add_sweep_definition is not flexible enough to take more parameters
#0427 - Access to the experiment object in analyzers
#0453 - cli: “idmtools local down –delete-data” not really delete any .local_data in user default dir
#0458 - There is no way to add custom tags to simulations
#0465 - BuilderExperiment for sweep “string” is wrong
#0545 - pymake docker-local always fail in centos
#0553 - BLOCKING: idmtools_model_r does not get built with make setup-dev
#0560 - docker-compose build does not work for r-model example
#0562 - workflow_item_operations get workitem querycriteria fails
#0564 - typing is missing in asset_collection.py which almost break every tests
#0565 - missing ‘copy’ in local_platform.py
#0566 - test_tasks.py fail for case test_command_is_required
#0567 - ‘platform_supports’ is missing for test_comps_plugin.py in idmtools_platform_comps/tests
#0570 - webui for localhost:5000 got 403 error
#0572 - python 3.7.3 less version will fail for task type changing
#0585 - print(platform) throws exception for Python 3.6
#0588 - Running the dev installation in a virtualenv “installs” it globally
#0598 - CSVAnalyzer pass wrong value to parse in super().__init__ call
#0602 - Analyzer doesn’t work for my Python SEIR model
#0605 - When running multiple analyzers together, ‘data’ in one analyzer should not contains data from other analyzer
#0606 - can not import cached_property
#0608 - Cannot add custom tag to AssetCollection in idmtools
#0613 - idmtools webui does not working anymore
#0616 - AssetCollection pre_creation failed if no tag
#0617 - AssetCollection’s find_index_of_asset is wrong
#0618 - analyzer-manager should fail if map status return False
#0641 - Remove unused code in the python_requirements_ac
#0644 - Platform cannot run workitem directly
#0646 - platform.get_items(ac) not return tags
#0667 - analyzer_manager could stuck on _run_and_wait_for_reducing
CLI#
#0009 - Boilerplate command
#0118 - Add the printing of children in the EntityContainer
#0187 - Move the CLI package to idmtools/cli
#0190 - Add a platform attribute to the CLI commands
#0191 - Create a PlatformFactory
#0241 - CLI should be distinct package and implement as plugins
#0251 - Setup for the CLI package should provide a entrypoint for easy use of commands
#0252 - Add –debug to cli main level
Configuration#
Core#
#0032 - Create NextPointAlgorithm Step
#0042 - Stabilize the IStep object
#0043 - Create the generic Workflow object
#0044 - Implement validation for the Steps of a workflow based on Marshmallow
#0058 - Filtering system for simulations
#0081 - Allows the sweeps to be created in arms
#0091 - Refactor the Experiment/Simulation objects to not persist the simulations
#0141 - Standard Logging throughout tools
#0169 - Handle 3.6 requirements automatically
#0172 - Decide what state to store for tasks
#0173 - workflows: Decide on state storage scheme
#0174 - workflows: Reimplement state storage
#0175 - workflows: Create unit tests of core classes and behaviors
#0176 - workflows: reorganize files into appropriate repo/directory
#0180 - switch prettytable for tabulate
#0200 - Platforms should be plugins
#0238 - Simulations of Experiment should be made pickle ignored
#0244 - Inputs values needs to be validated when creating a Platform
#0257 - CsvExperimentBuilder does not handle csv field with empty space
#0268 - demographics filenames should be loaded to asset collection
#0274 - Unify id attribute naming scheme
#0281 - Improve Platform to display selected Block info when creating a platform
#0297 - Fix issues with platform factory
#0308 - idmtools: Module names should be consistent
#0315 - Basic support of suite in the tools
#0357 - ExperimentPersistService.save are not consistent
#0359 - SimulationPersistService is not used in Idmtools
#0361 - assets in Experiment should be made “pickle-ignore”
#0362 - base_simulation in Experiment should be made “pickle-ignore”
#0368 - PersistService should support clear() method
#0369 - The method create_simulations of Experiment should consider pre-defined max_workers and batch_size in idmtools.ini
#0370 - Add unit test for deepcopy on simulations
#0371 - Wrong type for platform_id in IEntity definition
#0391 - Improve Asset and AssetCollection classes by using @dataclass (field) for clear comparison
#0394 - Remove the ExperimentPersistService
#0438 - Support pulling Eradication from URLs and bamboo
#0518 - Add a task class.
#0520 - Rename current experiment builders to sweep builders
#0526 - Create New Generic Experiment Class
#0527 - Create new Generic Simulation Class
#0528 - Remove old Experiments/Simulations
#0529 - Create New Task API
#0530 - Rename current model api to simulation/experiment API.
#0538 - Refactor platform interface into subinterfaces
#0681 - idmtools should have way to query comps with filter
Developer/Test#
#0631 - Ensure setup.py is consistent throughout
Documentation#
#0100 - Installation steps documented for users
#0312 - idmtools: there is a typo in README
#0360 - The tools should refer to “EMOD” not “DTK”
#0474 - Stand alone builder
#0486 - Overview of the analysis in idmtools
#0510 - Local platform options
#0512 - SSMT platform options
#0578 - Add installation for users
#0593 - Simple Python SEIR model demo example
#0632 - Update idmtools_core setup.py to remove model emod from idm install
Feature Request#
Models#
#0021 - Python model
#0024 - R Model support
#0053 - Support of demographics files
#0212 - Models should be plugins
#0287 - Add info about support models/docker support to platform
#0288 - Create DockerExperiment and subclasses
#0519 - Move experiment building to ExperimentBuilder
#0521 - Create Generic Dictionary Config Task
#0522 - Create PythonTask
#0523 - Create PythonDictionaryTask
#0524 - Create RTask
#0525 - Create EModTask
#0535 - Create DockerTask
Platforms#
#0025 - LOCAL Platform
#0027 - SSMT Platform
#0094 - Batch and parallelize simulation creation in the COMPSPlatform
#0122 - Ability to create an AssetCollection based on a COMPS asset collection id
#0130 - User configuration and data storage location
#0186 - The local_runner client should move to the idmtools package
#0194 - COMPS Files retrieval system
#0195 - LOCAL Files retrieval system
#0221 - Local runner for experiment/simulations have different file hierarchy than COMPS
#0254 - Local Platform Asset should be implemented via API or Docker socket
#0264 - idmtools_local_runner’s tasks/run.py should have better handle for unhandled exception
#0276 - Docker services should be started for end-users without needing to use docker-compose
#0280 - Generalize sim/exp/suite format of ISimulation, IExperiment, IPlatform
#0286 - Add special GPU queue to Local Platform
#0305 - Create a website for local platform
#0306 - AssetCollection’s assets_from_directory logic wrong if set flatten and relative path at same time
#0313 - idmtools: MAX_SUBDIRECTORY_LENGTH = 35 should be made Global in COMPSPlatform definition
#0314 - Fix local platform to work with latest analyze/platform updates
#0316 - Integrate website with Local Runner Container
#0321 - COMPSPlatform _retrieve_experiment errors on experiments with and without suites
#0329 - Experiment level status
#0330 - Paging on simulation/experiment APIs for better UI experience
#0333 - ensure pyComps allows compatible releases
#0364 - Local platform should use production artfactory for docker images
#0381 - Support Work Items in COMPS Platform
#0387 - Local platform webUI only show simulations up to 20
#0393 - local platform tests keep getting EOFError while logger is in DEBUG and console is on
#0405 - Support analysis of data from Work Items in Analyze Manager
#0407 - Support Service Side Analysis through SSMT
#0447 - Set limitation for docker container’s access to memory
#0532 - Make updates to ExperimentManager/Platform to support tasks
#0540 - Create initial SSMT Plaform from COMPS Platform
#0596 - COMPSPlatform.get_files(item,..) not working for Experiment or Suite
#0635 - Update SSMT base image
#0639 - Add a way for the python_requirements_ac to use additional wheel file
#0676 - ssmt missing QueryCriteria support
#0677 - ssmt: refresh_status returns None
User Experience#
#0457 - Option to analyze failed simulations