idmtools.utils.general module#

Tag Parsing and Normalization Utilities.

This module provides helper functions and classes for converting string-based metadata tags into native Python types (e.g., bool, int, float) to enable accurate filtering and comparison.

It includes: - JSON serialization/deserialization of tags - Conversion of sets to lists for JSON compatibility - Coercion logic via TagValue for safe and flexible comparisons

Typical use cases include tag-based filtering of simulations or experiments during analysis.

Typical Use Case:#

Used during filtering of simulations or experiments in analysis pipelines where user-defined tags are compared with numeric thresholds or exact matches.

Example:#

sim.tags={“Run_Number”: lambda v: 4 <= v <= 10, “Coverage”: “0.8”} parsed = parse_value_tags(sim.tags, wrap_with_tagvalue=True) assert parsed[“Run_Number”] >=4 and <=10 assert parsed[“Coverage”] == 0.8

idmtools.utils.general.parse_item_tags(item: IEntity)[source]#

Normalize and update an entity’s tags in place.

This function parses the given item’s tags using parse_value_tags and updates the item.tags dictionary with the normalized values.

Parameters:: item – An entity object that contains a .tags dictionary.
Returns:: The same item, with its .tags dictionary updated in-place.

idmtools.utils.general.parse_value_tags(tags: Dict, wrap_with_tagvalue: bool = False) → Dict[str, any][source]#

Parse and normalize a tag dictionary into native Python types.

Converts string tag values such as:

“true”/”false” → bool
“5” → int
“0.8” → float
sets → lists

Optionally wraps values using TagValue for comparison safety.

Parameters:

tags (dict) – The dictionary of raw tag values to parse.
wrap_with_tagvalue (bool) – If True, wraps each value in a TagValue object for smart comparison support.

Returns:

A dictionary with normalized or wrapped tag values.

Return type:

dict

class idmtools.utils.general.SetEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]#

Bases: JSONEncoder

Custom JSON encoder that converts Python sets into lists to ensure compatibility with JSON serialization.

default(obj)[source]#

Override default encoding behavior.

Parameters:: obj – The object being encoded.
Returns:: JSON-compatible representation of the object.

class idmtools.utils.general.CustomDecoder(*args, **kwargs)[source]#

Bases: JSONDecoder

Custom JSON decoder that converts string values into appropriate Python types (bool, int, float, None).

__init__(*args, **kwargs)[source]#

object_hook, if specified, will be called with the result of every JSON object decoded and its return value will be used in place of the given dict. This can be used to provide custom deserializations (e.g. to support JSON-RPC class hinting).

object_pairs_hook, if specified will be called with the result of every JSON object decoded with an ordered list of pairs. The return value of object_pairs_hook will be used instead of the dict. This feature can be used to implement custom decoders. If object_hook is also defined, the object_pairs_hook takes priority.

parse_float, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).

parse_int, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).

parse_constant, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN. This can be used to raise an exception if invalid JSON numbers are encountered.

If strict is false (true is the default), then control characters will be allowed inside strings. Control characters in this context are those with character codes in the 0-31 range, including '\t' (tab), '\n', '\r' and '\0'.

static denormalize_tag_value(val)[source]#

Convert a raw string tag value into a Python-native type.

Supported conversions: - “true”/”false” → bool - “null”/”none” → None - “5”, “0.8” → int, float

Parameters:: val (Any) – The value to convert.
Returns:: The normalized Python value.

custom_object_hook(obj: Dict)[source]#

Apply normalization to each value in a decoded JSON object.

Parameters:: obj (dict) – Dictionary of tag values.
Returns:: Normalized tag dictionary.
Return type:: dict

class idmtools.utils.general.TagValue(raw)[source]#

Bases: object

Wrapper for a tag value that supports smart comparisons.

Automatically converts strings like “5” or “0.8” to int/float and enables comparison against numbers or other tag values.

Useful when users perform tag-based filtering with operators like >, <, ==, etc., and tag values may be strings.

raw#

The original tag value before coercion.

Type:: Any

__init__(raw)[source]#

class idmtools.utils.general.FilterSafeItem(item)[source]#

Bases: object

A lightweight wrapper around an entity (e.g., Simulation or Experiment) to enable safe tag-based filtering during multiprocessing operations (e.g., within analyzers).

This wrapper: - Normalizes and wraps tag values via TagValue to support flexible comparisons (e.g., >, ==). - Supports safe pickling/unpickling by stripping unpickleable fields like _platform. - Delegates attribute access to the original wrapped item via __getattr__ (if implemented).

Typical usage:: safe_item = FilterSafeItem(simulation) tags = safe_item.tags # Now tags[“Run_Number”] supports TagValue comparison logic

_item#

The original entity being wrapped.

Type:: IEntity

__init__(item)[source]#

Initialize the filter-safe wrapper with a given entity.

Parameters:: item (IEntity) – The original simulation or experiment to wrap.

property tags#

Get normalized tags from the wrapped item, with each value wrapped in TagValue for type-safe comparison.

Returns:: Dictionary of tags.
Return type:: Dict[str, any]