idmtools.utils.hashing module

Fast hash of Python objects.

class idmtools.utils.hashing.Hasher(hash_name='md5')

Bases: pickle._Pickler

A subclass of pickler to do hashing, rather than pickling.

hash(obj, return_digest=True)

save(obj)

memoize(obj)

Disable memoization for strings so hashing happens on value and not reference.

save_set(set_items)

idmtools.utils.hashing.hash_obj(obj, hash_name='md5')

Quick calculation of a hash to identify uniquely Python objects.

Parameters

hash_name – The hashing algorithm to use. ‘md5’ is faster; ‘sha1’ is considered safer.

idmtools.utils.hashing.ignore_fields_in_dataclass_on_pickle(item)

idmtools.utils.hashing.calculate_md5(filename: str, chunk_size: int = 8192) → str

Calculate MD5

Parameters

• filename – Filename to caclulate md5 for

• chunk_size – Chunk size

Returns:

idmtools.utils.hashing.calculate_md5_stream(stream: Union[_io.BytesIO, BinaryIO], chunk_size: int = 8192, hash_type: str = 'md5', file_hash=None)

Calculate md5 on stream :param chunk_size: :param stream: :param hash_type: Hash function :param file_hash: File hash

Returns

md5 of stream