idmtools.utils.hashing module¶

Fast hash of Python objects.

class idmtools.utils.hashing.Hasher(hash_name='md5')¶

Bases: pickle._Pickler

A subclass of pickler to do hashing, rather than pickling.

__init__(hash_name='md5')¶

Initialize our hasher.

Parameters:: hash_name – Hash type to use. Defaults to md5

hash(obj, return_digest=True)¶

Hash an object.

Parameters:

obj – Object to hash
return_digest – Should the digest be returned?

Returns:

None if return_digest is False, otherwise the hash digest is returned

save(obj)¶

Save an object to hash.

Parameters:: obj – Obj to save.
Returns:: None

memoize(obj)¶: Disable memoization for strings so hashing happens on value and not reference.

save_set(set_items)¶

Save set hashing.

Parameters:: set_items – Set items
Returns:: None

idmtools.utils.hashing.hash_obj(obj, hash_name='md5')¶

Quick calculation of a hash to identify uniquely Python objects.

Parameters:

obj – Object to hash
hash_name – The hashing algorithm to use. ‘md5’ is faster; ‘sha1’ is considered safer.

idmtools.utils.hashing.ignore_fields_in_dataclass_on_pickle(item)¶

Ignore certain fields for pickling on dataclasses.

Parameters:: item – Item to pickle
Returns:: State of item to pickle

idmtools.utils.hashing.calculate_md5(filename: str, chunk_size: int = 8192) → str¶

Calculate MD5.

Parameters:

filename – Filename to caclulate md5 for
chunk_size – Chunk size

Returns:

md5 as string

idmtools.utils.hashing.calculate_md5_stream(stream: Union[_io.BytesIO, BinaryIO], chunk_size: int = 8192, hash_type: str = 'md5', file_hash=None)¶

Calculate md5 on stream.

Parameters:

chunk_size –
stream –
hash_type – Hash function
file_hash – File hash

Returns:

md5 of stream