tools.helpers.utils package¶
Submodules¶
tools.helpers.utils.decorators module¶
Wrappers for date functions
-
tools.helpers.utils.decorators.handle_datetime_dataframe(func)¶ Decorator to make functions converting dates usable for pandas DataFrame.
- Function requirements:
first argument can be a pandas Serie object
- keyword argument ‘inplace’ may exist (used for DataFrame objects) but:
must not be passed as a positional argument
is False by default
General warning: using complex objects with inheritances or __instancecheck__ method is not recommended
-
tools.helpers.utils.decorators.handle_datetime_pandas_objects(func)¶ Decorator to make functions converting dates usable for pandas Series and DataFrame.
- Function requirements:
first argument must be a datetime-like object
- keyword argument ‘inplace’ may exist (used for DataFrame objects) but:
must not be passed as a positional argument
must not be used by the function itself
- Performance warning: in many cases, the pandas Serie method “apply” is slower than direct/vectorial operations
Consider using the “.dt” accessor for example
General warning: using complex objects with inheritances or __instancecheck__ method is not recommended
-
tools.helpers.utils.decorators.handle_datetime_serie(func)¶ Decorator to make functions converting dates usable for pandas Series or lists.
- Function requirements:
first argument must be a datetime-like object
- Performance warning: in many cases, the pandas Serie method “apply” is slower than direct/vectorial operations
Consider using the “.dt” accessor for example
General warning: using complex objects with inheritances or __instancecheck__ method is not recommended
tools.helpers.utils.module_utils module¶
Utils to reload modules while python is launched. Be cautious when using them, because some weird behaviors can happen with some imports.
-
tools.helpers.utils.module_utils.reload_functions(module, functions: Union[list, tuple, Callable])¶ Reloads a function or a list/tuple of functions.
-
tools.helpers.utils.module_utils.reload_modules(modules: Union[list, module], reload_func=False, ls_func_names='__all__')¶ Reloads the modules in list ‘modules’ and the functions selected if ‘reload_func’ is True.
Reload twice to be sure the majority of modules and dependencies are reloaded. It is possible that some modules with tricky dependencies are not completely reloaded For example:
if ‘tools.helpers’ depends on ‘tools’, the first reload is be sufficient to completely reload ‘tools.helpers’ as ‘tools’ is reloaded before ‘tools.helpers’
if ‘tools’ depends on ‘tools.helpers’ (e.g. imports of ‘tools.helpers’ functions in __all__), the first reload is not sufficient to reload ‘tools’ as ‘tools.helpers’ is reloaded after ‘tools’. Thus, a new reload is needed.
if ‘tools’ depends on ‘tools.helpers’ which depends on ‘tools.helpers.utils’, ‘tools’ can not be reloaded properly in one execution, so it is necessary to execute this function twice. Normally, this case is rare with modules with few submodules.
To avoid uncompleted reloads, it is advised to pass a list of modules with few interdependence in the right order (lasts modules dependent of the first ones, not the contrary).
- Parameters
modules – list of modules
reload_func – if True, reload selected functions (see ls_func_names arg)
ls_func_names –
name or list of names of either: - an iterable (list or tuple) in __init__ which contains functions to reload (e.g. __all__).
The iterable must have one of this format: [func1, func2, …] or [(_, func1, _), (_, func2, _), …]
a function in __init__
- Returns
None
-
tools.helpers.utils.module_utils.reload_package(package, reload_func=False, ls_func_names: Union[list, str] = '__all__')¶ Reloads the module ‘package’ and the functions selected if ‘reload_func’ is True.
- Parameters
package – package to reload (includes subpackages)
reload_func – if True, reload selected functions (see ls_func_names arg)
ls_func_names – name or list of names of iterables in __init__ which contains functions to reload.
The iterable must have one of this format: [func1, func2, …] or [(_, func1, _), (_, func2, _), …] :return: None
tools.helpers.utils.utils module¶
Basic utils that can be used in the whole project This module should not need of other modules (except logger).
-
tools.helpers.utils.utils.coordinates_to_distance(lat1, lon1, lat2, lon2)¶ Compute the approximate distance between two points of coordinates (lat1, log1), (lat2, long2). Formula adapted from https://en.wikipedia.org/wiki/Haversine_formula
-
tools.helpers.utils.utils.isinlist(obj, iterable)¶ Returns whether an object is in the iterable or not.
Support is granted for numpy and pandas objects which have multiple definitions of truth. One numpy/pandas object is considered in the iterable if the object itself is in the iterable. Therefore, if a copy but not the object is present in the iterable, isinlist returns False.
>>> df = pd.DataFrame() >>> ls = [5, 'a', np.array([5, 4]), df] >>> isinlist(5, ls), isinlist(6, ls), isinlist('a', ls), isinlist('b', ls) (True, False, True, False) >>> isinlist(df, ls), isinlist(df.copy(), ls), isinlist(np.array([5, 4]), ls) (True, False, False)
-
tools.helpers.utils.utils.isinstance_filetypes(filetypes)¶ Returns True if ‘filetypes’ is a valid variable for filetypes argument in tkinter.filedialog functions, else False.
-
tools.helpers.utils.utils.isiterable(obj)¶ Returns whether an object is “iterable” or not. Are considered iterable tuple, list, set and inherited classes
-
tools.helpers.utils.utils.merge_dict(left_dict: dict, right_dict: dict, how: str = 'outer', inplace=True) → dict¶ Merges a right_dict into a left_dict with the method how. left_dict can be copied or modified inplace.
-
tools.helpers.utils.utils.merge_dict_preprocessing(left_dict: dict, right_dict: dict, how: str = 'outer') → Union[Tuple[Union[set, KeysView], bool], Tuple[None, None]]¶ Returns the tuple (keys, update) which permit to update a ‘left_dict’ with a ‘right_dict’ with the method ‘how’.
Examples:
>>> left_dict, right_dict = {1: "a", 2: "b", 3: "c"}, {1: "d", 4: "e"} >>> merge_dict_preprocessing(left_dict, right_dict, how='left') ({1}, True) >>> merge_dict_preprocessing(left_dict, right_dict, how='inner') ({1}, False) >>> merge_dict_preprocessing(left_dict, right_dict, how='right_anti') ({4}, False) >>> merge_dict_preprocessing(left_dict, right_dict, how='right') (dict_keys([1, 4]), False) >>> merge_dict_preprocessing(left_dict, right_dict, how='outer') (dict_keys([1, 4]), True) >>> merge_dict_preprocessing(left_dict, right_dict, how='append') ({4}, True) >>> merge_dict_preprocessing(left_dict, right_dict, how='none') (set(), True) >>> merge_dict_preprocessing(left_dict, right_dict, how='clear') (set(), False) >>> merge_dict_preprocessing(left_dict, right_dict, how='bad_string') (None, None)
Example of use:
>>> keys, update = merge_dict_preprocessing(left_dict, right_dict, how='outer') >>> if not update: left_dict.clear() >>> left_dict.update([(k, right_dict[k]) for k in keys]) >>> left_dict {1: 'd', 2: 'b', 3: 'c', 4: 'e'}
- Parameters
left_dict – left dictionary-like object
right_dict – right dictionary-like object
Dictionary-like object need the methods ‘keys’ which returns a set-like object with methods ‘__sub__’ and ‘__and__’. :param how: - ‘left’: only right_dict keys already in left_dict will be updated.
‘inner’: only keys in self AND right_dict will be kept.
‘right_anti’: only keys in right_dict AND NOT in left_dict.
‘right’ or ‘replace’: only right_dict keys will be kept.
‘outer’or ‘update’: all keys of self AND right_dict will be kept.
‘append’: append new keys of right_dict, but don’t modify existing keys in left_dict.
‘none’ or None: no modification
‘clear’: clear left_dict
- Returns
keys (set), update (bool)
-
tools.helpers.utils.utils.merge_two_levels_dict(left_dict: dict, right_dict: dict, how='outer', how_section=None, inplace=False) → dict¶ Modify left_dict inplace, updated with right_dict. Left and right dict must have at least two levels.
-
tools.helpers.utils.utils.shorten_dataframe_str(df: pandas.core.frame.DataFrame, column_name: str, max_length: int, replacement_str: str = '...', inplace: bool = False) → pandas.core.frame.DataFrame¶ Shorten string values that are longer than ‘max_length’ in column ‘column_name’. If some values are not strings, they are converted.
>>> df = pd.DataFrame(columns=["a", "b"], data=[["baobab", "octopus"], ["birthday", "mark"], [1, 2]]) >>> shorten_dataframe_str(df, 'a', 6) a b 0 baobab octopus 1 birthd... mark 2 1 2
- Parameters
df – pandas dataframe
column_name – column name of dataframe df
max_length – maximum length of strings
replacement_str – string to append to the cut strings. Default: ‘…’
inplace – if True, the input dataframe is modified inplace
- Returns
dataframe modified
Module contents¶
Basic utils that can be used in the whole project This module should not need of other modules (except logger).
-
tools.helpers.utils.isiterable(obj)¶ Returns whether an object is “iterable” or not. Are considered iterable tuple, list, set and inherited classes
-
tools.helpers.utils.isinstance_filetypes(filetypes)¶ Returns True if ‘filetypes’ is a valid variable for filetypes argument in tkinter.filedialog functions, else False.
-
tools.helpers.utils.isinlist(obj, iterable)¶ Returns whether an object is in the iterable or not.
Support is granted for numpy and pandas objects which have multiple definitions of truth. One numpy/pandas object is considered in the iterable if the object itself is in the iterable. Therefore, if a copy but not the object is present in the iterable, isinlist returns False.
>>> df = pd.DataFrame() >>> ls = [5, 'a', np.array([5, 4]), df] >>> isinlist(5, ls), isinlist(6, ls), isinlist('a', ls), isinlist('b', ls) (True, False, True, False) >>> isinlist(df, ls), isinlist(df.copy(), ls), isinlist(np.array([5, 4]), ls) (True, False, False)
-
tools.helpers.utils.merge_dict_preprocessing(left_dict: dict, right_dict: dict, how: str = 'outer') → Union[Tuple[Union[set, KeysView], bool], Tuple[None, None]]¶ Returns the tuple (keys, update) which permit to update a ‘left_dict’ with a ‘right_dict’ with the method ‘how’.
Examples:
>>> left_dict, right_dict = {1: "a", 2: "b", 3: "c"}, {1: "d", 4: "e"} >>> merge_dict_preprocessing(left_dict, right_dict, how='left') ({1}, True) >>> merge_dict_preprocessing(left_dict, right_dict, how='inner') ({1}, False) >>> merge_dict_preprocessing(left_dict, right_dict, how='right_anti') ({4}, False) >>> merge_dict_preprocessing(left_dict, right_dict, how='right') (dict_keys([1, 4]), False) >>> merge_dict_preprocessing(left_dict, right_dict, how='outer') (dict_keys([1, 4]), True) >>> merge_dict_preprocessing(left_dict, right_dict, how='append') ({4}, True) >>> merge_dict_preprocessing(left_dict, right_dict, how='none') (set(), True) >>> merge_dict_preprocessing(left_dict, right_dict, how='clear') (set(), False) >>> merge_dict_preprocessing(left_dict, right_dict, how='bad_string') (None, None)
Example of use:
>>> keys, update = merge_dict_preprocessing(left_dict, right_dict, how='outer') >>> if not update: left_dict.clear() >>> left_dict.update([(k, right_dict[k]) for k in keys]) >>> left_dict {1: 'd', 2: 'b', 3: 'c', 4: 'e'}
- Parameters
left_dict – left dictionary-like object
right_dict – right dictionary-like object
Dictionary-like object need the methods ‘keys’ which returns a set-like object with methods ‘__sub__’ and ‘__and__’. :param how: - ‘left’: only right_dict keys already in left_dict will be updated.
‘inner’: only keys in self AND right_dict will be kept.
‘right_anti’: only keys in right_dict AND NOT in left_dict.
‘right’ or ‘replace’: only right_dict keys will be kept.
‘outer’or ‘update’: all keys of self AND right_dict will be kept.
‘append’: append new keys of right_dict, but don’t modify existing keys in left_dict.
‘none’ or None: no modification
‘clear’: clear left_dict
- Returns
keys (set), update (bool)
-
tools.helpers.utils.merge_dict(left_dict: dict, right_dict: dict, how: str = 'outer', inplace=True) → dict¶ Merges a right_dict into a left_dict with the method how. left_dict can be copied or modified inplace.
-
tools.helpers.utils.merge_two_levels_dict(left_dict: dict, right_dict: dict, how='outer', how_section=None, inplace=False) → dict¶ Modify left_dict inplace, updated with right_dict. Left and right dict must have at least two levels.
-
tools.helpers.utils.shorten_dataframe_str(df: pandas.core.frame.DataFrame, column_name: str, max_length: int, replacement_str: str = '...', inplace: bool = False) → pandas.core.frame.DataFrame¶ Shorten string values that are longer than ‘max_length’ in column ‘column_name’. If some values are not strings, they are converted.
>>> df = pd.DataFrame(columns=["a", "b"], data=[["baobab", "octopus"], ["birthday", "mark"], [1, 2]]) >>> shorten_dataframe_str(df, 'a', 6) a b 0 baobab octopus 1 birthd... mark 2 1 2
- Parameters
df – pandas dataframe
column_name – column name of dataframe df
max_length – maximum length of strings
replacement_str – string to append to the cut strings. Default: ‘…’
inplace – if True, the input dataframe is modified inplace
- Returns
dataframe modified
-
tools.helpers.utils.coordinates_to_distance(lat1, lon1, lat2, lon2)¶ Compute the approximate distance between two points of coordinates (lat1, log1), (lat2, long2). Formula adapted from https://en.wikipedia.org/wiki/Haversine_formula
-
tools.helpers.utils.reload_modules(modules: Union[list, module], reload_func=False, ls_func_names='__all__')¶ Reloads the modules in list ‘modules’ and the functions selected if ‘reload_func’ is True.
Reload twice to be sure the majority of modules and dependencies are reloaded. It is possible that some modules with tricky dependencies are not completely reloaded For example:
if ‘tools.helpers’ depends on ‘tools’, the first reload is be sufficient to completely reload ‘tools.helpers’ as ‘tools’ is reloaded before ‘tools.helpers’
if ‘tools’ depends on ‘tools.helpers’ (e.g. imports of ‘tools.helpers’ functions in __all__), the first reload is not sufficient to reload ‘tools’ as ‘tools.helpers’ is reloaded after ‘tools’. Thus, a new reload is needed.
if ‘tools’ depends on ‘tools.helpers’ which depends on ‘tools.helpers.utils’, ‘tools’ can not be reloaded properly in one execution, so it is necessary to execute this function twice. Normally, this case is rare with modules with few submodules.
To avoid uncompleted reloads, it is advised to pass a list of modules with few interdependence in the right order (lasts modules dependent of the first ones, not the contrary).
- Parameters
modules – list of modules
reload_func – if True, reload selected functions (see ls_func_names arg)
ls_func_names –
name or list of names of either: - an iterable (list or tuple) in __init__ which contains functions to reload (e.g. __all__).
The iterable must have one of this format: [func1, func2, …] or [(_, func1, _), (_, func2, _), …]
a function in __init__
- Returns
None