-
Notifications
You must be signed in to change notification settings - Fork 27
Open
Labels
Description
utils.dataload.dict_sweep is used to:
Remove keys whose values are ".", "-", "", "NA", "none", " "; and remove empty dictionaries
It's quite powerful, but could be potentially dangerous to remove some fields we want to keep. So, it would be nice to have a final report at the end of its run, something like this:
<na_value_1>:
- <removed_field_name_1>: cnt_of_removed_values,
...
<na_value_2>:
...
can add a new report parameter to this function, default can be True.
Also worth checking if setting to True will significantly increase the processing time in a typical data plugin, if that is the case, we might set default to False and use it only if needed for debugging purpose.
Reactions are currently unavailable