Merge pull request #40 from brootware/dev

Dev

Author: brootware

README.md

@@ -132,30 +132,33 @@ nltk.download()" -->
 Run as below to redact a single file
 
 ```bash
-$ poetry run pyredactkit ip_test.txt 
-
-    ______       ______         _            _     _   ___ _   
-    | ___ \      | ___ \       | |          | |   | | / (_) |  
-    | |_/ /   _  | |_/ /___  __| | __ _  ___| |_  | |/ / _| |_ 
-    |  __/ | | | |    // _ \/ _` |/ _` |/ __| __| |    \| | __|
-    | |  | |_| | | |\ \  __/ (_| | (_| | (__| |_  | |\  \ | |_ 
-    \_|   \__, | \_| \_\___|\__,_|\__,_|\___|\__| \_| \_/_|\__|
-           __/ |                                               
-           |___/                                                                                                           
-            +-+-+-+-+-+-+-+ +-+-+ +-+-+-+-+-+-+-+-+-+
-            |P|o|w|e|r|e|d| |b|y| |B|r|o|o|t|w|a|r|e|
-            +-+-+-+-+-+-+-+ +-+-+ +-+-+-+-+-+-+-+-+-+
+$ poetry run pyredactkit -f logdata/test.txt                                    
+
+__________         __________           .___              __     ____  __.__  __   
+\______   \___.__. \______   \ ____   __| _/____    _____/  |_  |    |/ _|__|/  |_ 
+ |     ___<   |  |  |       _// __ \ / __ |\__  \ _/ ___\   __\ |      < |  \   __\
+ |    |    \___  |  |    |   \  ___// /_/ | / __ \\  \___|  |   |    |  \|  ||  |  
+ |____|    / ____|  |____|_  /\___  >____ |(____  /\___  >__|   |____|__ \__||__|  
+           \/              \/     \/     \/     \/     \/               \/                                                                                                                 
+                    +-+-+-+-+-+-+-+ +-+-+ +-+-+-+-+-+-+-+-+-+
+                    |P|o|w|e|r|e|d| |b|y| |B|r|o|o|t|w|a|r|e|
+                    +-+-+-+-+-+-+-+ +-+-+ +-+-+-+-+-+-+-+-+-+
 
     https://github.com/brootware
-    https://brootware.github.io                                                                             
+    https://brootware.github.io     
+    https://twitter.com/brootware                                                                        
 
-[ + ] Processing starts now. This may take some time depending on the file size. Monitor the redacted file size to monitor progress
-[ + ] No option supplied, will be redacting all the sensitive data supported
-[ + ] Redacted 10064 targets...
-[ + ] Redacted results saved to ./redacted_ip_test.txt
+[+] Processing starts now. This may take some time depending on the file size. Monitor the redacted file size to monitor progress
+[+] No custom regex pattern supplied, will be redacting all the core sensitive data supported
+[+] .hashshadow_test.txt.json file generated. Keep this safe if you need to undo the redaction.
+[+] Redacted 10068 targets...
+[+] Redacted results saved to ./redacted_test.txt
+[+] Estimated total words : 30316
+[+] Estimated total minutes saved : 405
+[+] Estimated total man hours saved : 6
 ```
 
-Sample Result (Note that name is not redacted by default):
+Sample Result:
 
 ```txt
 John, please get that article on b8bd54d3-34ee-4f31-8b2b-0d729929e8aa to me by 5:00PM on Jan 9th 2012. 4:00 would be ideal, actually. If you have any questions, You can reach me at(519)-236-2723 or get in touch with my associate at 7b3c7641-4b09-4e00-8e02-0e68e47b0ded.
@@ -201,51 +204,26 @@ e0b66cbd-6174-4491-b938-408a47d38fb9,Platinum,142000,CC90518
 24f31233-cba6-4f6a-a2d6-0ce49952b2cb,Premium,781000,CC66746
 ```
 
-Sample result:
-
-```txt
-███████████████, please get that article on www.linkedin.com to me by 5:00PM on Jan 9th 2012. 4:00 would be ideal, actually. If you have any questions, You can reach me at(519)-236-2723 or get in touch with my associate at [email protected].
-All rights reserved. Printed in the United States of America. No part of this book may be used or reproduced in any manner whatsoever without written permission except in the case of brief quotations embodied in critical articles and reviews. For information address HarperCollins Publishers, 10 East 53rd Street, New York, NY 10022. His name is ███████████████. I met him and ███████████████ last week. ███████████████ is a good student. I was born on Oct 4, 1995. My Indian mobile number is +91-7761975545. After coming to USA I got a new number +1-405-413-5255. I live on 1003 E ███████████████, Norman, Ok, 73071. I met  a child, who is playing with josh.
-this is my IP: 102.23.5.1
-My router is : 10.10.10.1
-71.159.188.33
-81.141.167.45
-165.65.59.139
-64.248.67.225
-
-https://tech.gov.sg
-
-My email is [email protected]
-
-this is my IP: 102.23.5.1
-My router is: 10.10.10.1
-71.159.188.33
-```
-
-To redact multiple files from a directory and place them in a new directory
-
-```bash
-poetry run pyredactkit dir_test -d redacted_dir
-```
-
 ## Optional Help Menu as below
 
 ```bash
-usage: pyredactkit [-h] [-f FILE [FILE ...]] [-u UNREDACT] [-d DIROUT] [-r] [-e EXTENSION] [text ...]
+usage: pyredactkit [-h] [-f FILE [FILE ...]] [-u UNREDACT] [-d DIROUT] [-c CUSTOMFILE] [-r] [-e EXTENSION] [text ...]
 
-Supply a sentence or paragraph to redact sensitive data from it. Or read in a file or set of files with -f , and return the result.
+Supply a sentence or paragraph to redact sensitive data from it. Or read in a file or set of files with -f to redact
 
 positional arguments:
   text                  Redact sensitive data of a sentence from command prompt. (default: None)
 
 optional arguments:
   -h, --help            show this help message and exit
   -f FILE [FILE ...], --file FILE [FILE ...]
-                        Path of a file or a directory of files. Usage: pyredactkit [file/filestoredact] (default: None)
+                        Path of a file or a directory of files. (default: None)
   -u UNREDACT, --unredact UNREDACT
                         Option to unredact masked data. Usage: pyredactkit -f [redacted_file] -u [.hashshadow.json] (default: None)
   -d DIROUT, --dirout DIROUT
                         Output directory of the file. Usage: pyredactkit -f [file/filestoredact] -d [redacted_dir] (default: None)
+  -c CUSTOMFILE, --customfile CUSTOMFILE
+                        User defined custom regex pattern for redaction. Usage: pyredactkit -f [file/filestoredact] -c [customfile.json] (default: None)
   -r, --recursive       Search through subfolders (default: True)
   -e EXTENSION, --extension EXTENSION
                         File extension to filter by. (default: )
@@ -254,5 +232,5 @@ optional arguments:
 ## Sample files
 
 - [All types of data](./logdata/test.txt)
-- [Differnt log file types](./logdata/)
+- [Different log file types](./logdata/)
 - [test_sample2.txt - 10002 lines of IP addresses](https://sanitizationbq.s3.ap-southeast-1.amazonaws.com/test_sample2.txt)