Skip to content

Commit ffa82f1

Browse files
committed
MagicRecon version 1.1
1 parent 5560f78 commit ffa82f1

File tree

3 files changed

+133
-102
lines changed

3 files changed

+133
-102
lines changed

README.md

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,17 @@
44
Recon is an essential element of any penetration testing. This repository contain a powerful shell script to maximize the recon and data collection process of an objective. With this script you can easily find:
55

66
* Sensitive information disclosure.
7-
* Missing HTTP headers
7+
* Missing HTTP headers.
8+
* Heartbleed Bug.
89
* Open S3 buckets.
910
* Subdomain takeovers.
11+
* Bugs in TLS/SSL ciphers, protocols and cryptographic flaws.
1012
* Open ports and services.
13+
* Email spoofing.
1114
* Endpoints.
1215
* Directories.
13-
* Javascript files with senstive info
14-
* CORS missconfigurations
16+
* Javascript files with senstive info.
17+
* CORS missconfigurations.
1518
* Other quick bugs.
1619

1720

@@ -22,40 +25,47 @@ Recon is an essential element of any penetration testing. This repository contai
2225
# Tools needed
2326
* [Amass](https://github.com/OWASP/Amass)
2427
* [Certsh.py](https://github.com/ghostlulzhacks/CertificateTransparencyLogs)
25-
* [Github-subdomains.py](https://github.com/gwen001/github-search)
2628
* [Gobuster](https://github.com/OJ/gobuster)
2729
* [Assetfinder](https://github.com/tomnomnom/assetfinder)
2830
* [Subjack](https://github.com/haccer/subjack)
2931
* [httprobe](https://github.com/tomnomnom/httprobe)
3032
* [Corsy](https://github.com/s0md3v/Corsy)
3133
* [Aquatone](https://github.com/michenriksen/aquatone)
32-
* [curl](https://curl.haxx.se/)
34+
* [securityheaders](https://github.com/koenbuyens/securityheaders)
35+
* [testssl.sh](https://github.com/drwetter/testssl.sh)
36+
* [spoofcheck](https://github.com/BishopFox/spoofcheck)
3337
* [relative-url-extractor](https://github.com/jobertabma/relative-url-extractor)
34-
* [Jsearch.py](https://github.com/incogbyte/jsearch)
38+
* [linkfiner](https://github.com/GerbenJavado/LinkFinder)
3539
* [Nmap](https://nmap.org/)
3640
* [SecLists](https://github.com/danielmiessler/SecLists)
3741

38-
### IMPORTANT: YOU NEED TO INSTALL ALL THE TOOLS IN YOUR HOME FOLDER AND INSERT YOUR GITHUB TOKEN IN THE SCRIPT CONFIGURATION TO USE Github-subdomains.py.
42+
### IMPORTANT: YOU NEED TO INSTALL ALL THE TOOLS IN YOUR HOME FOLDER.
3943

4044
# How does it work?
4145
The script has 5 phases:
4246

43-
1. Subdomain enumeration: Amass, Certsh.py, Github-subdomains.py, Gobuster DNS and Assetfinder tools are used to find the maximum possible number of subdomains. httprobe is used to probe for working http and https servers. Then Subjack is used to quickly check if it exists subdomains takeover. Corsy tool is used to find CORS missconfigurations. Finally, Aquatone takes screenshots of each subdomain.
47+
1. Subdomain enumeration: Amass, Certsh.py, Gobuster DNS and Assetfinder tools are used to find the maximum possible number of subdomains. httprobe is used to probe for working http and https servers. Then Subjack is used to quickly check if it exists subdomains takeover. Corsy tool is used to find CORS missconfigurations. Finally, Aquatone takes screenshots of each subdomain.
4448

45-
2. Headers: curl is used to obtain the headers of each subdomain.
49+
2. Scan for missing headers and bugs in SSL/TLS protocols: securityheaders is used to check quickly and easily the security of HTTP response headers and testssl.sh is used to check the TLS/SSL ciphers, protocols and cryptographic flaws.
4650

47-
3. Javascript: relative-url-extractor and Jsearch.py are used to inspect the javascript files of each subdomain for endpoints and sensitive information.
51+
3. Scan if a domain can be spoofed: spoofcheck is used to check SPF and DMARC records for weak configurations that allow spoofing.
4852

49-
4. Directories and hidden files: Gobuster DIR is used to collect hidden files and directories through a dictionary. You can change the dictionary in the script configuration.
53+
4. JavaScript files and hidden endpoints: LinkFinder is used to discover endpoints and their parameters in JavaScript files.
5054

51-
5. Nmap: Nmap is used to scan ports and services quiclky.
55+
5. Find directories and hidden files: Gobuster DIR is used to collect hidden files and directories through a dictionary. You can change the dictionary in the script configuration.
56+
57+
6. Nmap: Nmap is used to scan ports and services quiclky.
5258

5359
### All the data generated in the different processes are saved in different files and directories in different formats.
5460
![Example image](https://raw.githubusercontent.com/robotshell/magicRecon/master/example.png)
5561

5662
# Usage
5763
```
58-
./magicRecon.sh <domain>
64+
./magicRecon.sh [DOMAIN]
65+
66+
Parameters:
67+
68+
-h: Show the help message.
5969
```
6070
# Thanks
6171
* Special Thanks to Mohd Shibli for his great contributions in the article [Fasten your Recon process using Shell Scripting](https://medium.com/bugbountywriteup/fasten-your-recon-process-using-shell-scripting-359800905d2a#id_token=eyJhbGciOiJSUzI1NiIsImtpZCI6ImRiMDJhYjMwZTBiNzViOGVjZDRmODE2YmI5ZTE5NzhmNjI4NDk4OTQiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJuYmYiOjE1NzQxODIxNTUsImF1ZCI6IjIxNjI5NjAzNTgzNC1rMWs2cWUwNjBzMnRwMmEyamFtNGxqZGNtczAwc3R0Zy5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsInN1YiI6IjEwNjQzNTQ3NTE5MTA1NzIzOTYzOSIsImVtYWlsIjoicm9ib3RzaGVsbGRAZ21haWwuY29tIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsImF6cCI6IjIxNjI5NjAzNTgzNC1rMWs2cWUwNjBzMnRwMmEyamFtNGxqZGNtczAwc3R0Zy5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsIm5hbWUiOiJSb2JvdCBTaGVsbCIsInBpY3R1cmUiOiJodHRwczovL2xoMy5nb29nbGV1c2VyY29udGVudC5jb20vYS0vQUF1RTdtQnhZZklJNndVLXQ5OVNxbzFlaElpc1E4dzY4a2VJbWZrbE4yOD1zOTYtYyIsImdpdmVuX25hbWUiOiJSb2JvdCIsImZhbWlseV9uYW1lIjoiU2hlbGwiLCJpYXQiOjE1NzQxODI0NTUsImV4cCI6MTU3NDE4NjA1NSwianRpIjoiODYzMTNhZTQ3YTQ5NjJiNTdhMTBlZDA0NGJhYWUyMGQwZWM2Y2FlNCJ9.obOev9FLt7DWW2NbSIbFwPoUC-vNFrf5nru--6uL6knW1S6CjjqXAP_D0sedfukNC0DcJnqQDz88Yh48ECppB4wEv0ozgunc9Yx24m5OiNaEKvWr0D2WJsMsV9yN7Vxt7gJxTeVIstCLvWDYCl_1JBrDvJ2eXF4V9yamk61KCqmwoAJMjXEpwaDuzITFPIZM9V-nTpIgnsBh-BCERYqAcUc7Si0IpRAlyM9YG78va7o0Pe_zYrt4NbV8Cl--BzAzrFOfhIOxvk3CYWRfc9lrSz09TJRCEn4q-rR9v7LVIboKJAedhbkr8ShClMru8xRsdfne3fRIzV1iZxNn4GuW6A)

example.png

10.9 KB
Loading

magicRecon.sh

100644100755
Lines changed: 110 additions & 89 deletions
Original file line numberDiff line numberDiff line change
@@ -9,50 +9,69 @@ gobusterDictionaryPath=~/SecLists/Discovery/DNS/namelist.txt
99
aquatoneTimeout=50000
1010
gobusterDirThreads=50
1111
gobusterDictionaryPathDir=~/SecLists/Discovery/Web-Content/raft-medium-files-directories.txt
12-
githubToken=YOUR GITHUB TOKEN
12+
testsslParameters="--quiet --fast -p -s -S -h -U --color 3 --htmlfile"
1313

1414
#COLORS
1515
BOLD="\e[1m"
1616
NORMAL="\e[0m"
17-
GREEN="\e[92m"
17+
GREEN="\033[1;32m"
18+
MAGENTA="\e[95m"
19+
YELLOW="\e[33m"
20+
DEFAULT="\e[39m"
21+
22+
#########HELP SECTION#########
23+
if [ "$1" == "-h" ]; then
24+
echo -e "${BOLD}${GREEN}[+] MagicRecon is a powerful shell script to maximize the data collection process of an objective. Has 5 steps:"
25+
echo -e "${NORMAL}${GREEN}[+] STEP 1: Subdomain Enumeration"
26+
echo -e "${DEFAULT}${NORMAL}Amass, Certsh.py, Gobuster DNS and Assetfinder tools are used to find the maximum possible number of subdomains. httprobe is used to probe for working http and https servers. Then Subjack is used to quickly check if it exists subdomains takeover. Corsy tool is used to find CORS missconfigurations. Finally, Aquatone takes screenshots of each subdomain."
27+
echo -e "${NORMAL}${GREEN}[+] STEP 2: Scan for missing headers and bugs in SSL/TLS protocols"
28+
echo -e "${DEFAULT}${NORMAL}securityheaders is used to check quickly and easily the security of HTTP response headers and testssl.sh is used to check the TLS/SSL ciphers, protocols and cryptographic flaws."
29+
echo -e "${NORMAL}${GREEN}[+] STEP 3: Scan if a domain can be spoofed"
30+
echo -e "${DEFAULT}${NORMAL}spoofcheck is used to check SPF and DMARC records for weak configurations that allow spoofing."
31+
echo -e "${NORMAL}${GREEN}[+] STEP 4: JavaScript files and hidden endpoints"
32+
echo -e "${DEFAULT}${NORMAL}LinkFinder is used to discover endpoints and their parameters in JavaScript files."
33+
echo -e "${NORMAL}${GREEN}[+] STEP 5: Find directories and hidden files"
34+
echo -e "${DEFAULT}${NORMAL}Gobuster DIR is used to collect hidden files and directories through a dictionary. You can change the dictionary in the script configuration."
35+
echo -e "${NORMAL}${GREEN}[+] STEP 6: Port scan for alive domains"
36+
echo -e "${DEFAULT}${NORMAL}Nmap is used to scan ports and services quiclky."
37+
echo -e ""
38+
echo -e "${BOLD}${GREEN}You have more information in https://github.com/robotshell/magicRecon"
39+
echo -e ""
40+
echo -e "${BOLD}${YELLOW}[+] DON'T FORGET -> If you found the tool useful please consider donating to support it's development. You can help me to develop more useful tools. THANKS :)"
41+
echo -e ""
42+
echo -e "${BOLD}${GREEN}[+] AUTHOR: ROBOTSHELL"
43+
echo -e "${BOLD}${GREEN}[+] TWITTER: https://twitter.com/robotshelld"
44+
exit 0
45+
fi
1846

1947
#########SUBDOMAIN ENUMERATIONS#########
20-
echo -e "${BOLD}${GREEN}[+] Welcome to MagicRecon"
48+
printf "${MAGENTA}"
49+
figlet "MagicRecon"
2150
echo -e ""
22-
echo -e "${BOLD}${GREEN}[+] MagicRecon has 5 steps: "
23-
echo -e "${BOLD}${GREEN}[+] STEP 1: Subdomain Enumeration"
24-
echo -e "${BOLD}${GREEN}[+] STEP 2: Subdomain headers and response bodies"
25-
echo -e "${BOLD}${GREEN}[+] STEP 3: JavaScript files and Hidden Endpoints"
26-
echo -e "${BOLD}${GREEN}[+] STEP 4: Find directories and hidden files"
27-
echo -e "${BOLD}${GREEN}[+] STEP 5: Port scan for alive domains"
2851

2952
echo -e ""
3053
echo -e "${BOLD}${GREEN}[+] STEP 1: Starting Subdomain Enumeration"
3154

3255
#Amass
33-
echo -e "${GREEN}[+] Starting Amass"
56+
echo -e "${BOLD}${YELLOW}[+] Starting Amass${DEFAULT}${NORMAL}"
3457
amass enum -norecursive -noalts -d $1 -o domains.txt
3558

3659
#Crt.sh
37-
echo -e "${GREEN}[+] Starting Certsh.py"
60+
echo -e "${BOLD}${YELLOW}[+] Starting Certsh.py${DEFAULT}${NORMAL}"
3861
python ~/CertificateTransparencyLogs/certsh.py -d $1 | tee -a domains.txt
3962

40-
#Github-Search
41-
echo -e "${GREEN}[+] Starting Github-subdomains.py"
42-
python3 ~/github-search/github-subdomains.py -d $1 -t $githubToken | tee -a domains.txt
43-
4463
#Gobuster
45-
echo -e "${GREEN}[+] Starting Gobuster DNS"
64+
echo -e "${BOLD}${YELLOW}[+] Starting Gobuster DNS${DEFAULT}${NORMAL}"
4665
gobuster dns -d $1 -w $gobusterDictionaryPath -t $gobusterDNSThreads -o gobusterDomains.txt
4766
sed 's/Found: //g' gobusterDomains.txt >> domains.txt
4867
rm gobusterDomains.txt
4968

5069
#Assetfinder
51-
echo -e "${GREEN}[+] Starting Assetfinder"
70+
echo -e "${BOLD}${YELLOW}[+] Starting Assetfinder${DEFAULT}${NORMAL}"
5271
~/go/bin/assetfinder --subs-only $1 | tee -a domains.txt
5372

5473
#Subjack
55-
echo -e "${GREEN}[+] Starting Subjack for search subdomains takevoer"
74+
echo -e "${BOLD}${YELLOW}[+] Starting Subjack for search subdomains takevoer${DEFAULT}${NORMAL}"
5675
subjack -w domains.txt -t $subjackThreads -timeout $subjackTime -ssl -c ~/subjack/fingerprints.json -v 3
5776

5877
#Removing duplicate entries
@@ -61,126 +80,128 @@ sort -u domains.txt -o domains.txt
6180

6281
#Discovering alive domains
6382
echo -e ""
64-
echo "[+] Checking for alive domains.."
83+
echo "[+] Checking for alive domains..${DEFAULT}${NORMAL}"
6584
cat domains.txt | ~/go/bin/httprobe | tee -a alive.txt
6685

6786
sort alive.txt | uniq -u
6887

88+
#Removing http/https protocol from alive.txt
89+
cp alive.txt alive-subdomains.txt
90+
sed -i 's#^http://##; s#/score/$##' alive-subdomains.txt
91+
sed -i 's#^https://##; s#/score/$##' alive-subdomains.txt
92+
sort -u alive-subdomains.txt -o alive-subdomains.txt
93+
6994
#Corsy
7095
echo -e ""
71-
echo -e "${GREEN}[+] Starting Corsy to find CORS missconfigurations"
96+
echo -e "${BOLD}${YELLOW}[+] Starting Corsy to find CORS missconfigurations${DEFAULT}${NORMAL}"
7297
python3 ~/Corsy/corsy.py -i alive.txt -o CORS.txt
7398

7499
#Aquatone
75100
echo -e ""
76-
echo -e "${BOLD}${GREEN}[+] Starting Aquatone to take screenshots"
101+
echo -e "${BOLD}${YELLOW}[+] Starting Aquatone to take screenshots${DEFAULT}${NORMAL}"
102+
103+
if [ -d screenshots ]; then rm -Rf screenshots; fi
77104

78105
mkdir screenshots
79106

80107
CUR_DIR=$(pwd)
81108

82-
cat alive.txt | aquatone -screenshot-timeout $aquatoneTimeout -out screenshots/
109+
cat alive.txt | ~/aquatone/aquatone -screenshot-timeout $aquatoneTimeout -out screenshots/
110+
83111

84112
#Parse data jo JSON
85113

86114
cat alive.txt | python -c "import sys; import json; print (json.dumps({'domains':list(sys.stdin)}))" > alive.json
87115

88116
cat domains.txt | python -c "import sys; import json; print (json.dumps({'domains':list(sys.stdin)}))" > domains.json
89117

90-
#########SUBDOMAIN HEADERS#########
118+
cat alive-subdomains.txt | python -c "import sys; import json; print (json.dumps({'domains':list(sys.stdin)}))" > alive-subdomains.json
119+
120+
#########MISSING HEADERS AND SSL/TLS PROTOCOLS#########
91121
echo -e ""
92-
echo -e "${BOLD}${GREEN}[+] STEP 2: Storing subdomain headers and response bodies"
122+
echo -e "${BOLD}${GREEN}[+] STEP 2: Scan for missing headers and SSL/TLS protocols${DEFAULT}${NORMAL}"
123+
echo -e "${BOLD}${YELLOW}[+] Starting securityheaders for search missing headers${DEFAULT}${NORMAL}"
124+
125+
#Scan for missing headers
126+
if [ -d headers ]; then rm -Rf headers; fi
93127

94128
mkdir headers
95129

96130
CURRENT_PATH=$(pwd)
97131

98-
for x in $(cat alive.txt)
132+
for x in $(cat alive-subdomains.txt)
99133
do
100-
NAME=$(echo $x | awk -F/ '{print $3}')
101-
curl -X GET -H "X-Forwarded-For: evil.com" $x -I > "$CURRENT_PATH/headers/$NAME"
102-
curl -s -X GET -H "X-Forwarded-For: evil.com" -L $x > "$CURRENT_PATH/responsebody/$NAME"
134+
NAME=$(echo $x)
135+
echo -e "${YELLOW}Analyzing headers to" $x
136+
python ~/securityheaders/securityheaders.py $x --skipcheckers InfoCollector --formatter markdown > "$CURRENT_PATH/headers/$NAME"
103137
done
104138

105-
#########JAVASCRIPT FILES#########
139+
if [ -d ssl ]; then rm -Rf ssl; fi
140+
106141
echo -e ""
107-
echo -e "${BOLD}${GREEN}[+] STEP 3: Collecting JavaScript files and Hidden Endpoints"
108142

109-
mkdir scripts
110-
mkdir scriptsresponse
111-
mkdir responsebody
143+
#Scan for vulns in SSL/TLS protocols
144+
echo -e "${BOLD}${YELLOW}[+] Starting testssl.sh for search bugs in TLS/SSL ciphers, protocols and cryptographic flaws${DEFAULT}${NORMAL}"
145+
mkdir ssl
112146

113-
RED='\033[0;31m'
114-
NC='\033[0m'
115-
CUR_PATH=$(pwd)
147+
CURRENT_PATH=$(pwd)
148+
149+
cd ssl/
116150

117-
for x in $(ls "$CUR_PATH/responsebody")
151+
for x in $(cat ../alive-subdomains.txt)
118152
do
119-
printf "\n\n${RED}$x${NC}\n\n"
120-
END_POINTS=$(cat "$CUR_PATH/responsebody/$x" | grep -Eoi "src=\"[^>]+></script>" | cut -d '"' -f 2)
121-
for end_point in $END_POINTS
122-
do
123-
len=$(echo $end_point | grep "http" | wc -c)
124-
mkdir "scriptsresponse/$x/"
125-
URL=$end_point
126-
if [ $len == 0 ]
127-
then
128-
URL="https://$x$end_point"
129-
fi
130-
file=$(basename $end_point)
131-
curl -X GET $URL -L > "scriptsresponse/$x/$file"
132-
echo $URL >> "scripts/$x"
133-
done
153+
NAMEFILE=$x
154+
EXTENSION=".html"
155+
NAMEEXTENSION="$NAMEFILE$EXTENSION"
156+
echo -e "${YELLOW}Analyzing SSL/TLS ciphers, protocols and cryptographic flaws to" $x
157+
~/testssl.sh/./testssl.sh $testsslParameters $NAMEEXTENSION $x > /dev/null
158+
134159
done
135160

136-
mkdir endpoints
161+
cd ..
137162

138-
CUR_DIR=$(pwd)
163+
#########SPF and DMARC records#########
164+
echo -e ""
165+
echo -e "${BOLD}${GREEN}[+] STEP 3: Scan if a domain can be spoofed${DEFAULT}${NORMAL}"
166+
echo -e "${BOLD}${YELLOW}[+] Starting spoofcheck for search SPF and DMARC records${DEFAULT}${NORMAL}"
139167

140-
for domain in $(ls scriptsresponse)
141-
do
142-
#looping through files in each domain
143-
mkdir endpoints/$domain
144-
for file in $(ls scriptsresponse/$domain)
145-
do
146-
ruby ~/relative-url-extractor/extract.rb scriptsresponse/$domain/$file >> endpoints/$domain/$file
147-
148-
if [ ! -s endpoints/$domain/$file ] ;
149-
then
150-
rm endpoints/$domain/$file
151-
fi
152-
done
153-
done
168+
if [ -d JS ]; then rm -Rf JS; fi
154169

155-
echo -e "${GREEN}[+] Starting Jsearch.py"
156-
organitzationName= sed 's/.com//' <<< "$1"
157-
mkdir javascript
170+
mkdir email
158171

159-
for domain in $(cat alive.txt)
172+
CURRENT_PATH=$(pwd)
173+
174+
for domain in $(cat alive-subdomains.txt)
160175
do
161-
NAME=$(echo $domain | awk -F/ '{print $3}')
162-
cd javascript/
163-
mkdir $NAME
164-
echo -e "${GREEN}[+] Searching JS files for $NAME"
165-
echo -e ""
166-
python3 ~/jsearch/jsearch.py -u $domain -n "$organitzationName" | tee -a $NAME.txt
176+
NAMEFILE=$domain
177+
EXTENSION=".txt"
178+
NAMEEXTENSION="$NAMEFILE$EXTENSION"
179+
echo -e "${YELLOW}Analyzing SPF and DMARC records for weak configurations that allow spoofing to" $domain
180+
python ~/spoofcheck/spoofcheck.py $domain >> "$CURRENT_PATH/email/$NAMEEXTENSION"
181+
done
182+
#########JAVASCRIPT FILES#########
183+
echo -e ""
184+
echo -e "${BOLD}${GREEN}[+] STEP 4: Collecting JavaScript files and Hidden Endpoints${DEFAULT}${NORMAL}"
185+
echo -e "${BOLD}${YELLOW}[+] Starting LinkFinder for discover endpoints in JavaScript files${DEFAULT}${NORMAL}"
167186

168-
if [ -z "$(ls -A $NAME/)" ] ;
169-
then
170-
rmdir $NAME
171-
fi
187+
if [ -d JS ]; then rm -Rf JS; fi
172188

173-
if [ ! -s $NAME.txt ] ;
174-
then
175-
rm $NAME.txt
176-
fi
189+
mkdir JS
177190

178-
cd ..
191+
CURRENT_PATH=$(pwd)
192+
193+
for domain in $(cat alive.txt)
194+
do
195+
NAMEFILE=$(echo $domain | awk -F/ '{print $3}')
196+
EXTENSION=".txt"
197+
NAMEEXTENSION="$NAMEFILE$EXTENSION"
198+
echo -e "${YELLOW}Analyzing JS files for endpoints, API Keys and many more to" $domain
199+
python ~/LinkFinder/linkfinder.py -d -i $domain -o cli >> "$CURRENT_PATH/JS/$NAMEEXTENSION"
179200
done
180201

181202
#########FILES AND DIRECTORIES#########
182203
echo -e ""
183-
echo -e "${BOLD}${GREEN}[+] STEP 4: Starting Gobuster to find directories and hidden files"
204+
echo -e "${BOLD}${GREEN}[+] STEP 5: Starting Gobuster to find directories and hidden files${DEFAULT}${NORMAL}"
184205

185206
mkdir directories
186207

@@ -197,7 +218,7 @@ done
197218

198219
#########NMAP#########
199220
echo -e ""
200-
echo -e "${BOLD}${GREEN}[+]STEP 5: Starting Nmap Scan for alive domains"
221+
echo -e "${BOLD}${GREEN}[+]STEP 6: Starting Nmap Scan for alive domains${DEFAULT}${NORMAL}"
201222

202223
mkdir nmapscans
203224

0 commit comments

Comments
 (0)