GHDB scraper produces inaccurate output

[marz-hunter]

I found that the output from ghdb scraper was not precise. for example the title gives "Google Dork" but when viewed it produces site:".edu" intitle:"index of"|".db" and the tool saves the output "Google Dork" instead of site:".edu" intitle:" index of"|".db"

bandicam.2024-02-03.13-23-30-421.mp4

[marz-hunter]

output tools. I think crawling a url like https://www.exploit-db.com/ghdb/8389 would be more accurate although it would take a little longer

[marz-hunter]

crawling and then taking this part will be more accurate

[opsdisk]

Hi @marz-hunter - thanks for opening an issue. In the past, I've also noticed the data isn't structured as precisely as it should be by exploit-db.com. I vaguely recall thinking exploit-db.com needs to clean up the data instead of wanting to handle edge cases or invest any more time in the script. Give me a week or two to dig deeper into it though.

In the mean time, you could reach out to them and see if they could clean up the ones you found. Email can be found here: https://www.exploit-db.com/submit

[opsdisk]

Just found this as well that could be used https://gitlab.com/exploit-database/exploitdb/-/blob/main/ghdb.xml

Mind checking it for the same dependencies you found?

Edit: You may be able to submit a PR against it for any ones you find as well. I'm guessing that is what powers https://www.exploit-db.com/google-hacking-database

[marz-hunter]

yeah this one looks pretty good https://gitlab.com/exploit-database/exploitdb/-/blob/main/ghdb.xml, but updates seem to take a while

[opsdisk]

I'll keep this issue for a while to see if they update it. Let me know if you do submit a PR against https://gitlab.com/exploit-database/exploitdb/-/blob/main/ghdb.xml

[opsdisk]

Just wanted to check back on this one @marz-hunter Did you have any updates?