Skip to content

varungv/django-web-crawler-web-app

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

django-web-crawler

This a web application which crawls through a given website and returns all the links and images given of a URL. We can also select the level of the crawl to go into multiple levels of the links and get the data. The Crawl creates threads to improve the speed of the crawling of multiple levels.

I will be adding a feature to build the Website tree of the structure using this feature

This application has a simple built in UI, but i also have a React based UI in the below repository https://github.com/varungv/WebCrawler-UI

** A demo video of the application in action is present inside the repostory **

About

django-web-crawler

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published