Link Scraper in Python

[Xtha-Sunil]

1. A py script that takes url of website and scrape all links then dump them in txt file.
2. Categorize them based on anchor link, internal link, external link.
3. MetaData present in the site.

I think this is useful for SEO.

[devsdenepal]

Assigned!
How about enhancing the scraper to detect which frameworks a website uses by analyzing its linked resources? It could scan for framework-related files, libraries, or CDN links, then classify them to identify technologies like React, Vue, Laravel, or Django used in the site’s structure.

[Xtha-Sunil]

Thanks for suggestion. I will consider this as well in my script.