Skip to content
View cl0q-coder's full-sized avatar
🕸️
crawling
🕸️
crawling
3 followers · 0 following

Achievements

Achievement: Pull SharkAchievement: Quickdraw

Achievements

Achievement: Pull SharkAchievement: Quickdraw

Block or report cl0q-coder

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
cl0q-coder/README.md

cl0q — the independent search engine

cl0q.com is a free, independent web search engine with its own crawler and index: no cookies, no account needed. Built for OSINT and threat-hunting work — 47.1M domains mapped, 34.0M pages indexed, growing by ~525k pages a day. (Index stats: October 2026.)

Open source

  • crawler-node — the distributed web crawler behind the index
  • botid — bot detection and identification
  • netguard — SSRF protection for the crawler's outbound requests
  • robotstxt — fast robots.txt parsing

Use it

Pinned Loading

  1. cl0qsearch/crawler-node cl0qsearch/crawler-node Public

    cl0q crawler-node - open source

    Go

  2. botid botid Public

    cl0q botid - open source

    Go

  3. robotstxt robotstxt Public

    cl0q robotstxt - open source

    Go

  4. netguard netguard Public

    cl0q netguard - open source

    Go