# Rust Based SEO and AEO Crawler

Project, Worldwide.

> Turn any website into clean, RAG-ready Markdown. Runs locally, renders JavaScript, respects robots.txt. No headless Chrome, no per-page fees.
>
> — in its own words, from crawler.sh, read 2026-09-30

- Website: https://crawler.sh/
- Listed since: 2026
- launched: 2026-03-02
- launch post: https://news.ycombinator.com/item?id=47218365
- hn score: 2
- Last verified: 2026-03-02


## Sources

- Hacker News, "Show HN" — https://news.ycombinator.com/item?id=47218365, read 2026-03-02

---
startup.is · https://startup.is/projects/rust-based-seo-and-aeo-crawler · open data: /api/v1, /llms.txt