robots.txt Explain

User-agent, allow/disallow, sitemap from pasted robots.txt.

4.1 (57)

4799 views

Input

Text Or load a text file

Runs in your browser. Nothing is uploaded to Velix.

HTML Links HTML Meta HTML Select HTML Table → CSV Extract URLs Extract Emails JSON Path RSS / Atom Sitemap URLs robots.txt Explain JSON-LD Extract