3 tools
The files and tags a site needs, written correctly
Generate the meta tags, crawler rules and sitemap a site needs, with the syntax handled for you and a preview of what each one actually does.
3
Tools in this category, all free
0 bytes
Uploaded — there is no server to send to
No
Account, watermark or daily limit
- Meta Tag GeneratorBuild title, description, Open Graph and Twitter tags for a page.
- Robots.txt GeneratorWrite crawler allow/disallow rules without syntax mistakes.
- XML Sitemap GeneratorTurn a list of URLs into a valid sitemap.xml for Google and Bing.
About these tools
These three files are unusual in that a small syntax mistake in any of them is both easy to make and expensive: one careless `Disallow: /` line keeps an entire site out of search results, and a malformed sitemap is quietly ignored. Generating them from a form removes the class of error that comes from typing them by hand.
Worth being straight about a limit here. These tools build files from what you give them — they do not crawl your website. The sitemap generator turns a list of URLs you supply into valid sitemap.xml; it cannot discover your pages for you, because a page in your browser is not allowed to fetch another site, and this one’s security policy forbids outbound connections in any case. If you need a crawler, you need software that runs on a server or on your desktop. If you already know your URLs — from a CMS export, a build script or your own sitemap route — this will produce the file.
Two things the robots.txt generator will not let you assume: `Disallow` controls crawling, not indexing, and a URL blocked there can still appear in results if other sites link to it. To remove a page from search you need a `noindex` directive on a page a crawler is allowed to fetch.
Frequently asked questions
No, and no browser-based tool can. A page is not permitted to fetch arbitrary other sites, and this one is additionally forbidden from making outbound connections at all. Supply your list of URLs — from a CMS export or your own build — and it will produce a valid sitemap.xml.
No. It asks crawlers not to fetch the page. A blocked URL can still be indexed and shown if other pages link to it, because the block prevents reading, not listing. Use a noindex directive on a crawlable page when you want something out of the index.
Roughly 140 to 160 characters. Shorter wastes space you have been given; longer is cut off. The generator shows you a preview at realistic width so you can see where it lands rather than counting characters.
The rest of the toolbox
Every category works the same way — the file or the text stays on your device.