sitemap.xml: What It Is For and How to Build a Site Map

A brand new site has two hundred pages and fifteen of them show up in search. Search Console has no report of discovered URLs at all: the crawler simply does not know these pages exist. One file tells it, and that file is called sitemap.xml.

What a site map actually is

A site map is an XML file listing the addresses a crawler should visit. It has no direct effect on rankings and it is not mandatory: crawlers find pages through links perfectly well. What a site map does do is mark which addresses matter and speed up the discovery of new ones.

It helps in three situations. First, a site with many pages, some of them weakly linked, reachable only through a footer menu or through sorting parameters. Second, content that barely has internal links pointing at it, such as a date archive. Third, sites so large that a crawler physically cannot get through everything within a sensible time budget.

Discovery and indexing are different things, and conflating them causes most of the confusion. A site map says «these addresses exist, go and look». From there the crawler decides whether what it saw deserves to be indexed. If a page returns 404, carries a noindex, or duplicates another one, the site map will not save it. If anything, junk in the map burns the crawl budget for nothing.

The urlset format: what goes inside the file

A page map is a urlset root element with a mandatory namespace. Inside sit url elements, and each one requires exactly one child: loc, holding the full address of the page. The other three are optional, and most of the confusion around site maps comes from them.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
  </url>
  <url>
    <loc>https://example.com/blog/</loc>
    <lastmod>2026-11-14</lastmod>
  </url>
</urlset>

• loc, the full address of the page. The only mandatory child, without it the entry means nothing;
• lastmod, the date of the last meaningful content change. Either YYYY-MM-DD or a W3C date and time with a time zone;
• changefreq, the expected update frequency: always, hourly, daily, weekly, monthly, yearly or never;
• priority, the page priority from 0.0 to 1.0. The default is 0.5.

Every address in loc has to be absolute, with the protocol and the domain. Relative paths such as /blog/post/ are not allowed. Special XML characters have to be escaped: an ampersand in an address is written as &, and angle brackets simply never occur in ordinary URLs.

lastmod, changefreq and priority: what they really do

This deserves precision, because a pile of myths has grown around those three tags. Google states plainly in its documentation that it ignores both changefreq and priority. So if you set priority 0.9 on the home page and 0.1 on an internal section, that has no effect whatsoever on how the crawler behaves.

lastmod is more subtle but still widely misunderstood. In the official wording Google treats the value as a hint, and only when it is consistently and verifiably accurate. If every page in the map claims today while nothing actually changed, the crawler stops trusting the field. The practical conclusion: set lastmod only where a real change took place.

• lastmod does carry meaning and can influence how quickly a page is revisited, but it is a hint rather than a command to recrawl now;
• changefreq is ignored outright by Google, and other engines interpret it differently;
• priority is ignored outright by Google, so writing it only makes the format uniform;
• all three fields are optional, and a map with nothing but a loc per address is perfectly valid.

None of that cancels the convenience of the format for the site owner: the map shows when a page was last touched, which helps when planning a recrawl.

Types of maps: urlset, sitemapindex, images, video, news

The page map is only the base type. On top of it there are extensions, added through a separate namespace on the root element, that describe things which normally do not fit into a plain list of addresses.

• page map, the urlset element. The base type, listing addresses for crawling;
• sitemap index, the sitemapindex element. A list of map files, for when there is more than one;
• image map. An image tag inside a url entry carries the picture address and a caption;
• video map. Describes the clip, its duration, the publication date and a link to the page with the player;
• news map. For news items, with language, publication date and headline;
• localized versions. An xhtml:link tag inside one entry lists the alternates for different languages and regions.

A sitemap index is what you need past 50 000 pages or when the file outgrows the size limit. It points at separate map files, and inside it only the loc of each link is required, while lastmod says when the file itself changed rather than when a page inside it did.

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://example.com/sitemap-pages.xml</loc>
    <lastmod>2026-11-14</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://example.com/sitemap-blog.xml</loc>
    <lastmod>2026-11-14</lastmod>
  </sitemap>
</sitemapindex>

Generating the map on the server

Maintaining a map by hand in a text editor is pointless: within a week it will have gaps in it. The right approach is to generate the file from the source of truth, usually a database holding the page list and a column with the last modification date.

Three approaches are common. First, a server-side generator: a script or an application route that assembles the XML from the database on request. Second, a file rewritten on a schedule, typically once a day. Third, a built-in CMS mechanism: in WordPress that is an SEO plugin, in Next.js and similar frameworks it is a route of their own.

This project serves its map through a server route: the addresses are collected from the article registry and the tool pages, so fresh publications land in the map without anybody touching a file. One detail matters here: the addresses have to be complete, with the same scheme and the same domain as the pages that actually work.

export default function sitemap(): MetadataRoute.Sitemap {
  const base = "https://example.com";
  return [
    { url: `${base}/`, lastModified: new Date() },
    { url: `${base}/blog/`, lastModified: new Date() },
  ];
}

The XML formatter helps confirm that the file was assembled correctly: it shows the structure with indentation and points at an unclosed tag if there is one. To take every address apart and spot parameters and duplicates, the URL parser is the right tool.

Where to put the file and how to announce it

The conventional name is sitemap.xml at the domain root. That is the path crawlers probe by default, though relying on the guess is a poor idea.

The file may live anywhere on the site, but there is a condition: unless the map is submitted through Search Console, it covers only the files inside the directory where it sits. A map at the root covers the whole site, while a map in /blog/ covers the blog only. That is why a file parked in a subfolder without an explicit submission behaves differently from what people expect.

1. put the file at the domain root as sitemap.xml;
2. check that it returns 200 with an application/xml or text/xml content type;
3. add a Sitemap: directive to robots.txt so crawlers find it without guessing. The directive itself is described in the article about robots.txt and its directives;
4. submit the address in Search Console under the Sitemaps report, which then shows when the file was read and how many addresses were taken from it;
5. look at that report from time to time, because it also reports format errors and addresses the crawler read and then dropped.

Gzip deserves a note of its own. A site map can be compressed, and plenty of people do it because XML text compresses many times over. Some crawlers accept such files, but it is worth checking against the report: if Search Console does not read the map, try serving it uncompressed first and compare. The HTTP status reference confirms the response code and content type.

Junk URLs: what to keep out of the map

A site map is not a dumping ground for everything that returns 200. Every junk address burns part of the 50 000 URL budget and makes the crawler spend time on pages that were never worth indexing. The URL parser helps here, since it shows immediately where an unwanted parameter is hiding.

• pages with sorting and filtering parameters, such as /catalog/?sort=price&order=asc;
• internal site search results, which change with every query;
• login, cart and account pages;
• duplicates that differ only in parameter order: /a?x=1&y=2 and /a?y=2&x=1 are one address as far as a search engine is concerned;
• addresses carrying UTM tags, should any have made it into the index;
• service pages such as /search, /cart, /print and their relatives.

A separate category consists of addresses that return 404 or a redirect. Those must not go into the map: the crawler spends a fetch, sees an error, and loses some of its trust in the site. If such addresses have already surfaced in Search Console, close them with robots.txt or noindex and remove them from the map.

Typical mistakes

• a missing namespace. Without xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" the file counts as invalid;
• relative addresses where absolute ones are required;
• an unescaped ampersand, which breaks parsing of the whole XML document;
• addresses with query parameters that flood the map;
• a map in a subfolder with no Search Console submission, which then covers less than expected;
• lastmod set to today on every page, which teaches the crawler to stop trusting the field;
• unescaped characters in URLs, where a space has to be written as %20;
• a file full of 404 addresses and noindex pages, which only wastes the crawl budget.

Common questions

Is sitemap.xml mandatory?

Formally, no: crawlers discover pages through links and a small site indexes perfectly well without a map. What a map does add is a way to stop the crawler from guessing, to signal which addresses matter, and to give it a reason to visit pages that barely have links pointing at them. For a hundred pages it takes ten minutes to write by hand and then you forget about it, and for a large project it is generated automatically.

Does a site map improve rankings?

There is no direct effect on rankings, and promising one would be wrong. What a map does is let the crawler find and visit pages sooner, and that does affect indexing speed. On a site where crawling is not a problem in the first place, there may be no visible effect at all.

The map exists but pages are not indexed. Now what?

Read the indexing report in Search Console rather than the map itself. There are several likely causes: the page returns 404 or a redirect, it carries a noindex, it is blocked in robots.txt, it duplicates another page by canonical, or it simply has too few internal links and too little authority. The indexing report names the specific reason for each address.

Should I set priority and change frequency?

There is no practical benefit: Google ignores both fields. You can leave them out, or fill them in for consistency, especially if other systems read the map. lastmod is a different story, since it is genuinely taken as a hint, but only when the date is truthful. To check how addresses are structured, the URL parser takes them apart.

Root folder or subfolder?

The root. Here is why: a map placed in a subdirectory covers only the files inside its own parent directory unless it is submitted separately through Search Console. For a large site with several sections, the cleaner shape is separate maps tied together by a sitemapindex, with the index itself sitting at the root.

How do I know the map works?

Submit the address in Search Console and wait for it to be processed. The report then shows how many addresses were found, how many got indexed, and lists any parse errors with the offending lines. Check the file itself at the same time: the XML formatter shows the structure and highlights unclosed tags, while the status code reference confirms it is served with a 200.