XML Sitemap Best Practices in 2026: What Google Uses, Checked Against a Real Sitemap

What an XML sitemap does, and what it does not

An XML sitemap is a file, usually at /sitemap.xml, that lists the URLs on a site you want search engines to know about. Each entry can carry the date the page last changed. Google's documentation is careful about its limits. Submitting a sitemap "is merely a hint: it doesn't guarantee that Google will download the sitemap or use the sitemap for crawling URLs on the site" [1].

Google also says who needs one. A site of "about 500 pages or fewer" that is "comprehensively linked internally" may not need a sitemap at all. Large sites, new sites with few external links, and sites with rich media or news content are the cases where it helps [2]. Anobee, at 94 URLs, is in the "may not need one" band. Nevertheless, it keeps one, for a reason the rest of this article demonstrates: the sitemap is the cleanest way to get a filtered view of indexing in Search Console.

What a sitemap does not do: it does not rank pages and does not force indexing. It also does not override a noindex tag, a robots.txt block or a canonical pointing elsewhere. Discovery is the only stage it speeds up.

Sitemap update frequency: two different questions

The phrase covers two things that get mixed together, and they have opposite answers.

How often should the sitemap file itself be updated? Every time content changes: a new page, a meaningful edit, a removal. In practice that means the CMS should generate the file on request rather than someone exporting it. Anobee's sitemap is generated on each request and cached for five minutes at the edge. Therefore a page published at 10:00 is in the file by 10:05. You do not need to resubmit it. Google's help page says that once a submission succeeds, "Google will continue to recrawl the sitemap at a pace independent of the site crawl schedule" [4].

The old "ping" endpoint that plugins used to notify Google of changes was deprecated in June 2023 and stopped working six months later. Gary Illyes wrote that "the vast majority of the submissions lead to spam" [3]. Plugins that still ping are not harming anything. However, they are not doing anything either.

Should you set the changefreq tag? No. Google "still doesn't use the changefreq or priority elements at all", and changefreq "is also conceptually overlapping with lastmod" [3]. If the file updates itself when content changes and carries an honest lastmod, Google already has the information. Changefreq was a rougher version of the same thing.

lastmod, priority and changefreq: what Google reads

The sitemap protocol allows three optional elements per URL. Here is Google's current position on each, from its own documentation:

Two details on lastmod matter in practice. First, "significant" means the main content, structured data or links changed. A new copyright year or a sidebar tweak does not count [1]. Second, the value has to be earned. Illyes put it plainly: "if your page changed 7 years ago, but you're telling us in the lastmod element that it changed yesterday, eventually we're not going to believe you anymore" [3]. Similarly, it is fine to omit lastmod on pages where the CMS cannot know it. A homepage or category page that only aggregates other pages is the usual example [3].

What Anobee's sitemap actually contains

Opening anobee.com/sitemap.xml on 21 September 2026 showed 94 URLs in the XML sitemap. Every one carried a priority value (0.75 on 71 of them, 0.7 on 16, 0.8 on 6, 1.0 on the homepage). Every one also carried a changefreq (monthly on 86, weekly on 8). None of that is read by Google. It is not wrong, and there is no reason to rush a code change over it. Nevertheless, it is a good example of a generator emitting fields because the protocol allows them, not because anyone decided to.

The lastmod values are more interesting. 79 of the 94 URLs have one, taken from the CMS record's last-updated timestamp. The 15 without are static pages: the homepage, the blog index, the policies. Leaving those out is exactly what Google recommends.

The remaining question is whether the CMS timestamp changes only on significant edits. It does not. Attaching an image or fixing a typo updates the record, and therefore the lastmod. For a site publishing a few articles a week that is a small inaccuracy. The dates still track real editing closely enough to be believed. However, on a site where a plugin resaves every post nightly, the same setup would make lastmod worthless. If you cannot make the timestamp mean "significant change", the honest options are to accept the small drift or to omit the element. Faking it is not one of them.

What to include in an XML sitemap, and what to leave out

The rule is one line: list the URLs you would be happy to see indexed, exactly as they should be indexed. Consequently, everything below follows from it.

Anobee's numbers show why the exclusions matter more than the inclusions. Search Console knows 518 URLs for the site: 120 indexed and 398 not. Of the 398, 174 are redirects and 135 are 404s, almost all of them addresses from the site's earlier WordPress build. The XML sitemap lists 94. That gap is correct. The old URLs redirect where they should and have no business in the file. Because they are not in it, the sitemap-filtered view of the indexing report stays readable.

The one thing the audit did turn up is a URL-form detail. Anobee's canonical tags and sitemap both use URLs without a trailing slash. The slash version returns a 308 redirect to the canonical. That is consistent, which is the point: whichever form you pick, the sitemap, the canonical tag and the redirect should all agree. Otherwise you get the "Google chose different canonical than user" line in the indexing report. That line appears in Anobee's data below.

How to submit an XML sitemap

There are two channels Google reads and one it does not. Accordingly, use the first two.

1. robots.txt. Add one line anywhere in the file: Sitemap: https://example.com/sitemap.xml [1]. Anobee's robots.txt carries exactly that line. This is the channel every crawler that respects robots.txt can see, not only Google.
2. Search Console. Indexing → Sitemaps → enter the path → Submit [1]. The Search Console setup guide covers verifying the property first. Submit once; there is no need to resubmit after routine edits.
3. The ping endpoint. Retired [3]. Ignore any tool that still offers it.

Bing Webmaster Tools accepts a sitemap the same way. It also reads the robots.txt line, so step 1 covers it.

What Anobee's Sitemaps report showed

The report listed two submissions, both with a Success status and both showing 94 discovered pages. One was /sitemap.xml, submitted 18 August 2026. The other was /sitemap_index.xml, submitted 14 July 2026. The second is a leftover. sitemap_index.xml is the filename Rank Math and Yoast generate on WordPress. After the move to the current stack it no longer exists as a file and returns a 308 redirect to /sitemap.xml. Google follows the redirect, reads the same file, and reports it twice. Harmless, but a two-minute cleanup: delete the old submission so the report stops double-counting. If your site has ever changed CMS or SEO plugin, check this report for the same thing.

Success is not indexed: reading the two reports together

The Sitemaps report answers one question: could Google fetch and parse the XML sitemap? Its three statuses are Success ("fetched and read without any errors"), Has errors, and Couldn't fetch [4]. "Discovered pages" is the count of URLs parsed from the file. The help page adds that there is no guarantee those pages will be crawled or indexed [4].

The question you actually care about is how many of those URLs are indexed. That lives in Indexing → Pages, with the dropdown switched from "All known pages" to "All submitted pages". That switch is the most useful thing an XML sitemap gives a small site. It turns 518 known URLs into the 94 you chose.

For Anobee on 21 September 2026 (report data to 18 September), that filtered view showed 73 indexed and 21 not, for four reasons:

One caution on timing. Inspecting one of the six "duplicate" URLs with the URL Inspection tool on 21 September showed it indexed and last crawled 20 September. The Google-selected canonical matched the declared one. The Page indexing report lags the inspection tool by days. Therefore, before fixing a "duplicate" entry, inspect it; it may already have resolved.

The 11 "discovered, not indexed" pages are the honest part of the picture. A sitemap put them in front of Google. However, Google has not decided they are worth a crawl yet. That is the limit of what the file can do. The technical SEO mistakes guide covers the internal-linking and content work that moves pages out of that bucket.

Sitemap files, sitemap indexes and HTML sitemaps

A single XML sitemap is limited to 50,000 URLs or 50 MB uncompressed [1]. Above that, split the URLs across several files and list the files in a sitemap index. WordPress SEO plugins do this by default, which is why sitemap_index.xml is such a common filename. Below the limit, one file is simpler to check and to submit.

Google also documents three extensions [2]. Image sitemaps list the images on each page. Video sitemaps can carry duration, rating and age-suitability. News sitemaps are for publishers in Google News. Anobee's file uses the image extension, with 72 image entries across the 94 URLs. The file must be UTF-8 encoded and use absolute URLs [1].

An HTML sitemap is a different object: an ordinary page listing the site's sections for people, not crawlers. It is not a substitute for the XML file. Nevertheless, on a small site it doubles as a useful internal-linking page.

Where the XML sitemap comes from

• WordPress. Core generates /wp-sitemap.xml on its own. Yoast and Rank Math replace it with /sitemap_index.xml and disable the core one. Run one or the other, not both, and make sure Search Console has the live filename.
• Shopify, Wix, Squarespace. Generated automatically at /sitemap.xml; you control it by unpublishing or hiding pages, not by editing the file.
• Custom builds (Next.js, Astro, Rails and similar). The sitemap is a route your code renders from the content database. This is Anobee's setup, and it is where the priority and changefreq fields came from: the generator template included them. It is also the setup where lastmod is most under your control, since you decide which field feeds it.

Whichever generator you use, open the file in a browser and read it once. Ultimately, that is the whole method. Most of the findings in this article came from doing exactly that.

Common XML sitemap errors and fixes

Most of these trace back to one of three things. The sitemap disagrees with robots.txt, it disagrees with the canonical tags, or the server failed to serve it. Check those three before anything else.

XML sitemaps and AI crawlers

The previous version of this guide claimed a clean sitemap helps AI crawlers find content to cite. That claim did not survive verification. OpenAI's crawler documentation describes three bots: GPTBot for training, OAI-SearchBot for ChatGPT search and ChatGPT-User for pages a user asks about. It does not mention sitemaps at all [5]. Google-Extended, sometimes listed as a crawler, is a robots.txt control token rather than a bot with its own crawl. None of this means AI crawlers never read an XML sitemap. It means their operators have not said so, and an article should not say it for them.

What is verifiable is narrower. Keep the sitemap clean for Google and Bing. Keep robots.txt deliberate about which AI bots you allow. Then measure citations directly. Anobee's guide to checking AI citations covers how. Meanwhile, the Perplexity SEO guide covers the one AI system whose crawler documentation is detailed enough to act on.

The 15-minute XML sitemap audit

This is the check that produced the findings above, in the order that surfaces problems fastest.

1. Open the file. Confirm it loads, is XML, and lists URLs you recognise. Also note the count.
2. Spot-check five URLs. Each should return 200 and carry a canonical tag matching the sitemap entry exactly, slash for slash.
3. Check robots.txt. Confirm the Sitemap: line points at the live filename and that no Disallow rule covers sitemap URLs.
4. Open the Sitemaps report. One entry per real file. Delete leftovers from old plugins or migrations.
5. Switch the Page indexing report to "All submitted pages". Compare indexed to discovered; read the not-indexed reasons.
6. Inspect one URL from the largest not-indexed reason. The report lags. Meanwhile, the inspection tool is current.
7. Read the lastmod values. They should cluster around real publishing and editing dates. If every URL shows today, the field is being updated on trivial saves.
8. Ignore priority and changefreq. If they are present, leave them. However, if you are writing a generator, leave them out.

Repeat the XML sitemap audit after any migration, plugin change or bulk edit, and otherwise quarterly. The content audit guide is the natural companion. Usually the pages that fail step 5 are the ones a content audit would have flagged anyway.

Bottom line on XML sitemap best practices

An XML sitemap is a shortlist, not a lever. Google reads the URLs and, if you have been honest, the lastmod dates; it ignores the rest. Generate the file automatically, keep it to canonical URLs that return 200, reference it in robots.txt, and submit it once. Then do the part most people skip: read the indexing report filtered to submitted pages. On Anobee's site that single view showed 21 URLs out of 94 not indexed, two duplicate submissions and 94 priority values nobody reads. Ultimately, none of it was visible from the green "Success" status.