Static vs Dynamic Sitemaps for Large Publishers

Small sites never think about how a sitemap is built. Large ones have to. The difference between a sitemap that gets read and one that gets ignored often comes down to a single number: how long it takes to respond.

The hidden timeout

Most plugins build sitemaps on the fly, querying the database every time the file is requested. On a few hundred posts that is instant. On tens of thousands — especially with images attached to each URL — generation can stretch past twenty seconds. Googlebot does not wait that long. The fetch fails, the sitemap shows as pending, and child sitemaps go unread for weeks.

Static is boring and that is the point

The fix is to pre-generate the XML and serve it as a plain file. The web server hands it over in milliseconds, with no PHP and no database in the path. A scheduled job rebuilds the files once a day, or after a publish burst. We have seen Google go from “could not fetch” to downloading the index in about a second after this switch — same content, different delivery.

Keep the index clean

Split by type and cap each child file well under the 50,000-URL limit. A focused blockchain news archive with tidy, fast sitemaps gives the crawler a clear map instead of a slow maze.

When to bother

Under a few thousand URLs, dynamic is fine. Past that, watch your sitemap fetch time in Search Console. If “last read” lags days behind “last submitted,” you have found your bottleneck — and static files are the cure.