Upgrading Umbraco from version 13 to 17 is not a lift-and-shift. Views get renamed, property editors change, and whole rendering pipelines shift under your feet. When the site has hundreds of pages, manually checking each one after an upgrade is not realistic. I built a small .NET console tool that uses the sitemap to crawl every URL on both environments and flags what broke.
The comparison only makes sense if both environments serve the same content. Before running the tool, restore a copy of the production database to your upgrade target environment. If the databases diverge, even by a single content publish, he HTML diff results become noise. Same DB, same content, different Umbraco version: now any difference is a real problem.
The tool loads all URLs from the sitemap (supports <sitemapindex> with nested sitemaps and .gz compression), then asks which mode to run.
Mode 1: Status Check crawls every URL in parallel and writes any non-200 response to a CSV. This is the first pass: find your 500 errors, missing routes, and redirects that went wrong.
Mode 2: HTML Comparison fetches each URL from both environments simultaneously and compares the rendered HTML. The similarity score is length-based:
var similarityPercentage = maxLength > 0
? (1.0 - (double)lengthDiff / maxLength) * 100
: 100;
If similarity drops below the configured threshold (default 95%), it retries up to MaxRetries times before writing the result to CSV. This catches pages where a view was not migrated, a partial is missing, or a block editor change left a component silent instead of crashing.
Mode 3: Single URL Comparison does a deeper line-by-line and element-count analysis on one URL and writes a timestamped .txt report. Use this when Mode 2 flagged a page and you want to understand exactly what changed, it lists modified lines, added/removed content, and differences in <script>, <style>, and other element counts.
Everything lives in appsettings.json and can be overridden from the command line:
{
"Sitemap": {
"BaseUrl": "https://localhost:44368",
"SitemapPath": "/sitemap",
"ComparisonBaseUrl": "https://production.example.com",
"Concurrency": 8,
"MinSimilarityThreshold": 95.0,
"MaxRetries": 3,
"RetryDelayMs": 1000,
"ErrorOutputFile": "sitemap-errors.csv",
"ComparisonErrorOutputFile": "html-comparison-results.csv"
}
}
To run against a different host without editing the file:
dotnet run -- Sitemap:BaseUrl=https://staging.example.com Sitemap:Concurrency=4
Warning: Run the comparison with the upgraded site as BaseUrl and the known-good site as ComparisonBaseUrl, not the other way around. The diff labels ("more in original", "removed in comparison") only make sense with this orientation.
In practice, Mode 1 finds the hard failures, routes that no longer exist, controllers that throw, template files that did not get moved.
Mode 2 finds the subtle ones: a block grid component that renders nothing instead of throwing, a partial view that was never migrated so a section of the page is just gone, or a property editor whose output format changed enough to shift the page size by more than 5%.
A page that silently renders 30% less HTML than production is harder to find than a 500. This is exactly the kind of regression that slips through a manual upgrade review.
I published the tool on GitHub so you can inspect the implementation, adjust it for your own upgrade flow, or use it as a starting point:
An Umbraco upgrade is not finished when the site starts. It is finished when you can prove that the important pages still render correctly.
A sitemap-based checker will not replace proper functional testing, but it gives you a fast and repeatable safety net. It helps catch the obvious failures first, then points you to the silent rendering differences that are easy to miss during a manual review.