Monitoring New 404 URLs After a Content Management System Migration

Quick Answer

Monitoring New 404 URLs After a Content Management System Migration requires comparing post-launch 404 reports with the old URL inventory and approved redirect map. Check each suspicious URL's current HTTP response and intended destination. Prioritize unexpected failures of important pages, while keeping legitimate 404 responses for content that was intentionally removed.

Key Takeaways

A newly reported 404 does not always mean the CMS migration broke a page. Teams need evidence of what existed before launch, what was meant to change, and what the server returns now.

• Preserve the old URL inventory, planned destinations, expected status codes, and prelaunch test results.
• Separate newly observed 404 URLs from failures proven to result from the migration.
• Combine server logs, CDN logs, targeted crawls, and Google Search Console findings for better coverage.
• Track unique affected URLs separately from request counts, since repeated requests can distort trends.
• Repair broken legacy mappings and redirect targets, but accept valid 404 responses for retired content.
• Give the highest priority to failed revenue pages, key navigation paths, and important migrated content.
• Assign actionable errors to named owners and close them only after recording successful live tests.

The aim is not zero 404 responses. It is zero unresolved failures against the agreed migration plan, within the scope the team has checked.

Monitoring New 404 URLs After a Content Management System Migration: Start With a Baseline

The most useful starting point is a saved record of how each important URL worked before deployment. That record should also state how the URL was meant to work after the move. Without both facts, a new 404 report may lead to guesswork.

A baseline serves two jobs. It helps detect changes across the release, and it gives developers a clear result to restore. Keep its capture time, source, and environment with the data. A test made on staging is not the same as a test against the old production site.

Preserve legacy URLs and planned destinations

Export URLs from the old CMS before launch. Add the old XML sitemaps, approved redirect map, and a crawl of important pages. Include old URLs with external links or past organic search visits where those records are available.

Keep the full original address for each URL. A path alone may hide a host change, locale, query string, or protocol difference. Old sitemaps also miss some pages, including orphaned URLs that have no current internal links.

Google's site move guidance supports mapping old URLs to their new locations and checking traffic and crawl results after the move.

A useful baseline record states the old URL, planned new URL, expected response, observed prelaunch status, source file, and capture timestamp. Store the release identifier as well. That makes it easier to tie a later failure to the correct deployment.

Capture the expected behavior before launch

Test a representative set of old URLs before the release, with full coverage of the most important mappings where feasible. Save the status code and final destination, not just a pass or fail label.

Consider a legacy category page that currently returns HTTP 200. The migration plan says it should return HTTP 301 to a new category URL. Those two expected states belong in the record.

A retired campaign page is different. If its content has no relevant replacement, the approved outcome may be HTTP 404 or 410. Google's Page indexing documentation distinguishes missing pages from other indexing issues and provides the basis for reviewing those responses.

Where old records are incomplete, label the baseline as partial. You can still find broken URLs, but you should not claim the migration caused every newly discovered error.

Find Newly Observed 404s Across the Evidence Sources

Use several evidence streams instead of relying on one dashboard. Server or CDN logs show requests and responses captured by that system. Targeted crawls test known URLs and discover broken links on pages they can reach. Search Console adds Google's view of affected pages.

Each source answers a different question. Store observations separately before combining them. That preserves the source time, test scope, and response details that may explain conflicting results.

Compare request logs and current crawler results

Extract HTTP 404 responses from the periods before and after deployment. Record the requested address, response code, timestamp, and request source. Include the referrer when available.

Check whether the CDN and origin server record the same responses. A cached error page may be served at the edge without reaching the origin. Log retention, bot filtering, and routing rules can also change what each system sees.

Next, run a targeted crawl of the old URL list and approved redirect map. Also test current internal links. Keep the referring page for each broken link, since it often tells developers where the repair belongs.

For the detailed process of finding broken links and their source pages, see how to crawl a website for broken links.

A crawl is a current test of reachable or supplied URLs. It is not proof that every historical address has been checked. Old product links, external backlinks, and removed sitemap entries may escape a normal crawl.

Use Search Console without treating it as a complete error log

Google Search Console can reveal missing pages found during Google's crawls. Yet the Page indexing report is not an exhaustive list of broken URLs. Its example lists are limited to 1,000 URLs and may omit examples even below that limit.

Compare the report's observation dates with live tests. A URL may still appear in a report after the server issue has been fixed. The reported state and the current HTTP state describe different points in time.

The Search Console Crawl Stats report groups Googlebot requests by response. Those figures count requests, not unique URLs. One missing URL requested many times may affect the chart more than several URLs requested once.

Likewise, Search Console bulk data export sends Search performance data to BigQuery. It does not provide a complete event-level log of every HTTP 404 request.

Keep a source label on every observation. A Search Console example, a CDN request, and a manual HTTP test should not be treated as three separate broken pages merely because they came from three tools.

Separate Newly Observed URLs From Confirmed Migration Failures

First calculate which distinct 404 URLs appeared in the current evidence but not in the comparable baseline. Then test those URLs against the migration plan. Only the second check can establish an unexpected change that needs repair.

This distinction matters when a monitoring tool sees an old broken URL for the first time. Its first appearance after launch does not prove that the new CMS broke it.

Compute the newly observed URL set

Define B as the set of distinct URLs observed returning HTTP 404 in the baseline interval. Define C as the equivalent set in the current interval. Use matching collection rules and a documented URL comparison method.

Newly observed 404 URL set = C − B.

Here is a hypothetical calculation. Assume the baseline set contains /old-sale and /retired-news. The current set contains /retired-news, /old-product, and /missing-image.

The newly observed set contains /old-product and /missing-image. The count is two distinct URLs. It is not two new migration defects, and it is not the total number of 404 requests.

If the baseline logs did not capture image requests, the comparison is weaker. /missing-image might have been broken before launch.

Keep both the raw URL and a comparison key. Do not automatically strip query parameters, change path case, merge hostnames, or remove locale prefixes. Those details can change routing. Percent-encoded characters and trailing slashes may also matter. Set normalization rules only after reviewing how the site treats each variation.

Validate the expected response against the URL mapping

For every newly observed candidate, retrieve the current live response. Follow redirects and record the final target and status. Then compare the outcome with the approved migration map.

Suppose /old-product should redirect to /products/widget-a, but it returns HTTP 404. The mapping provides a reason to investigate a deployment regression.

The same response from /retired-news may be correct if the page was meant to disappear. An unknown path requested by a bot needs further history checks before it receives a migration defect label.

If an old address returns HTTP 301 but the destination returns HTTP 404, the first redirect does not prove success. The final resource is still missing.

Record an evidence confidence level alongside the classification. A prelaunch successful test, approved mapping, and failed live response offer stronger proof than a first-seen Search Console entry alone.

Use a Migration 404 Change Register to Assign the Right Fix

A migration 404 change register turns scattered error observations into a repair queue. Each record should show what was requested, what should happen, what actually happened, who owns the issue, and how the result will be checked.

The register must also hold valid missing-page outcomes. Keeping them visible avoids reopening accepted decisions each time a crawler reports the same URL.

The fields that make a 404 ticket reproducible

Use one record per affected URL and expected outcome, with linked evidence for repeated observations. A practical record holds the raw old URL, planned target, baseline status, first-seen time and timezone, current response, source reference, redirect result, classification, priority, owner, and ticket state.

Add the latest test time and closure proof. Optional fields include the referring page, sitemap membership, release ID, request volume, and relevant CMS route or redirect rule.

Separate the first-seen time from the suspected failure time. These can differ by days or longer. Record whether the cause is confirmed, likely, or still unknown.

Worked example: three post-migration URL outcomes

The table below is entirely hypothetical. All URLs use the reserved example.com domain. Dates, roles, responses, and findings illustrate how an engineering team could use the register. They are not results from a real migration.

The first record deserves a routing repair because the old page was meant to survive through a redirect. The second is an accepted outcome, not an unresolved migration defect. The third needs a full-chain repair, not merely a check of its first response.

Google's crawl error troubleshooting guidance helps distinguish genuine missing responses from other HTTP and soft-404 issues. A missing-content page served with HTTP 200 may still be treated as a soft 404. Record the actual server code and Google's reported classification separately.

For the broader choice between restoring content, redirecting it, or allowing a missing response, use the existing guide to Google crawl errors for pages that do not exist.

Do not send every old URL to the homepage. A redirect should lead to a relevant replacement. If the original content was removed without a suitable match, a genuine missing response may be the right result.

Prioritize Errors by Impact, Not by Raw 404 Volume

Fix unexpected failures on important pages first. A single missing revenue page may matter more than hundreds of requests to an unrelated, never-valid path. Request volume helps show exposure, but it cannot establish business value on its own.

Use page history, approved migration behavior, current internal references, and user importance to assign severity. The following priority model is an editorial operating framework. It is not an official Google severity scale.

This model separates urgency from the mere presence of HTTP 404. An accepted retired URL should not remain in the open defect count. A high-volume unknown URL may deserve investigation, but its priority should rise only when evidence shows real impact or an expected state was violated.

For a large migration, group failures by route pattern or CMS component. If many product paths fail under one rule, assign the shared cause rather than creating unrelated tickets for each request. Keep the affected URLs linked to that parent issue so the repair can be tested across the whole group.

If the problem extends to blocked crawlers, widespread server errors, or other response problems, use the site's crawlability issue diagnosis guide for the broader investigation.

Product migrations can also produce confusing results when thin variant pages return HTTP 200 but appear missing to Google. The separate product variant soft-404 investigation addresses that specialized case.

No universal safe 404 rate or repair deadline has been established for all CMS migrations. Let the release risk, page value, and evidence set the response order.

Verify Repairs and Keep the Monitoring Loop Open

Close a migration defect only when the live behavior matches the approved outcome and the test evidence is saved. A redirect rule added to a configuration file is not, by itself, proof that the user reaches a working page.

The monitoring loop should stay active through the release stabilization period and continue at a frequency suited to the site. Search Console can provide later signals, but it should not replace fresh HTTP tests.

Record proof that the failure is resolved

Retest the original address against production. Save the date, timezone, response code, redirect sequence, final URL, and final HTTP status. Where relevant, check that the destination serves the intended content rather than an unrelated fallback.

For intentionally retired pages, confirm the agreed missing response. Check for stale internal links and sitemap entries that should be removed. The desired closure result is not always HTTP 200.

A completed ticket should identify the tester, deployed fix, observed outcome, and evidence location. Keep a link to the prior failed test. That gives future reviewers proof of the before-and-after change.

Do not reopen a successfully tested fix solely because Search Console still lists the old error. Compare its last crawl with the current response. If the live failure returns later, reopen the ticket with fresh evidence.

Adapt monitoring frequency to deployment risk

A practical stabilization plan starts with baseline capture before launch. Test critical mappings during deployment. Review fresh log and crawl findings frequently during early stabilization, then move to periodic checks once failures are resolved and new reports have settled.

Daily review is one possible operating choice for a busy launch. It is not a Google-mandated schedule. Large sites, active release pipelines, and critical revenue paths may need more frequent checks. Smaller sites can choose a lighter schedule based on observed risk.

Keep the monitoring record tied to release dates. A later CMS update can reintroduce an old failure, even after the first migration passed its checks.

Google generally recommends keeping site-move redirects for at least one year, and longer where possible. That guidance concerns redirect retention. It does not set a fixed duration for daily 404 monitoring.

Common Monitoring Questions After a CMS Migration

Teams often disagree about what a fresh error report proves. The answer depends on the planned URL state, the age of the evidence, and which system recorded it. These questions help separate unresolved defects from valid outcomes and reporting delays.

Does every newly reported 404 need a redirect?

No. A retired page with no relevant replacement can correctly return HTTP 404 or 410. Add a relevant permanent redirect when the content moved to a new address. Check the approved content decision before editing the redirect map.

Why does Search Console show an error after a successful repair?

Google may have recorded the error before the fix. Compare the report's last crawl date with a current live HTTP test. Keep monitoring the affected page while Google processes newer crawl results.

Can Search Console identify every newly broken URL?

No. Its example reports do not cover every affected address. Use old CMS exports, redirect maps, server or CDN logs, and targeted crawls. Each adds coverage that a single report may miss.

How can we prove a 404 was caused by the migration?

Match a successful prelaunch response with the approved migration plan and a failed postlaunch test. Record the deployment time and relevant route change. If historical evidence is missing, use a lower-confidence classification until the cause is clearer.

Should we count requests or unique URLs?

Use unique URLs to measure the affected inventory. Use request counts to measure how often observed failures were requested. Keep both, but label them clearly. They have different denominators and answer different questions.

What if an old URL redirects to another 404?

Treat the final result as a failed route when the plan requires a working destination. Record each redirect hop, repair the target or mapping, and retest from the original address.

When should a migration 404 ticket be marked resolved?

Close the ticket after the intended live outcome is confirmed and recorded. Include the tester, timestamp, URL, response, final destination where relevant, and ticket reference. A dashboard entry disappearing is not enough on its own.

Turn Migration Error Reports Into Verified Outcomes

The strongest post-migration monitoring process compares three things: the old URL state, the approved new state, and the live state after deployment. It uses that comparison to distinguish real regressions from missing pages that are working as planned.

Start with the saved inventory and redirect map. Gather observations from logs, crawls, and Search Console. Preserve their timestamps and evidence sources. Then test suspicious URLs against the approved outcome before assigning a repair.

Use the change register as the shared record between SEO, engineering, QA, and content teams. It should show the affected URL, business impact, issue owner, expected fix, and closure proof. This creates a more useful record than a growing list of unexplained HTTP errors.

The process also needs a clear limit. A partial baseline cannot prove complete coverage. A Search Console report cannot prove that an error began on its first reporting date. A successful redirect cannot be assumed to work until its final destination is tested.

Keep monitoring after the first round of repairs. New releases, changed routing rules, and removed content can create fresh failures. A documented change register lets the team spot those changes and reopen only the issues that need action.

The goal is straightforward: preserve access to content that should remain available, accept legitimate removals, and keep proof that each required repair works. That is a more reliable outcome than trying to eliminate every 404 response.