Fixing Missing Product URLs After a Magento 2 CSV Import (Visibility Changed, No url_key Column)

Here is a Magento 2 failure that looks like a URL rewrite bug but is really a CSV import gotcha. A store runs a routine product import that flips a few thousand SKUs from Not Visible Individually to Catalog, Search. The products start appearing in category listings and search results, as intended. But instead of the clean URL you expect, every one of those newly visible products links to a raw path like this:

https://www.example.com/catalog/product/view/id/354692/s/left-rear-wheel-arch-repair-panel/category/9923/

Meanwhile the URL Key field in the admin looks perfectly correct for every affected product. Products that were already visible before the import still have their normal /left-rear-wheel-arch-repair-panel.html URLs. Nothing is in the logs, no reindex is pending, and flushing cache changes nothing. The impact is real: thousands of ugly, non-canonical URLs on live category pages and in search results, and no SEO-friendly landing URL for products you just decided to start selling individually.

This is stock Magento behavior, not a data-entry mistake and not a third-party module. We hit it on a client store last week with 1,617 affected SKUs out of a 4,202-row import. Below is why it happens, how to confirm which products are affected, and three ways to fix it depending on how many products you are dealing with.

Why Magento 2 does this

Two separate pieces of core behavior combine to produce the problem.

First: Magento never creates url_rewrite rows for a product whose visibility is Not Visible Individually. That is deliberate. A product that cannot be viewed on its own page does not need a URL, so when the product is created the url_rewrite table simply has no entry for it. The product still has a url_key attribute value, which is why the admin field looks fine. The URL key and the URL rewrite are two different things stored in two different tables.

Second: when you later change that product’s visibility through the admin product edit form, Magento notices. The save observer Magento\CatalogUrlRewrite\Observer\ProductProcessUrlRewriteSavingObserver explicitly regenerates rewrites when visibility changes, and newer releases also check for missing rewrites on every save. So a hand edit in the admin works.

The CSV importer does not use that observer. It uses a separate one, Magento\CatalogUrlRewrite\Observer\AfterImportDataObserver, and that observer decides per row whether it is even worth generating URLs. The decision method in 2.4 reads like this:

// Magento\CatalogUrlRewrite\Observer\AfterImportDataObserver
private function isNeedToPopulateForUrlGeneration($rowData, $newSku): bool
{
    if ((
        (empty($newSku) || !isset($newSku['entity_id']))
        || ($this->import->getRowScope($rowData) == ImportProduct::SCOPE_STORE
            && empty($rowData[self::URL_KEY_ATTRIBUTE_CODE]))
        || ($this->skuStorage->has($rowData[ImportProduct::COL_SKU] ?? '')
            && !isset($rowData[self::URL_KEY_ATTRIBUTE_CODE])
            && $this->import->getBehavior() === ImportExport::BEHAVIOR_APPEND)
    )
    && !isset($rowData["categories"])
    ) {
        return false;
    }
    return true;
}

Read the third condition carefully. If the SKU already exists, the import behavior is Add/Update (which is append internally), and the row has no url_key column, the observer returns false and skips URL generation for that row entirely. The only escape hatch is a categories column. The observer never looks at the visibility column at all.

So an import file with the header sku,name,description,visibility is the worst case: it is exactly the shape that changes visibility while giving the URL observer no reason to run. Put the two behaviors together and you get a product that was legitimately URL-less yesterday, is visible today, and has nobody responsible for building its URL. The frontend falls back to the internal catalog/product/view/id/... route, which is what you are seeing.

• Products that were already visible are unaffected. Their rewrites existed before the import and nothing deleted them.
• Products that were Not Visible Individually and became visible are all broken, every one of them.
• Reindexing does not help. URL rewrites are not an indexer in Magento 2. The rows have to be written by the generator.
• The admin URL Key field is a red herring. It is populated and correct. The missing data is in url_rewrite, not in the EAV attribute.

1. Confirm which products are affected

This query lists enabled, visible products on a given store view that have no system-generated product rewrite. Set the store ID to the storefront that is showing the bad URLs. The metadata IS NULL filter restricts the join to the plain product URL rather than category-path variants, and redirect_type = 0 excludes 301/302 history rows.

SET @store_id = 1;

SELECT e.sku, uk.value AS url_key, vis.value AS visibility
FROM catalog_product_entity e
JOIN eav_attribute av  ON av.attribute_code = 'visibility' AND av.entity_type_id = 4
JOIN catalog_product_entity_int vis
     ON vis.entity_id = e.entity_id AND vis.attribute_id = av.attribute_id AND vis.store_id = 0
JOIN eav_attribute ast ON ast.attribute_code = 'status' AND ast.entity_type_id = 4
JOIN catalog_product_entity_int st
     ON st.entity_id = e.entity_id AND st.attribute_id = ast.attribute_id AND st.store_id = 0
JOIN eav_attribute auk ON auk.attribute_code = 'url_key' AND auk.entity_type_id = 4
LEFT JOIN catalog_product_entity_varchar uk
     ON uk.entity_id = e.entity_id AND uk.attribute_id = auk.attribute_id AND uk.store_id = 0
LEFT JOIN url_rewrite u
     ON u.entity_type = 'product' AND u.entity_id = e.entity_id
    AND u.store_id = @store_id AND u.redirect_type = 0 AND u.metadata IS NULL
WHERE vis.value IN (2, 3, 4)      -- Catalog / Search / Catalog, Search
  AND st.value = 1                -- Enabled
  AND u.url_rewrite_id IS NULL
ORDER BY e.sku;

On Adobe Commerce (the paid edition with content staging) the EAV value tables key on row_id instead of entity_id, and you need to pick the currently live staging row. Join catalog_product_entity_int and catalog_product_entity_varchar on e.row_id, and add e.created_in <= UNIX_TIMESTAMP() AND e.updated_in > UNIX_TIMESTAMP() to the WHERE clause. The url_rewrite.entity_id join stays on e.entity_id.

Export the result to a file. You will want the SKU list for whichever fix you choose, and it is your before-and-after proof.

2. The quick fix for a handful of products: save them in the admin

If the query returns a dozen rows, open each product in the admin and click Save without changing anything. The save observer sees the rewrites are missing and generates them. This is slow and manual, but it is zero-risk and needs no deploy. Beyond a couple dozen products, move on to one of the next two options.

3. The no-code fix: re-import the same file with a url_key column

This is the fix that makes people say "but we didn’t change the URLs." You are not changing them. You are adding the column so the import observer agrees to do its job. Take the exact file you already imported and add one column, url_key, filled with each product’s current URL key:

sku,name,description,visibility,url_key
905-373,"Left Rear Wheel Arch Repair Panel","...","Catalog, Search",905-373-left-rear-wheel-arch-repair-panel

Three rules that matter here:

• The column must exist. That is the whole trick. The observer checks isset($rowData['url_key']), so a missing column skips URL generation and a present column enables it.
• Fill it with the existing value. Pull the current keys from a product export (System > Export, entity type Products) or from catalog_product_entity_varchar, and paste them in. If you leave the cell empty, Magento falls back to generating a key from the product name, which may differ from what the admin shows today and will silently create a different URL.
• Do not change the value. The importer does not create 301 redirects when a URL key changes. A typo in that column on an already visible product replaces its working URL with a new one and leaves the old one dead.

Alternatively, a categories column also unlocks the observer, since that is the other condition in the method above. But a categories column rewrites product-to-category assignments on import, so unless the file already carries correct category paths for every row, url_key is the safer column to add.

Run the import with the usual Add/Update behavior. When it finishes, re-run the query from step 1. It should return zero rows.

4. The scripted fix: regenerate rewrites for the affected SKUs

When the import file is gone, or the client is not comfortable re-importing, or the count is large enough that you want a repeatable fix with a log, rebuild the rewrites directly with Magento’s own generator. This script bootstraps the application, loads each SKU in the global scope, generates the full rewrite set for every store view the product is assigned to, and persists it. Run it from the Magento root as the web user.

<?php
// regenerate-product-urls.php
// Usage: php regenerate-product-urls.php affected-skus.txt
use Magento\Catalog\Api\ProductRepositoryInterface;
use Magento\Catalog\Model\Product\Visibility;
use Magento\CatalogUrlRewrite\Model\ProductUrlRewriteGenerator;
use Magento\Framework\App\Area;
use Magento\Framework\App\Bootstrap;
use Magento\Framework\App\State;
use Magento\UrlRewrite\Model\Exception\UrlAlreadyExistsException;
use Magento\UrlRewrite\Model\UrlPersistInterface;

require __DIR__ . '/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);
$om = $bootstrap->getObjectManager();
$om->get(State::class)->setAreaCode(Area::AREA_ADMINHTML);

$skus = array_filter(array_map('trim', file($argv[1] ?? 'affected-skus.txt')));

$repository = $om->get(ProductRepositoryInterface::class);
$generator  = $om->get(ProductUrlRewriteGenerator::class);
$persist    = $om->get(UrlPersistInterface::class);

foreach ($skus as $sku) {
    try {
        // Store 0 = global scope: the generator fans out to every store view the product is in.
        $product = $repository->get($sku, false, 0, true);

        if ((int)$product->getVisibility() === Visibility::VISIBILITY_NOT_VISIBLE) {
            echo "SKIP {$sku}: not visible individually\n";
            continue;
        }

        $rewrites = $generator->generate($product);
        $persist->replace($rewrites);
        echo "OK   {$sku}: " . count($rewrites) . " rewrite(s)\n";
    } catch (UrlAlreadyExistsException $e) {
        echo "DUP  {$sku}: " . $e->getMessage() . "\n";
    } catch (\Throwable $e) {
        echo "ERR  {$sku}: " . $e->getMessage() . "\n";
    }
}

A few notes on running it safely:

• UrlPersistInterface::replace() deletes the existing rewrites for that product and store before inserting the new set, so the script is idempotent. Running it twice is harmless.
• Feed it only the SKUs from the step 1 query. Those products have no rewrites to lose, so there is no risk of changing a working URL. If you run it against the whole catalog, products with hand-edited request paths will be reset to the generated ones.
• Any DUP line means another entity already owns that request path, usually a second product with the same URL key or a leftover redirect row in url_rewrite. Fix those individually.
• Take a backup of url_rewrite first. It is cheap and this table is painful to reconstruct otherwise.
• Flush the full_page and block_html caches afterward so category pages stop rendering the fallback URLs.

There are also community modules that wrap the same generator in a bin/magento command, such as elgentos/regenerate-catalog-urls. They work, but most of them default to the whole catalog, so read the options before pointing one at production.

5. Prevent it on the next import

Add this to the import checklist for anyone who touches product CSVs on the store:

• Any import that changes visibility must also carry the url_key column, populated with the product’s current value. That one line prevents this entire article.
• Start from a product export rather than a hand-built sheet. Exports include url_key by default, and you can delete the columns you do not need instead of forgetting the ones you do.
• After any bulk visibility change, run the step 1 query. It takes seconds and tells you immediately whether the import left URL-less products behind.

The underlying logic is sound from Magento’s point of view: an import that does not mention URLs should not touch URLs. The blind spot is that a visibility change implies a URL change for products that never had one, and the import observer was never taught to see it. Until that changes upstream, the url_key column is the fix.