baldwin-agency / magento2-module-url-data-integrity-checker Goto Github PK
View Code? Open in Web Editor NEWMagento 2 module which can find potential url related problems in your catalog data
License: MIT License
Magento 2 module which can find potential url related problems in your catalog data
License: MIT License
After the process is finished and the percentage is 100%, the console printed "Killed" and the backend shows the message "We are already refreshing the product url key's, just have a little patience". But nothing happens and when I try to run the process again from the console, an exception is displayed with the same message. "An unexpected exception occured: 'We are already refreshing the product url key's, just have a little patience"
I hope they can help me. Thank you very much.
Just ran into an issue on a shop where the url_key for the category exists on a certain storeview, but the url_path is missing:
mysql> SELECT * FROM catalog_category_entity_varchar WHERE entity_id = 6677 ORDER BY attribute_id;
+----------+--------------+----------+-----------+------------------------------+
| value_id | attribute_id | store_id | entity_id | value |
+----------+--------------+----------+-----------+------------------------------+
| 33040 | 45 | 0 | 6677 | Pillendoosjes |
| 37132 | 45 | 1 | 6677 | Pillendoosjes |
| 35086 | 45 | 2 | 6677 | Piluliers |
| 32017 | 52 | 0 | 6677 | PRODUCTS |
| 34063 | 117 | 0 | 6677 | pillendoosjes |
| 38155 | 117 | 1 | 6677 | pillendoosjes |
| 36109 | 117 | 2 | 6677 | piluliers |
| 43785 | 118 | 0 | 6677 | thuiszorg-ehbo/pillendoosjes |
| 39594 | 118 | 1 | 6677 | thuiszorg-ehbo/pillendoosjes |
+----------+--------------+----------+-----------+------------------------------+
9 rows in set (0.00 sec)
(attribute id 117 is url_key and 118 is url_path, the url_path is missing on storeview with id 2)
We should try to detect and report this.
What does the message "Product has a non-null url_path attribute, this is known to cause problems with url rewrites in Magento. It's advised to remove this value from the database." mean? How to fix it?
I can't find any problems.
Also new created products are listed when running catalog:product:integrity:urlpath.
I have this on different Magento installations.
This is more a question than an issue so sorry if this is the wrong place to ask such questions.
As far as I know url_path is now deprecated in newer versions of Magento 2. My question here is would it make sense to only flag these errors on older version of Magento or if you are not supporting older versions could it simply be removed?
Hey @hostep!
I see that you already created a branch to support php 8.2 :). When do you expect you will supply a stable tag?
/baldwin_urldataintegritychecker/catalog_category_urlPath/
:
Call to undefined method Baldwin\UrlDataIntegrityChecker\Model\ResourceModel\Catalog\Category\UrlPathCollection\Interceptor::getMainTable()
Exception in vendor/swissup/module-checkout-fields/Plugin/Ui/Component/OrderListingData.php:42
/baldwin_urldataintegritychecker/catalog_category_urlKey/
:
Call to undefined method Baldwin\UrlDataIntegrityChecker\Model\ResourceModel\Catalog\Category\UrlKeyCollection\Interceptor::getMainTable()
Exception in vendor/swissup/module-checkout-fields/Plugin/Ui/Component/OrderListingData.php:42
/baldwin_urldataintegritychecker/catalog_product_urlPath/
:
Call to undefined method Baldwin\UrlDataIntegrityChecker\Model\ResourceModel\Catalog\Product\UrlPathCollection\Interceptor::getMainTable()
Exception in vendor/swissup/module-checkout-fields/Plugin/Ui/Component/OrderListingData.php:42
/baldwin_urldataintegritychecker/catalog_product_urlKey/
:
Call to undefined method Baldwin\UrlDataIntegrityChecker\Model\ResourceModel\Catalog\Product\UrlKeyCollection\Interceptor::getMainTable()
Exception in vendor/swissup/module-checkout-fields/Plugin/Ui/Component/OrderListingData.php:42
Is this something that can be fixed here or do I need to address conflicting module?
It's https://www.firecheckout.net/ by https://swissuplabs.com/
Hello,
I have installed it yesterday in am magento 2.3.5p1 installation via composer.
CLI ist working.
But backend not.
I am getting this error:
Call to undefined method Baldwin\UrlDataIntegrityChecker\Model\ResourceModel\Catalog\Product\UrlKeyCollection::getMainTable()
kind regards
When I run bin/magento catalog:product:integrity:urlkey
these are the results;
+------------+-----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Product ID | SKU | Store ID | Problem |
+------------+-----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 300 | KG101 | 1 | Product has a duplicated url_key value (playgrass). It's the same as another product (ID: 346, Store: 1). Please fix because this will cause problems. |
| 300 | KG101 | 14 | Product has a duplicated url_key value (playgrass). It's the same as another product (ID: 346, Store: 14). Please fix because this will cause problems. |
| 346 | DE115 | 1 | Product has a duplicated url_key value (playgrass). It's the same as another product (ID: 300, Store: 1). Please fix because this will cause problems. |
| 346 | DE115 | 14 | Product has a duplicated url_key value (playgrass). It's the same as another product (ID: 300, Store: 14). Please fix because this will cause problems. |
+------------+-----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+
When I check my database using this query, I get the following results;
SELECT t.* FROM catalog_product_entity_varchar t WHERE attribute_id = (SELECT attribute_id FROM eav_attribute WHERE attribute_code = 'url_key' AND entity_type_id = 4) and entity_id IN (300,346)
value_id | attribute_id | store_id | entity_id | value |
---|---|---|---|---|
19165 | 86 | 0 | 300 | playgrass-KG101 |
3761705 | 86 | 1 | 300 | playgrass |
3761706 | 86 | 14 | 300 | playgrass |
26872 | 86 | 0 | 346 | playgrass-DE115 |
3761707 | 86 | 1 | 346 | playgrass |
3640255 | 86 | 14 | 346 | playgrass |
I then remove all url_rewrites for these products and regenerate them using our regenerate catalog URL extension;
bin/magento regenerate:product:url -s all 300 346
These are the results;
url_rewrite_id | entity_type | entity_id | request_path | target_path | redirect_type | store_id | description | is_autogenerated | metadata |
---|---|---|---|---|---|---|---|---|---|
2 | product | 300 | playgrass | catalog/product/view/id/300 | 0 | 1 | NULL | 1 | NULL |
3 | product | 346 | playgrass | catalog/product/view/id/346 | 0 | 14 | NULL | 1 | NULL |
Which is perfectly fine, no URL collissions! Therefore I believe this extension returns false positives.
We should add some checks to see if url_key's in the database are normalized or not (contain unexpected characters). If they aren't normalized these can cause bad things.
Apparently there was a bug introduced in Magento 2.3.3 (which has been fixed in 2.4.0) which allowed this problem to happen.
So it would be nice if this tool could detect such problems.
The bug only existed for product url_keys, but maybe we should also check the categories, you never know ...
@hostep could you create a new release? There have been 20 commits since 1.0.0 :)
Might be a bug in Magento but on a project, I noticed some URL rewrites still existed for disabled products pointing to categories that had been removed. (Magento is 2.2.7 so update is needed anyway, only not clear if the URLs will get cleaned up after the upgrade)
Hi we got an issue as we use ctrl+c by mistake while checker was running in CLI
How to clear that?
An unexpected exception occured: 'We are already refreshing the product url key's, just have a little patience ๐'
#0 /var/www/vhosts/xxxx/httpdocs/vendor/baldwin/magento2-module-url-data-integrity-checker/Console/Command/CheckProductUrlKeys.php(69): Baldwin\UrlDataIntegrityChecker\Updater\Catalog\Product\UrlKey->refresh()
#1 /var/www/vhosts/xxxx/httpdocs/vendor/symfony/console/Command/Command.php(255): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckProductUrlKeys->execute()
#2 /var/www/vhosts/xxxxx/httpdocs/vendor/magento/framework/Interception/Interceptor.php(58): Symfony\Component\Console\Command\Command->run()
#3 /var/www/vhosts/xxxx/httpdocs/vendor/magento/framework/Interception/Interceptor.php(138): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckProductUrlKeys\Interceptor->___callParent()
#4 /var/www/vhosts/xxxx/httpdocs/vendor/magento/framework/Interception/Interceptor.php(153): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckProductUrlKeys\Interceptor->Magento\Framework\Interception{closure}()
#5 /var/www/vhosts/xxxx/httpdocs/generated/code/Baldwin/UrlDataIntegrityChecker/Console/Command/CheckProductUrlKeys/Interceptor.php(23): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckProductUrlKeys\Interceptor->___callPlugins()
#6 /var/www/vhosts/xxxxx/httpdocs/vendor/symfony/console/Application.php(1009): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckProductUrlKeys\Interceptor->run()
#7 /var/www/vhosts/xxxxx/httpdocs/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand()
#8 /var/www/vhosts/xxxxx/httpdocs/vendor/magento/framework/Console/Cli.php(115): Symfony\Component\Console\Application->doRun()
#9 /var/www/vhosts/xxxx/httpdocs/vendor/symfony/console/Application.php(149): Magento\Framework\Console\Cli->doRun()
#10 /var/www/vhosts/xxxxx/httpdocs/bin/magento(23): Symfony\Component\Console\Application->run()
#11 {main}
Seems that a task is stuck.
Regards
Joerg
Wether using the cli or the admin menu entry does not work on Magento 2.4.1 CE
There are no commands defined in the "catalog:category:integrity" namespace. Did you mean this? wyomind:dynamiccategory
In the magento backend you're redirected to the dashboard.
Invalid security or form key. Please refresh the page.
Hitting the memory limit when trying to open product_url key report
The file size of the json file is 96mb and the memory limit is 1gb.
Anybody who can clarify why it was decided to store these reports in the filesystem instead of the database?
Using a dataset of 422 problems, the grids in the backend of Magento 2.4.0 only show 20 records and the pager doesn't indicate additional pages are available:
Increasing the page size or sorting columns still work.
This needs some investigation, we need to figure out what goes wrong here ...
Tested with commit 2aa2dd6
Magento Version: 2.2.1
Reproduce
Run any of the CLI commands provided in the README.md of the module.
Output in text format
application@f38ef7308717:/app$ bin/magento catalog:category:integrity:urlpath An unexpected exception occured: 'Area code is already set' #0 /app/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Framework\App\State->setAreaCode('crontab') #1 /app/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Framework\App\State\Interceptor->___callParent('setAreaCode', Array) #2 /app/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Framework\App\State\Interceptor->Magento\Framework\Interception\{closure}('crontab') #3 /app/generated/code/Magento/Framework/App/State/Interceptor.php(52): Magento\Framework\App\State\Interceptor->___callPlugins('setAreaCode', Array, Array) #4 /app/app/code/Baldwin/UrlDataIntegrityChecker/Console/Command/CheckCategoryUrlPaths.php(57): Magento\Framework\App\State\Interceptor->setAreaCode('crontab') #5 /app/vendor/symfony/console/Command/Command.php(245): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlPaths->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #6 /app/vendor/symfony/console/Application.php(835): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #7 /app/vendor/symfony/console/Application.php(185): Symfony\Component\Console\Application->doRunCommand(Object(Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlPaths), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #8 /app/vendor/magento/framework/Console/Cli.php(102): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #9 /app/vendor/symfony/console/Application.php(117): Magento\Framework\Console\Cli->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #10 /app/bin/magento(23): Symfony\Component\Console\Application->run() #11 {main}
Expected result
List of URL path / URL key issues.
Note
Admin panel version of the module works.
Hello,
on /baldwin_urldataintegritychecker/catalog_category_urlPath/
I'm getting the following error:
Call to undefined method Baldwin\UrlDataIntegrityChecker\Model\ResourceModel\Catalog\Category\UrlPathCollection\Interceptor::getSelect()
Exception in vendor/magento/framework/View/Element/UiComponent/DataProvider/FilterPool.php:45
All other grids are throwing the same exception.
Using version 1.2.0 of this module and Magento version 2.4.2 - maybe it is incompatible?
On first try of this command ( catalog:category:integrity:urlkey
)
I got this error : Can't find category with id
And if try to launch again : We are already refreshing the category url key's, just have a little patience ๐
An unexpected exception occured: 'Can't find category with id: '36764''
#0 /var/www/html/vendor/baldwin/magento2-module-url-data-integrity-checker/Checker/Catalog/Category/UrlPath.php(117): Baldwin\UrlDataIntegrityChecker\Checker\Catalog\Category\UrlPath->fetchAllCategoriesWithUrlPathCalculatedByUrlKey()
#1 /var/www/html/vendor/baldwin/magento2-module-url-data-integrity-checker/Checker/Catalog/Category/UrlKey/DuplicateUrlKey.php(87): Baldwin\UrlDataIntegrityChecker\Checker\Catalog\Category\UrlPath->getCalculatedUrlPathForCategory(Object(Magento\Catalog\Model\Category\Interceptor), 1)
#2 /var/www/html/vendor/baldwin/magento2-module-url-data-integrity-checker/Checker/Catalog/Category/UrlKey/DuplicateUrlKey.php(48): Baldwin\UrlDataIntegrityChecker\Checker\Catalog\Category\UrlKey\DuplicateUrlKey->getCategoryUrlPathsByStoreId(1)
#3 /var/www/html/vendor/baldwin/magento2-module-url-data-integrity-checker/Checker/Catalog/Category/UrlKey/DuplicateUrlKey.php(34): Baldwin\UrlDataIntegrityChecker\Checker\Catalog\Category\UrlKey\DuplicateUrlKey->checkForDuplicatedUrlKeyAttributeValues()
#4 /var/www/html/vendor/baldwin/magento2-module-url-data-integrity-checker/Checker/Catalog/Category/UrlKey.php(32): Baldwin\UrlDataIntegrityChecker\Checker\Catalog\Category\UrlKey\DuplicateUrlKey->execute()
#5 /var/www/html/vendor/baldwin/magento2-module-url-data-integrity-checker/Updater/Catalog/Category/UrlKey.php(47): Baldwin\UrlDataIntegrityChecker\Checker\Catalog\Category\UrlKey->execute()
#6 /var/www/html/vendor/baldwin/magento2-module-url-data-integrity-checker/Console/Command/CheckCategoryUrlKeys.php(64): Baldwin\UrlDataIntegrityChecker\Updater\Catalog\Category\UrlKey->refresh('CLI')
#7 /var/www/html/vendor/symfony/console/Command/Command.php(255): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/html/vendor/magento/framework/Interception/Interceptor.php(58): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/html/vendor/magento/framework/Interception/Interceptor.php(138): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor->___callParent('run', Array)
#10 /var/www/html/vendor/magento/framework/Interception/Interceptor.php(153): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor->Magento\Framework\Interception\{closure}(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /var/www/html/generated/code/Baldwin/UrlDataIntegrityChecker/Console/Command/CheckCategoryUrlKeys/Interceptor.php(23): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor->___callPlugins('run', Array, Array)
#12 /var/www/html/vendor/symfony/console/Application.php(1009): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /var/www/html/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand(Object(Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/html/vendor/magento/framework/Console/Cli.php(115): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/html/vendor/symfony/console/Application.php(149): Magento\Framework\Console\Cli->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /var/www/html/bin/magento(23): Symfony\Component\Console\Application->run()
#17 {main}
root@prod-m2-fo-6959cb5f58-p6bvb:/var/www/html# ./magento catalog:category:integrity:urlkey
An unexpected exception occured: 'We are already refreshing the category url key's, just have a little patience ๐'
#0 /var/www/html/vendor/baldwin/magento2-module-url-data-integrity-checker/Console/Command/CheckCategoryUrlKeys.php(64): Baldwin\UrlDataIntegrityChecker\Updater\Catalog\Category\UrlKey->refresh('CLI')
#1 /var/www/html/vendor/symfony/console/Command/Command.php(255): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#2 /var/www/html/vendor/magento/framework/Interception/Interceptor.php(58): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#3 /var/www/html/vendor/magento/framework/Interception/Interceptor.php(138): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor->___callParent('run', Array)
#4 /var/www/html/vendor/magento/framework/Interception/Interceptor.php(153): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor->Magento\Framework\Interception\{closure}(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/html/generated/code/Baldwin/UrlDataIntegrityChecker/Console/Command/CheckCategoryUrlKeys/Interceptor.php(23): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor->___callPlugins('run', Array, Array)
#6 /var/www/html/vendor/symfony/console/Application.php(1009): Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/html/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand(Object(Baldwin\UrlDataIntegrityChecker\Console\Command\CheckCategoryUrlKeys\Interceptor), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/html/vendor/magento/framework/Console/Cli.php(115): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/html/vendor/symfony/console/Application.php(149): Magento\Framework\Console\Cli->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#10 /var/www/html/bin/magento(23): Symfony\Component\Console\Application->run()
#11 {main}
First, thanks a lot for this useful tool!
Shouldn't we ignore products, which are set to "Not Visible Individually" in the catalog:product:integrity:urlkey
command? The reason is that these products are ignored when catalog URL rewrites are generated, see \Magento\CatalogUrlRewrite\Model\ProductUrlRewriteGenerator::generate
. Hence, I think we need to add ->addAttributeToFilter('visibility', ['neq' => Visibility::VISIBILITY_NOT_VISIBLE])
here:
Does that make sense to you?
Hi,
in M 2.3.5-p2
Catalog > Data Integrity Checker > Product Url Key
Call to undefined method Baldwin\UrlDataIntegrityChecker\Model\ResourceModel\Catalog\Product\UrlKeyCollection::getMainTable()
Data Integrity - Product Url Path
Call to undefined method Baldwin\UrlDataIntegrityChecker\Model\ResourceModel\Catalog\Product\UrlPathCollection::getMainTable()
Thank you
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.