elgentos / magento2-serversideanalytics

Server side analytics for Magento 2

Server Side Analytics for Magento 2

This extension aims to solve the problem of discrepancies between Magento revenue reports and the revenue reports in Google Analytics.

That problem arises due to the fact that a certain number of people close the browser window before returning to Magento's success page. Since Google Analytics is Javascript based, and thus client based, the GA Purchase Event will not be fired and the order will not be registered in Analytics.

Another reason why this problem arises is that people decide to pay at a later point in time through a different platform (like the PSP's), using a link in an email for example.

Installations

composer require elgentos/serversideanalytics2
bin/magento setup:upgrade

Sample query for GraphQL to set the GA data

mutation AddGaUserId($cartId: String!, $gaUserId: String, $gaSessionId: String) {
                            AddGaUserId(input: {
                                    cartId: $cartId
                                    gaUserId: $gaUserId
                                    gaSessionId: $gaSessionId
                                }
                            ),
                            {
                               cartId
                               maskedId
                            }
                    }

Caveats

• This extension disables the JS Purchase Event on the success page altogether. It will however track the pageview.
• This extension only tracks paid orders (it fires on sales_order_payment_pay). Non-paid orders will never show up in Analytics. This is our current clients' use case, mileage may differ. PR's for code to also track non-paid orders are welcomed.

Further info

• Compatible with UA Measurement Protocol and GA4 Measurement Protocol;
• Debugging is enabled when Magento is in developer mode. See var/log/system.log for the log;
• Exceptions will be logged to var/log/exceptions.log;
• The products in the payload are retrieve on invoice-basis, not on order-basis;
• An event has been added for you to add or overwrite custom fields to products in the purchase event; elgentos_serversideanalytics_product_item_transport_object;
• An event has been added for you to add or overwrite custom fields to transaction data in the purchase event; elgentos_serversideanalytics_transaction_data_transport_object;
• An event has been added for you to add or overwrite fields to tracking data in the purchase event; elgentos_serversideanalytics_tracking_data_transport_object;
• Testing can be done by dispatching test_event_for_serversideanalytics with a $payment (\Magento\Sales\Order\Payment) object in the payload;
• magerun2 dev:events:fire --eventName sales_order_payment_pay --parameters "payment::\Magento\Sales\Model\Order\Payment:X;invoice::\Magento\Sales\Model\Order\Invoice:X";

magento2-serversideanalytics's Issues

Some transactions are recorded twice

Not sure how to reproduce (yet), but some transactions are logged twice in Google Analytics. Many transactions also appear twice in the system.log with a few seconds apart, but most of them are shown only once in Google Analytics. So there might be some filtering/deduplication happening on the Google side.

[2021-02-04 09:31:35] main.INFO: elgentos_serversideanalytics_requests:  ["https://ssl.google-analytics.com/collect?ti=16000000118&tr=175&tt=30.37&ts=0.0000&ta=B2C&v=1&tid=UA-XXX-2&uip=92.xx.xx.xx&cid=xxx.xxx&dp=%2Fcheckout%2Fonepage%2Fsuccess%2F&pa=purchase&ec=Checkout&ea=Purchase&t=event&pr1id=sku.42&pr1nm=Product&pr1pr=175.0000&pr1qt=1.0000"] []
[2021-02-04 09:31:36] main.INFO: elgentos_serversideanalytics_requests:  ["https://ssl.google-analytics.com/collect?ti=16000000118&tr=175.0000&tt=30.3700&ts=0.0000&ta=B2C&v=1&tid=UA-XXX-2&uip=92.xx.xx.xx&cid=xxx.xxx&dp=%2Fcheckout%2Fonepage%2Fsuccess%2F&pa=purchase&ec=Checkout&ea=Purchase&t=event&pr1id=sku.42&pr1nm=Product&pr1pr=175.0000&pr1qt=1.0000&pr1ps=27432"] []

As you can see, both are very similar with a few tiny differences:

Rounding is different:

tr=175
tr=175.0000

tt=30.37
tt=30.3700

Item position is missing in the first:

pr1qt=1.0000"
pr1qt=1.0000&pr1ps=27432"

Again, not really sure if this is causing the issue or just coincidence. But leaving here for reference.

Not working with PHP 8.1

Latest version is not working with Magento 2.4.4 and PHP 8.1

[09-Aug-2022 08:09:13 UTC] PHP Fatal error: During inheritance of IteratorAggregate: Uncaught Exception: Deprecated Functionality: Return type of TheIconic\Tracking\GoogleAnalytics\Parameters\CompoundParameterCollection::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /var/domains/domain.nl/releases/20220809065139/vendor/theiconic/php-ga-measurement-protocol/src/Parameters/CompoundParameterCollection.php on line 95 in /var/domains/domain.nl/releases/20220809065139/vendor/magento/framework/App/ErrorHandler.php:61

When a sessionId is not stored in the DB, dont create a random one, but the same for all these cases

Error 'For Analytics object, you must specify a value to be set for setAffiliation' while creating CreditCard payment with Stripe

When an order is created with Creditcard, this message occurs:

For Analytics object, you must specify a value to be set for setAffiliation

I think that can only occur when $order->getStoreName() returns null. With iDeal this doesn't happen, but it does with CreditCard. Perhaps because the order is captured directly?

Fetching Mollie order status in backend causes error

There is a Mollie Fetch Status button in the Magento backend:

When clicked, and the status is Paid, the observer in this extension is triggered. But it gives an error:

[2023-02-02T14:17:53.610338+00:00] main.INFO: elgentos_serversideanalytics_requests:  ["https://ssl.google-analytics.com/collect?ti=2000000384&tr=13.94&tt=2.42&ts=8.9900&ta=Main%20Website%0ATepper%20Efka%20-%20Store%0AEnglish&v=1&tid=UA-22992695-19&uip=144.178.244.242&cid=470046374.1675346692&dp=%2Fcheckout%2Fonepage%2Fsuccess%2F&pa=purchase&ec=Checkout&ea=Purchase&t=event&pr1id=AL%20668000&pr1nm=Airolube%20De-Icer%20%2F%20Windshield%20defroster%20-%20500ml%20Spray&pr1pr=4.9500&pr1qt=1.0000"] []
[2023-02-02T14:17:55.987210+00:00] main.ERROR: Environment emulation nesting is not allowed. [] []

This is because the environment emulation is already running for this store.

Maybe we can check which store it is emulating and if it's the same, skip starting it here? https://github.com/elgentos/magento2-serversideanalytics/blob/master/Observer/SendPurchaseEvent.php#L65

And if it is not the same, stop the current emulation, start the one we need, and restart the original? Or would that have some adverse effects?

Missing purchase events in case of adblock

Because of

magento2-serversideanalytics/Observer/SendPurchaseEvent.php

Line 75 in 879f2a9

!$elgentosSalesOrder->getGaSessionId()

Purchase events will no longer be fired at all once the customer placing the order does not have a session id in their cookie.

This is possible because a customer blocks GTM causing their GA id and GA session ids to never be created.
Previously this was solved for the GA id by creating a temporary id

INFO: Google Analytics cookie not found, generated temporary value: 000000000.000000000

magento2-serversideanalytics/Plugin/SaveGaUserDataToDb.php

Line 80 in 879f2a9

if ($gaUserId === null) {

However this same process does not exist for the session id

I'm afraid i do not know wether we should send the event with an empty session id or a generated session id

Client ID not correct

The Client ID is retrieved from the Cookie _ga.

The prefix is GA1 but the part of the code does the checking for GA4 and UA1. So there is no match and the code returns null. Now the module created a random Client id and the data is Google Analytics is wrong because the conversion uses a different client id then the original from the visitor session.

@indykoning maybe you can check this out, I saw the addition came from you

if (
            $gaCookieVersion != 'GA' . $this->gaclient->getVersion() &&
            $gaCookieVersion != 'UA' . $this->uaclient->getVersion()
        ) {
            $this->logger->info('Google Analytics cookie version differs from Measurement Protocol API version; please upgrade.');
            return null;
        }

https://github.com/elgentos/magento2-serversideanalytics/blob/master/Observer/SaveGaUserId.php#L132

Some info;

Every Client ID has the structure: GA.1.1 + unique identifier + Unix timestamp. The Unix timestamp is the exact time and date the user
first interacted with your pag

https://louder.com.au/2022/06/27/client-id-in-ga4-what-is-it-and-how-to-get-it-in-your-report/

We created a simple patch for testing with some clients and it's working.

@package elgentos/serversideanalytics2

diff --git a/Observer/SaveGaUserId.php b/Observer/SaveGaUserId.php
--- a/Observer/SaveGaUserId.php	
+++ b/Observer/SaveGaUserId.php	(date 1678796766430)
@@ -129,10 +129,7 @@
             return null;
         }
 
-        if (
-            $gaCookieVersion != 'GA' . $this->gaclient->getVersion() &&
-            $gaCookieVersion != 'UA' . $this->uaclient->getVersion()
-        ) {
+        if ($gaCookieVersion != 'GA' . $this->uaclient->getVersion()) {
             $this->logger->info('Google Analytics cookie version differs from Measurement Protocol API version; please upgrade.');
             return null;
         }