ckan / ckanext-googleanalytics

CKAN extension to integrate Google Analytics data into CKAN. Gives download stats on package pages, list of most popular packages, etc.

CKAN Google Analytics Extension

Status: Production

CKAN Version: >= 2.8

A CKAN extension that both sends tracking data to Google Analytics and retrieves statistics from Google Analytics and inserts them into CKAN pages.

Features

• Puts the Google Analytics asynchronous tracking code into your page headers for basic Google Analytics page tracking.

• Adds Google Analytics Event Tracking to resource download links, so that resource downloads will be displayed as Events in the Google Analytics reporting interface.

• Adds Google Analytics Event Tracking to some API calls so that usage of the API can be reported on via Google Analytics.

• Add Google Analytics Event Tracking function that can be used in any exstension to create your custom events tracking.

    ckanext.googleanalytics.plugin._post_analytics
  

Installation

1. Install the extension as usual, e.g. (from an activated virtualenv):

    $ pip install -e  git+https://github.com/ckan/ckanext-googleanalytics.git#egg=ckanext-googleanalytics
    $ pip install -r ckanext-googleanalytics/requirements.txt
   

2. Edit your development.ini (or similar) to provide these necessary parameters:

    googleanalytics.id = UA-1010101-1
    googleanalytics.account = Account name (i.e. data.gov.uk, see top level item at https://www.google.com/analytics)
    googleanalytics.username = [email protected]
    googleanalytics.password = googlepassword
   

   Note that your password will probably be readable by other people; so you may want to set up a new gmail account specifically for accessing your gmail profile.

3. Edit again your configuration ini file to activate the plugin with:

    ckan.plugins = googleanalytics
   

   (If there are other plugins activated, add this to the list. Each plugin should be separated with a space).

4. Finally, there are some optional configuration settings (shown here with their default settings)

    googleanalytics_resource_prefix = /downloads/
    googleanalytics.domain = auto
    googleanalytics.fields = {}
    googleanalytics.enable_user_id = false
    googleanalytics.download_handler = ckan.views.resource:download
   

   resource_prefix is an arbitrary identifier so that we can query for downloads in Google Analytics. It can theoretically be any string, but should ideally resemble a URL path segment, to make filtering for all resources easier in the Google Analytics web interface.

   domain allows you to specify a domain against which Analytics will track users. You will usually want to leave this as auto; if you are tracking users from multiple subdomains, you might want to specify something like .mydomain.com. See Google's documentation <http://code.google.com/apis/analytics/docs/gaJS/gaJSApiDomainDirectory.html#_gat.GA_Tracker_._setDomainName>_ for more info.

   fields allows you to specify various options when creating the tracker. See Google's documentation <https://developers.google.com/analytics/devguides/collection/analyticsjs/field-reference>.

   If enable_user_id is set to true, then logged in users will be tracked into the Google Analytics' dashboard. This way metrics can be tracked for every logged in user. You can read more about this feature and its benefits here <https://support.google.com/analytics/answer/3123662>_.

   When resource is downloaded, ckanext-googleanalytics posts event to GA and calls default download callback. If you are using CKAN>=2.9 and some of your plugins redefines resource.download route(ckanext-cloudstorage, for example), you can specify which function must be called instead of ckan.views.resource:download via ckanext.googleanalytics.download_handler config variable. For ckanext-cloudstorage you can use:

    googleanalytics.download_handler = ckanext.cloudstorage.views:download
   

Domain Linking

This plugin supports cross-domain tracking using Googles' site linking feature.

To use this, set the googleanalytics.linked_domains configuration option to a (comma seperated) list of domains to report for.

See Googles' documentation<https://support.google.com/analytics/answer/1034342?hl=en>_ for more information

Setting Up Statistics Retrieval from Google Analytics

1. Run the following command from src/ckanext-googleanalytics to set up the required database tables (of course, altering the --config option to point to your site config file):

    ckan -c ../ckan/ckan.ini initdb
   

2. Follow the steps in the Authorization section below.

3. Restart CKAN (e.g. by restarting Apache)

4. Wait a while for some stats to be recorded in Google

5. Import Google stats by running the following command from src/ckanext-googleanalytics::

    paster loadanalytics credentials.json --config=../ckan/development.ini
   

   (Of course, pointing config at your specific site config and credentials.json at the key file obtained from the authorization step) Ignore warning ImportError: file_cache is unavailable when using oauth2client >= 4.0.0

6. Look at some stats within CKAN

   Once your GA account has gathered some data, you can see some basic information about the most popular packages at: http://mydomain.com/analytics/dataset/top

   By default the only data that is injected into the public-facing website is on the package page, where number of downloads are displayed next to each resource.

7. Consider running the import command reguarly as a cron job, or remember to run it by hand, or your statistics won't get updated.

Authorization

Before ckanext-googleanalytics can retrieve statistics from Google Analytics, you need to set up the OAUTH details which you can do by following the instructions <https://developers.google.com/analytics/devguides/reporting/core/v3/quickstart/service-py>_ the outcome of which will be a file with authentication key. These steps are below for convenience:

1. Visit the Google APIs Console <https://code.google.com/apis/console>_

2. Sign-in and create a project or use an existing project.

3. In the Service accounts pane <https://console.developers.google.com/iam-admin/serviceaccounts>_ choose your project and create new account. During creation check "Furnish a new private key" -> JSON type. Write down "Service account ID"(looks like email) - it will be used later.

4. Save downloaded file - it will be used by loadanalytics command(referenced as <credentials.json>)

5. Go to GoogleAnalytics console <https://analytics.google.com/analytics/web/#management>_ and chose ADMIN tab.

6. Find "User management" button in corresponding column. Add service account using Service account ID(email) generated in 3rd step and grant "Read" role to it.

Testing

There are some very high-level functional tests that you can run using::

$ pip install -r dev-requirements.txt $ pytest --ckan-ini=test.ini

Future

This is a bare-bones, first release of the software. There are several directions it could take in the future.

Because we use Google Analytics for recording statistics, we can hook into any of its features. For example, as a measure of popularity, we could record bounce rate, or new visits only; we could also display which datasets are popular where, or highlight packages that have been linked to from other locations.

We could also embed extra metadata information in tracking links, to enable reports on particular types of data (e.g. most popular data format by country of origin, or most downloaded resource by license)

ckanext-googleanalytics's Issues

GA resource download tracking - not all links are handled

Resource page shows links to resource in two places, but only one link is handled by GA event tracking. The problem is that link is less visible and untracked link is more likely to be clicked by user:

The other link is not handled because css selector looks for resource-url-analytics class, which is not present in second link. One way would be to use

Also, dataset view has resource link, which is not handled:

Checked with CKAN 2.6 and master branch of extension

Does not work with CKAN 1.8

Plugin module was installed correctly, but then following the instructions the first error is:

Command 'initdb' not known (you may need to run setup.py egg_info)

Error when initialize ckan with the googleanalytics extension

I install the googleanalytic extension in ckan but when I initialize it throughout the paster serve command I get the following error:

File "/home/ubuntu/workspace/ckan/default/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/script/serve.py", line 284, in command
    relative_to=base, global_conf=vars)
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/script/serve.py", line 329, in loadapp
    **kw)
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
    return context.create()
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
    return fix_call(context.object, context.global_conf, **context.local_conf)
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
    val = callable(*args, **kw)
  File "/home/ubuntu/workspace/ckan/default/src/ckan/ckan/config/middleware/__init__.py", line 45, in make_app
    load_environment(conf, app_conf)
  File "/home/ubuntu/workspace/ckan/default/src/ckan/ckan/config/environment.py", line 97, in load_environment
    p.load_all()
  File "/home/ubuntu/workspace/ckan/default/src/ckan/ckan/plugins/core.py", line 139, in load_all
    load(*plugins)
  File "/home/ubuntu/workspace/ckan/default/src/ckan/ckan/plugins/core.py", line 153, in load
    service = _get_service(plugin)
  File "/home/ubuntu/workspace/ckan/default/src/ckan/ckan/plugins/core.py", line 255, in _get_service
    return plugin.load()(name=plugin_name)
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2249, in load
    return self.resolve()
  File "/home/ubuntu/workspace/ckan/default/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2255, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/ubuntu/workspace/ckan/default/src/ckanext-googleanalytics/ckanext/googleanalytics/plugin.py", line 6, in <module>
    import genshi
ImportError: No module named genshi

I am using the ckan version 2.6.0.

Textual SQL expression should be explicitly declared as text

When calling get_top_packages on return result set filtering im getting this error

sqlalchemy.exc.ArgumentError: Textual SQL expression "package.id = 'b4e66cda-2e..." should be explicitly declared as text("package.id = 'b4e66cda-2e...")

Possible solution is to wrap the expression with the sqlalchemy text

q.filter(text("package.id = '%s'" % package_id))

Unsupported reflection of expression-based index

I am getting the following warnings when running the loadanalytics command to retrieve GA statistics (https://github.com/ckan/ckanext-googleanalytics#setting-up-statistics-retrieval-from-google-analytics) with ckan 2.6.1 (not tested with previous versions)

/usr/lib/ckan/dcat/local/lib/python2.7/site-packages/sqlalchemy/engine/reflection.py:53: SAWarning: Skipped unsupported reflection of expression-based index idx_pkg_lname
  ret = fn(self, con, *args, **kw)
/usr/lib/ckan/dcat/local/lib/python2.7/site-packages/sqlalchemy/engine/reflection.py:53: SAWarning: Skipped unsupported reflection of expression-based index idx_pkg_slname
  ret = fn(self, con, *args, **kw)
/usr/lib/ckan/dcat/local/lib/python2.7/site-packages/sqlalchemy/engine/reflection.py:53: SAWarning: Skipped unsupported reflection of expression-based index idx_pkg_suname
  ret = fn(self, con, *args, **kw)
/usr/lib/ckan/dcat/local/lib/python2.7/site-packages/sqlalchemy/engine/reflection.py:53: SAWarning: Skipped unsupported reflection of expression-based index idx_pkg_uname
  ret = fn(self, con, *args, **kw)
/usr/lib/ckan/dcat/local/lib/python2.7/site-packages/sqlalchemy/engine/reflection.py:53: SAWarning: Skipped unsupported reflection of expression-based index idx_user_name_index
  ret = fn(self, con, *args, **kw)

Any ideas?

Statistics from Google Analytics on CKAN 2.2

Does statistics retrieval work on CKAN 2.2? In documentation it says "CKAN 1.X only". What would it take to support statistic retrieval on CKAN 2.2?

Error - <type 'exceptions.NameError'>: global name 'Authorizer' is not defined

Using CKAN master branch

Extension broken by discontinuation of version 2.3 of Analytics API

With the discontinuation of version 2.3 of the Google Analytics API by Google, this extension became broken. This has been reported on the CKAN Dev mailing list a couple of months ago, but apparently no one is working on a fix yet.

http://lists.okfn.org/pipermail/ckan-dev/2012-August/003016.html
http://lists.okfn.org/pipermail/ckan-dev/2012-October/003242.html

This extension needs to be updated to support the new version of the Analytics API. A migration guide is available from Google to help someone interested in doing that:

https://developers.google.com/analytics/resources/articles/gdata-migration-guide

No data being sent to Google Aanlytics

After not having any luck with a GA4 tag, I decided to "downgrade" and create a Universal Analytics tag instead. When using the Tag Assistant Chrome extension, it's giving the errror No HTTP response detected, which is summarized below.

Missing HTTP Response denotes that, while the Google Analytics code was detected in the page source, the script itself did not execute. Each time the Google Analytics script executes, we expect an HTTP Response carrying the tracking request to Google’s servers.
This means that the Google Analytics code is not implemented properly on the page and no data will be collected.
Recommended Action: Reinstall the Google Analytics snippet by removing the current iteration, copy a new version from the Analytics interface, and insert that version into the site.

I'm not seeing any errors in the docker container and the script is in the head tag of the HTML body (and being detected by the extension). We're running CKAN 2.9.

Events not being logged under CKAN 2.0

Viewing stats for CKAN 2.0 sites it would appear that no events are being logged by Google Analytics. This means that no data is available for resource downloads etc.

Replace "flame" symbol with actual download count

Show actual count of downloads rather than "flame" ...

TypeError and ImportError

After I executing loadanalytics command,
paster loadanalytics credentials.json --config=/etc/ckan/etc/default/production.ini
It showed 1 warni & 2 errors :

• WARNI [googleapiclient.discovery_cache] file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth
• ImportError: file_cache is unavailable when using oauth2client >= 4.0.0 or google-auth 2005-01-01 -> 2019-10-28
• TypeError: cannot concatenate 'str' and 'NoneType' objects

I thought WARNI can be ignored, but how about the other two errors?

My ckan version is 2.7

Installing googleanalytics plugin returns "BackendUnavailable" error

From everything I can tell, it looks like this error typically revolves around compatibility issues with python2 and python3. I'm running ckan 2.8, which should be compatible.

pip install -e  git+https://github.com/ckan/ckanext-googleanalytics.git#egg=ckanext-googleanalytics
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Obtaining ckanext-googleanalytics from git+https://github.com/ckan/ckanext-googleanalytics.git#egg=ckanext-googleanalytics
  Updating /usr/lib/ckan/venv/src/ckanext-googleanalytics clone
  Running command git fetch -q --tags
  Running command git reset --hard -q cbee3f06d6374492e12c204af7ef1ddd3ed00a4a
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 333, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 300, in _get_abstract_dist_for
    return self.preparer.prepare_editable_requirement(req)
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 532, in prepare_editable_requirement
    req, self.req_tracker, self.finder, self.build_isolation,
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/wrappers.py", line 161, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "/usr/lib/ckan/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
BackendUnavailable

Rework or remove `dbutil.get_top_packages()`

As noted in the dbutil.py module itself, "get_top_packages is broken, and needs to be rewritten to work with CKAN 2.*. This is because ckan.authz has been removed in CKAN 2.*"

Extension does not respect custom uploader implementations

ckanext--s3filestore implements a custom uploader and downloader, overriding the core download endpoint. ckanext-googleanalytics does the same, to register the download and calls the core download action from the package controller afterwards.

So if you load the googleanalytics before the s3filestore one, your downloads return 404, because the core filesystem filestore is being called:

ckan.plugins = ... googleanalytics s3filestore        # Does not work

Loading googleanalytics last works:

ckan.plugins = ... s3filestore googleanalytics        # Works

Question: does it track DataStore API usage?

Does this track DataStore API usage? My read of the code is that it doesn't; I can't see it overriding the datastore actions mapping anywhere, but I just wanted to confirm.

API tracking: missing dataset id in api calls

Event tracking which is added to API calls doesn't extract package id correctly, leaving no label in GA:

This can be fixed by extending https://github.com/ckan/ckanext-googleanalytics/blob/master/ckanext/googleanalytics/controller.py#L56 to check also for name_or_id.

Use of builtins module in controller breaks CKAN Version: >= 2.7, <=2.9 compatibility

The builtins module isn't available in Python2.7 and so controller.py:2 from builtins import str breaks on CKAN 2.7,2.8

Enabling Google Analytics Breaks the CKAN API

Enabling Google Analytics on CKAN 2.8 breaks using the CKAN API. Here are the logs

[Tue Jul 19 16:18:00.967516 2022] [wsgi:error] [pid 5736:tid 140046066464512] [remote 127.0.0.1:58312] 2022-07-19 16:18:00,967 INFO  [ckan.lib.base]  /api/3/action/package_show render time 0.072 seconds
[Tue Jul 19 16:18:01.019323 2022] [wsgi:error] [pid 5735:tid 140046066464512] [remote 127.0.0.1:58318] 2022-07-19 16:18:01,019 DEBUG [ckanext.googleanalytics] Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'id=ID'
[Tue Jul 19 16:18:01.020749 2022] [wsgi:error] [pid 5735:tid 140046066464512] [remote 127.0.0.1:58318] 2022-07-19 16:18:01,020 INFO  [ckan.controllers.api] Bad Action API request data: Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'id=ID'
[Tue Jul 19 16:18:01.021251 2022] [wsgi:error] [pid 5735:tid 140046066464512] [remote 127.0.0.1:58318] 2022-07-19 16:18:01,021 INFO  [ckan.lib.base]  /api/3/action/resource_delete render time 0.006 seconds
[Tue Jul 19 16:18:01.054942 2022] [wsgi:error] [pid 5736:tid 140045948966656] [remote 127.0.0.1:58320] 2022-07-19 16:18:01,054 DEBUG [ckanext.googleanalytics] Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'id=ID'
[Tue Jul 19 16:18:01.056197 2022] [wsgi:error] [pid 5736:tid 140045948966656] [remote 127.0.0.1:58320] 2022-07-19 16:18:01,056 INFO  [ckan.controllers.api] Bad Action API request data: Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'id=ID'
[Tue Jul 19 16:18:01.056583 2022] [wsgi:error] [pid 5736:tid 140045948966656] [remote 127.0.0.1:58320] 2022-07-19 16:18:01,056 INFO  [ckan.lib.base]  /api/3/action/resource_delete render time 0.005 seconds
[Tue Jul 19 16:18:01.091801 2022] [wsgi:error] [pid 5735:tid 140045948966656] [remote 127.0.0.1:58322] 2022-07-19 16:18:01,091 DEBUG [ckanext.googleanalytics] Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'id=ID'
[Tue Jul 19 16:18:01.093187 2022] [wsgi:error] [pid 5735:tid 140045948966656] [remote 127.0.0.1:58322] 2022-07-19 16:18:01,093 INFO  [ckan.controllers.api] Bad Action API request data: Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'id=ID'
[Tue Jul 19 16:18:01.093598 2022] [wsgi:error] [pid 5735:tid 140045948966656] [remote 127.0.0.1:58322] 2022-07-19 16:18:01,093 INFO  [ckan.lib.base]  /api/3/action/resource_delete render time 0.006 seconds
[Tue Jul 19 16:18:01.127201 2022] [wsgi:error] [pid 5735:tid 140046066464512] [remote 127.0.0.1:58324] 2022-07-19 16:18:01,127 DEBUG [ckanext.googleanalytics] Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'id=ID'
[Tue Jul 19 16:18:01.128604 2022] [wsgi:error] [pid 5735:tid 140046066464512] [remote 127.0.0.1:58324] 2022-07-19 16:18:01,128 INFO  [ckan.controllers.api] Bad Action API request data: Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'id=ID'
[Tue Jul 19 16:18:01.129115 2022] [wsgi:error] [pid 5735:tid 140046066464512] [remote 127.0.0.1:58324] 2022-07-19 16:18:01,129 INFO  [ckan.lib.base]  /api/3/action/dataset_purge render time 0.007 seconds
[Tue Jul 19 16:18:01.161557 2022] [wsgi:error] [pid 5736:tid 140046066464512] [remote 127.0.0.1:58326] 2022-07-19 16:18:01,161 DEBUG [ckanext.googleanalytics] Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'name=Multiple_crops_suitability_and_yield&title=Multiple crops suitability & yield'
[Tue Jul 19 16:18:01.162822 2022] [wsgi:error] [pid 5736:tid 140046066464512] [remote 127.0.0.1:58326] 2022-07-19 16:18:01,162 INFO  [ckan.controllers.api] Bad Action API request data: Error decoding JSON data. Error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)',) JSON data extracted from the request: 'name=Multiple_crops_suitability_and_yield&title=Multiple crops suitability & yield'
[Tue Jul 19 16:18:01.163270 2022] [wsgi:error] [pid 5736:tid 140046066464512] [remote 127.0.0.1:58326] 2022-07-19 16:18:01,163 INFO  [ckan.lib.base]  /api/3/action/package_create render time 0.006 seconds

Any ideas?

python 2.7 support

Hi,
This extension is documented as compatible with ckan >= 2.7
I'm still using a CKAN 2.8 version, using python 2.7 (I know, it won't last). When I want to compile the extension, with python 2.7, I get a dependency issue with rsa: rsa stopped supporting python 2.7 after 4.0 version.

So, to still support python 2.7, you'd need to specify the rsa version in the requirements (rsa>=3.1.4, <=4.0)

Switching to Google Analytics 4?

Google Analytics 4 was introduced within the last few years, and I'm curious if any work has been done on this extension to enable a migration to a GA4 property. It appears that it's only set up for the old "Universal Analytics" but I'm a bit out of my depth so hoping this is a quick answer.

Is this extension compatible with ckan 2.9.2?

Is this extension compatible with ckan 2.9.2?

httpd threads max out

Some recent change brought in this bug. We observed that after a fresh Apache start, the httpd process count will keep increasing, and max out after a few hours. At this point any new api call such as /api/3/action/package_search will cause 500 error with apache error:
Error - <class 'thread.error'>: can't start new thread

Here is the detailed error info.

...
Module ckanext.googleanalytics.controller:65 in __init__
<<              t = AnalyticsPostThread(self.analytics_queue)
                   t.setDaemon(True)
                   t.start()

           def _post_analytics(
>>  t.start()
Module threading:474 in start
<<          _active_limbo_lock.release()
               try:
                   _start_new_thread(self.__bootstrap, ())
               except Exception:
                   with _active_limbo_lock:
>>  _start_new_thread(self.__bootstrap, ())
error: can't start new thread

Google password?

Why it requires google password? Google analytics doesn't require providing google password to track page views.

Consider merging datagovuk/ckanext-ga-report

The https://github.com/datagovuk/ckanext-ga-report extension provides usage information and charts based on the information collected in GA (see http://data.gov.uk/data/site-usage for the information we collect). It may well be worth merging into this repository (which will definitely need some work) as they are most definitely related.

Problems with the paster loadanalytics

I have the following error after executing paster loadanalytics

Encountered 403 Forbidden with reason "insufficientPermissions"
"User does not have any Google Analytics account"

I seems that could have something to do with the production.ini file (or maybe not). I don't know if something changed and that's the reason or I wrote wrong the configuration variables on the production.ini file. Can you share an example of how it should looks like? Or if you know another reason please share it. Between I am using CKAN2.8.

Thank you

pip install failed to build wheel

I ran the command to install the extension:
pip install -e git+https://github.com/ckan/ckanext-googleanalytics.git#egg=ckanext-googleanalytics

And I am getting the following error:

Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: /app/ckan/venv/bin/python /app/ckan/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpDHg8Cg
cwd: /app/ckan/venv/src/ckanext-googleanalytics
Complete output (18 lines):
Traceback (most recent call last):
File "/app/ckan/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in
main()
File "/app/ckan/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/app/ckan/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py", line 91, in get_requires_for_build_wheel
return hook(config_settings)
File "/app/ckan/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 146, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/app/ckan/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 127, in _get_build_requires
self.run_setup()
File "/app/ckan/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 237, in run_setup
self).run_setup(setup_script=setup_script)
File "/app/ckan/venv/local/lib/python2.7/site-packages/setuptools/build_meta.py", line 142, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 16, in
with open(os.path.join(HERE, 'README.md'), encoding='utf-8') as f:
TypeError: 'encoding' is an invalid keyword argument for this function

ERROR: Command errored out with exit status 1: /app/ckan/venv/bin/python /app/ckan/venv/local/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpDHg8Cg Check the logs for full command output.

No documented demonstration

We would like to see if ckanext_googleanalytics would be interesting for us. Unfortunately, we are reluctant to install and configure an extension which does not officially support CKAN 2.8, without having a fair idea of what it offers.

It would be helpful to offer a demonstration website, or - if one already exists - to document its location. Failing that, it would also help to mention an example of a site which uses ckanext_googleanalytics.

Allow configurable value for recent view days when getting data from Google Analytics API

Currently, when getting data from Google Analytics API via paster loadanalytics or ckan googleanalytics load it stores stats in tracking_summary.recent_views, package_stats.visits_recently and resource_stats.visits_recently from the last 14 days of views/visits.

Allow customization of this to be configurable per site via a CKAN configurable value. eg. 30 days

Summary page (/analytics/dataset/top) not working on CKAN 2.9

I've got the extension working on version 2.9 and am able to view the stats on Google Analytics & use ckan -c /etc/ckan/default/ckan.ini googleanalytics load mycredentials.json to update the database.

Unfortunately, I'm running into issues with this part:

7. Look at some stats within CKAN

Once your GA account has gathered some data, you can see some basic information about the most popular packages at: http://mydomain.com/analytics/dataset/top

I've tried visiting localhost:5000/analytics/dataset/top but that's just giving a 404 error. Not sure if this page is working on CKAN 2.9 or if I've messed up somewhere along the way.

Extension adds "?_ga=..." parameters to all external links

When googleanalytics is enabled in CKAN 2.6.1 (not tested with other versions), it adds parameters like ?_ga=1.259127497.2133035748.1425991248 to all outgoing external links (like in the footer) on mouse up event. It does not do this for links that go back to the same instance (like to datasets or organizations).

If the target of the external link processes HTTP GET parameters, it gets confused by this. I have not found a switch to turn this off.

requirements.txt vs. install_requires within setup.py

Hi there,

Is there a specific reason for keeping the dependencies of this Python package in a requirements.txt file and not in the install_requires=[] part of the setup.py?

The way this project is made ignores requirements.txt during pip install -e <repo>, making it mandatory to do an extra pip install -r requirements.txt. If we use install_requires, all dependencies would be handled during pip install -e <repo>, simplifying the installation steps.

I would like to have everything handles by pip and I can open a PR.

Also, if you think this way is interesting for the project, I can drop both requirements.txt and setup.py adopting the PEP 517's pyproject.toml, which would give even more flexibility with tools like Poetry (such as dependencies and development dependencies). Any thoughts?

Event Client Location

The event location is always my server's location, perhaps because uip isn't set so I guess the GA client IP is used as the event IP.

Have you considered setting uip? Do you experience the same? Is there a good reason for not setting it?

https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#uip

complete domain name not tracked

all events and absolute paths are tracked but I would like to have complete url in my analytics dashboard. I tried to set domain property but is not working.

[Question] Does the plugin support retrieving access information of CKAN API in CKAN >2.7

After I add-on the plugin in my CKAN (v2.7), it can correctly show user accessing ckan website behavior in google analytics platform.
But I also need to collect the statistics information of user's access behaviors by "ckan api" or "tool" (ie: wget) without through ckan web site, does the plugin support it ( ckan >2.7) ?

Add CI

Taks

• Define a CI platform to run tests.
• Run tests on each commit and PRs

Broken compatibility with python 2.7

Hi guys,
looks like it's impossible to install this plugin (from master branch) over python 2.7:

bash-5.1# python setup.py develop
Traceback (most recent call last):
File "setup.py", line 16, in
with open(os.path.join(HERE, 'README.md'), encoding='utf-8') as f:
TypeError: 'encoding' is an invalid keyword argument for this function

The above open(..., encoding) looks not supported on python2.7
This is my last working upgrade: 3e620b5