grow / grow-ext-kintaro

A preprocessor for binding Grow content to Kintaro content.

Grow.dev

Grow is a declarative tool for rapidly building, launching, and maintaining high-quality static HTML.

• Easy installation
• Jinja template engine
• Content managed in YAML and JSON files
• Data-binding between content and templates
• Configuration-based site architecture
• Easy URL changes
• Flexible internationalization and translation
• Integration with external CMSes
• Integration with Google Sheets
• Fast builds

Quick start

One time only: install Pipenv and libyaml.

# On Mac with Homebrew (https://brew.sh/).
brew install pipenv libyaml

# On Ubuntu.
sudo apt install -y pipenv libyaml-dev

# On other distributions.
sudo apt install python-pip; pip install pipenv

Next: install and run Grow using a starter.

git clone https://github.com/grow/starter
cd starter
pipenv install
pipenv run grow install
pipenv run grow run

Documentation

Visit https://grow.dev to read the documentation.

grow-ext-kintaro's Issues

Workaround for handling deleted documents

In Kintaro, there is no way to delete a published document. Via the Grow-Kintaro extension, it should be possible to indicate to Grow somehow that a document should be deleted, and then Grow should delete its local copy of the document upon preprocessing.

For example, Grow can infer a document's deleted status by reading the value of a specific key, such as deleted. If deleted is True, then the document should be removed in Grow.

The name of this key should be customizable by the user (with a sane default, like deleted). In the podspec config options, deleted_key: "deleted" would be the default, but a user could override it with something like deleted_key: "is_deleted".

Python 3 compatibility

Add cache + cache timeout

For staging with Google App Engine or on the development server, there should be a way to tweak a cache with timeout (e.g. 30 seconds) to prevent the server from reloading Kintaro content all the time. Users should be able to control injection with a template tag, too, for partial/reference documents.

Unable to include this extension in podspec.yaml

I went through the install steps and it seemed to unpack correctly. When I include the extension in my podspec I get the following error:

Traceback (most recent call last):
  File "grow", line 86, in <module>
  File "site-packages/click/core.py", line 722, in __call__
  File "site-packages/click/core.py", line 697, in main
  File "site-packages/click/core.py", line 1066, in invoke
  File "site-packages/click/core.py", line 895, in invoke
  File "site-packages/click/core.py", line 535, in invoke
  File "grow/commands/run.py", line 37, in run
  File "grow/pods/pods.py", line 84, in __init__
  File "grow/common/utils.py", line 147, in __call__
  File "grow/pods/pods.py", line 532, in list_preprocessors
  File "grow/preprocessors/preprocessors.py", line 52, in register_extensions
  File "grow/common/utils.py", line 343, in import_string
ImportError: No module named extensions
Failed to execute script grow

I'm running version 0.1.7 on Windows 10 bash on Ubuntu but was able to reproduce the error on my mac. I also ran into this error when trying to install some of the other non-jinja2 extensions.

Can't run grow preprocess because of Google auth problems

Lately we haven't been able to run grow preprocess because we are not able to authenticate in Google.

Not sure if something changed in Google's authentication method, but this is the error that we are getting:

Looks like it is something related to

https://developers.google.com/identity/protocols/oauth2/resources/loopback-migration#what-is-the-loopback-ip-address-flow

https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html#dates-loopback

Support for handling locale-varied content

Kintaro has a concept of locale-varied content.

When content is locale-varied in Kintaro, it should be downloaded either to an key@<locale>: value within a YAML file or to a separate <file>@<locale>.yaml file. Currently, the Grow-Kintaro integration does not have any customized support for locale-varied content when downloading from Kintaro -> Grow.

Suffix translatable fields with @

Key names that should be translatable should be suffixed with a @ to be compatible with Grow's translation workflow.

https://github.com/grow/grow-ext-kintaro/blob/master/kintaro/kintaro.py#L91

Support for deep injection

Currently, if the document has a ReferenceField it is not updated when the doing the injection. Instead it would be good to use an instance of grow's DependencyGraph to track the dependencies and update them as well when doing an live injection.

Remove extra metadata

Currently all the metadata from kintaro is included in the yamls saved. A lot of the data isn't needed by grow and could be trimmed down to keep diffs and yaml updates more refined.

Add replicator feature to watcher

The watcher service should support an optional replicator which can replicate Kintaro data to GCS. This will support self-publishing content via Kintaro and client-side integrations that read Kintaro data directly from GCS.

The data hosted on GCS should not contain any information about the author / internal data / etc.

Example doesn't render partials

The example doesn't render the partial templates. Solution via Jeremy:

...
{% set basename = doc.collection.schemas_to_partials.get(field.schema_name) %}
{# If the schema name is not set, skip. #}
{% if not basename %}
  {% continue %}
{% endif %}

{# Determine whether the field corresponds to a partial. #}
{% set partial_filename = "/views/partials/{}.html".format(basename) %}
...

Add example for deployment to Google App Engine

Use same libraries version as Grow

Installing grow-ext-kintaro results in some pip incompatibilities.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
grow 2.1.0 requires google-api-python-client==1.9.3, but you have google-api-python-client 1.6.2 which is incompatible.
grow 2.1.0 requires httplib2==0.19.0, but you have httplib2 0.20.2 which is incompatible.
grow 2.1.0 requires python-slugify==4.0.0, but you have python-slugify 3.0.2 which is incompatible.

Looks like Grow is using some library versions python-slugify==4.0.0 but grow-ext-kintaro is using python-slugify==3.0.2

Same for google-api-python-client. Grow is using version 1.9.3 and grow-ext-kintaro is using 1.6.2

Auth refreshing repeatedly

Currently when attempting to sync with kintaro, each time it tries to sync a collection it is showing this:

Attempting refresh to obtain initial access_token
Refreshing access_token

The token should only need to be refreshed once.

Missing documents causes failure in import

After upgrading to grow 0.8.22 we found that there is an error when extension tries to download new documents.

grow.pods.errors.DocumentDoesNotExistError: Referenced document does not exist: /content/Components/Cards/5407287246848000.yaml
[47748] Failed to execute script grow

Our current workaround is to create an empty file but this is a lot of trouble when multiple team members are creating new documents.

Allow mapping filenames to field value

Currently when importing a collection it writes the file using the document id. Would be nice to be able to choose a field to use as the filename instead of the id.

Something like this in the config:

preprocessors:
- kind: kintaro
  repo: repo_name
  project: project_name
  bind:
  - collection: /content/kintaro/cars
    kintaro_collection: PageCar
    key_field: id

Support Kintaro Branches, Staging, and Publishing

Just getting a thread started. I'll keep this first comment updated with high-level items.

Issues to solve:

• Failure when collections with child collection document dependencies are called out of order
• Failure when a child document is deleted and it's removed before the parent is updated with the child change
• When a page is created or updated in Kintaro, a git branch should be created and it should only contain changes relevant to the new/updated page. (Don't push all changes in Kintaro)