rsc / tiddly

TiddlyWiki (actually TiddlyWeb) server for Google App Engine

TiddlyWiki App Engine Server

This is a minimal Google App Engine app, written in Go, that can serve as the back end for a personal TiddlyWiki

The TiddlyWiki5 implementation has a number of back end options. This app implements the backend expected by the “TiddlyWeb and TiddlySpace components” plugin.

The usual way to deploy TiddlyWeb is to run a fairly complex Python web server program. I'd rather not. Instead I implemented a minimal Go server that responds appropriately to the (relatively few) needed JSON API calls.

Authentication

The TiddlyWeb JSON API envisions a multiuser system in which different users have access to different sets of tiddlers. This Go server contains none of that: it assumes that all users have full access to everything, although it does record who created which tiddlers. The only access control is that the app.yaml here requires HTTPS and administrator login for all URLs, and as a “belt and suspenders” measure, the app itself also refuses to serve to non-admins, as checked by user.IsAdmin.

See the "Re Authentication" comment in tiddly.go for information about making the server publicly read-only (it's not quite perfect).

Data model

The app stores the current tiddlers in Cloud Datastore as Tiddler entities. It also stores every version of every tiddler as TiddlerHistory entities. Currently nothing reads the TiddlerHistory, but in case of a mistake that wipes out important Tiddler contents it should be possible to reconstruct lost data from the TiddlerHistory.

The TiddlyWiki downloaded as index.html that runs in the browser downloads (through the JSON API) a master list of all tiddlers and their metadata when the page first loads and then lazily fetches individual tiddler content on demand.

Deployment

Create an Google App Engine standard app and deploy with

appcfg.py -A your-app -V your-version update .

Then visit https://your-app.appspot.com/. As noted above, only admins will have access to the content.

Plugins

TiddlyWiki supports extension through plugins. Plugins need to be in the downloaded index.html, not lazily like other tiddlers. Therefore, installing a plugin means updating a local copy of index.html and redeploying it to the server.

Macros

TiddlyWiki allows tiddlers with the tag $:/tags/Macro to contain global macro definitions made available to all tiddlers. The lazy loading of tiddler bodies interferes with this: something has to load the tiddler body before the macros it contains take effect. To work around this, the app includes the body of all macro tiddlers in the initial tiddler list (which otherwise does not contain bodies). This is sufficient to make macros take effect on reload.

For some reason, no such special hack is needed for $:/tags/Stylesheet tiddlers.

Synchronization

If you set Control Panel > Info > Basics > Default tiddlers by clicking “retain story ordering”, then the story list (the list of tiddlers shown on the page) is written to the server as it changes and is polled back from the server every 60 seconds. This means that if you have the web site open in two different browsers (for example, at home and at work), changes to what you're viewing in one propagate to the other.

TiddlyWiki base image

The TiddlyWiki code is stored in and served from index.html, which (as you can see by clicking on the Tools tab) is TiddlyWiki version 5.1.21.

Plugins must be pre-baked into the TiddlyWiki file, not stored on the server as lazily loaded Tiddlers. The index.html in this directory is 5.1.21 with the TiddlyWeb and Markdown plugins added. The TiddlyWeb plugin is required, so that index.html talks back to the server for content.

The process for preparing a new index.html is:

• Open tiddlywiki-5.1.21.html in your web browser.
• Click the control panel (gear) icon.
• Click the Plugins tab.
• Click "Get more plugins".
• Click "Open plugin library".
• Type "tiddlyweb" into the search box. The "TiddlyWeb and TiddlySpace components" should appear.
• Click Install. A bar at the top of the page should say "Please save and reload for the changes to take effect."
• Click the icon next to save, and an updated file will be downloaded.
• Open the downloaded file in the web browser.
• Repeat, adding any more plugins.
• Copy the final download to index.html.

tiddly's Issues

Search does not search the text content of unloaded tiddlers

Quoting from tiddlywiki.com/dev/#LazyLoadingMechanism:

The browser-based search built into TiddlyWiki5 will only search the text of tiddlers that have been fully loaded. The expectation is that when lazy loading is used in a client-server configuration, then it is the server that really needs to handle search operations, because it is only the server that can "see" the text of all tiddlers. So, the plan is to integrate the built in search with TiddlyWeb's search API. The simplest approach is that any local search triggers an asynchronous server side search. The results of the search would be asynchronously loaded such that they would dynamically appear in the local search results.

TiddlyWeb's search API tiddlyweb/search

I guess that google's app engine datastore is easily searchable from Go code.
Is asynchronous request->response feasible for dynamic search results?

Or, is there a workaround for now such as loading fat tiddlers instead of skinny tiddlers?
(Surely for any truly tiddly tiddler worth its title, its text will be smaller than its metadata anyhow.)

Thanks for publishing this cool tiddly project 👍 🐈 🐡

Periodic backup to git

I wrote a App Engine service to extend this app that periodically backup the datastore to git. If #7 merges I am happy to send a PR.

https://github.com/philips/tiddly/tree/master/gitbackup

World readable, admin-only editing

I'm looking at how to make the wiki no-login-required to read it, while still admin for editing.

I poked around app.yaml definitions a bit https://cloud.google.com/appengine/docs/go/config/appref

From reading that, it looks like only URL regexs are supported, and I'm guessing since TW uses anchors, this might be a little tough? Any help is appreciated.

(Even with Google Cloud / App Engine etc. yak shaving, this was a slick and simple way to get TW running on a server. Thank you!)

This is actually a question - Changing the index.html do you need to do anything else?

I am new to Google App Engine so this has taken some time.
I was able to create, install, and now I am in the process of figuring out how to update this system.
I have added plug in's and other things that are specific to my TW5.

I followed the steps to update the index.html and it is working locally correctly.
Unfortunately it is not working via the app engine so I wanted to know do you need to also 'restart' something?(The plug in I added did not show up - the powered by tiddlywiki banner)

Thank you very much for creating this it is awesome.

Rich Shumaker