Add memcache for Room instances to avoid hitting Datastore on every message about samples

[jiayliu]

It seems NDB datastore is exactly what we need: https://cloud.google.com/appengine/docs/python/ndb/
"NDB uses Memcache as a cache service for "hot spots" in the data. If the application reads some entities often, NDB can read them quickly from cache."

[jiayliu]

Hmm, I changed the code to use NDB datastore and uploaded it to https://9-dot-apprtc.appspot.com. But the setup time does not seem to improve based on both client side log and server side log. (scratching head)

[jiayliu]

After more investigation I found most of the time for loading the first GET is spent in getting the lock. Acquiring the lock typically takes 100-200ms, but may take as long as >1s in some cases.
Querying the datastore takes 10-20ms for DB, 20-40 for NDB.
Writing to the datastore takes 30-40ms for DB and NDB.

[jiayliu]

BTW, we have a decent 80-90% memcache hit rate, so I can confirm memcache is indeed used in NDB.

[jiayliu]

Our code does not release the lock when calling channel.send_message, which is very slow.
Moving it out of the lock can save ~60ms.

[jiayliu]

Hmm, now I don't think using memcache or NDB will help us much.

NDB makes reading fast when it reads from cache. But apprtc usually does read + write to add a user, where the write will invalidate the cache, so the next read for the second user will load from datastore.
Using appstats, I see NDB almost doubles the RPC calls to add a user and takes more time than DB.

If we write our own memcache logic, we can make the read for the second user faster (20ms vs. <10ms). But the writing part is not any better, since we need to write to both memcache and datastore, and we cannot return the request before the datastore write is completed using async functions, because that will abort the async write.

Similar to adding a message to the memcache/datastore.

[juberti]

Let's discuss when I get back to Kirkland. Each candidate should be fast since that is only read from memcache, right?

[jiayliu]

We shouldn't do one datastore/memcache read per candidate, but only one
read in total when the second user joins.
If candidate is sent to GAE after the second user joins, GAE will send it
to GCE.

On Fri, Nov 14, 2014 at 4:36 AM, Justin Uberti [email protected]
wrote:

Let's discuss when I get back to Kirkland. Each candidate should be fast
since that is only read from memcache, right?

—
Reply to this email directly or view it on GitHub
#163 (comment).

-jiayl