Consider an API with fewer mutable classes about handwriting-recognition

[wacky6]

In the current design, HandwritingDrawing and HandwritingStroke makes it possible to track drawing / stroke changes, so the implementation can reuse existing results (for better efficiency).

The number of strokes and points in a drawing could be very large (thousands, or tens of thousands of points). Subsequent predictions may only involve a small change to the drawing.

For example:

1. User wrote 3 paragraphs of text.
2. The first getPrediction() is called, the implementation segment text into paragraphs, and stores each paragraph's prediction internally.
3. User adds some new strokes to the 3rd paragraph
4. The second getPrediction() is called, the implementation knows the strokes (for the first 2 paragraphs) aren't modified and reuse their prediction.

AFAIK, built-in JS arrays can't do this efficiently.

[domenic]

As I said, that makes sense as to why you might need a HandwritingDrawing class, since maybe it's easier to cache getPrediction() results on a HandwritingDrawing class than on a JS array. (But see below.) But it doesn't explain why you need a HandwritingStroke class...

AFAIK, built-in JS arrays can't do this efficiently.

I think you could use either the built-in JS array, or a HandwritingDrawing instance, as the cache key for your internal store. If you used a built-in JS array, you'd have to either compute some sort of running hash of the array contents, or use the array object itself as the cache key and account for mutations of its contents during lookup. This is indeed more complicated, but maybe worth it in terms of simplifying the API for web developers.

[wacky6]

Thanks for the suggestion. I agree arrays are simpler, but may not be a good choice for some use cases. Please let me show a more concrete example.

Say, the user writes cursive text "hello" in a single stroke. The web application requests a prediction midway (e.g. after "hel" are written). The code will look like this:

const drawing = new HandwritingDrawing()
const stroke = new HandwritingStroke()

// The stroke starts.
drawing.addStroke(stroke)

// Points for "hel"
stroke.addPoint({x, y, t})
drawing.getPrediction()    // => prediction is "hel"

// Points for "lo"
drawing.addPoint({x,y,t})
drawing.getPrediction()   // => prediction is "hello"

The implementation can easily determine stroke has been changed after being added to the drawing, by updating drawing's state in addPoint.

If we replace stroke with an array:

const drawing = new HandwritingDrawing()
const stroke = []

// The stroke starts.
drawing.addStroke(stroke)

// Points for "hel"
stroke.push({x, y, t})
drawing.getPrediction()

// Points for "lo"
stroke.push({x, y, t})
drawing.getPrediction()

// Or mutate the value of a point.
stroke[1].x = newX

It's very hard (or impossible?) to determine the changed strokes without inspecting all strokes:

• Array object's identity isn't useful here, because it's the same object for two getPrediction calls
• Hashing has to compute all points in the stroke, because the point themselves are mutable. The end result is iterating through all the points.
• The hashing cost isn't trivial (iterating through tens of thousands of points in the drawing) compared to the time to perform an incremental prediction (on thousands of points, could be as low as 10 ms).

[domenic]

The hashing cost isn't trivial (iterating through tens of thousands of points in the drawing) compared to the time to perform an incremental prediction (on thousands of points, could be as low as 10 ms).

When I benchmark iterating through a ten-thousand item vector<> in C++, I get numbers much lower than 10 ms. I am not a benchmarking expert though; perhaps you could show me how you are getting larger numbers for this sort of operation?

[wacky6]

Discussed offline with @domenic:

The key is balance between API flexibility and easy-of-use.

Let's propose / plan a future addition to HandwritingStroke's constructor to accept Array<Point> (perhaps HandwritingDrawing as well), and convert them into addPoint calls internally.

With this addition, developers can avoid boilerplate-ish addPoint calls, and use array manipulations (which they are already familiar with) if suitable. This may not made into the first spec revision / release though.

HandwritingStroke enables easy implementation of incremental recognition (better performance if the drawing is large / contains lots of points), and future extension to per-stroke query methods (like the Recognized field in MS Ink API).