UTF8 in json data about zestdb

[jptmoore]

Do you have an example I can pass from my test client to trigger this? It is handling things like this for example:

Johns-MacBook-Pro:test john$ _build/default/client.exe --server-key 'vl6wu0A@XP?}Or/&BR#LSxn>A+}L)p44/W[wXL3<' --path '/ts/foo/latest' --mode get
{"timestamp":1512063381380,"data":{"event":"☀ ☁ ☂ ☃ ☄ ★ ☆ ☇ ☈ ☉ ☊ ☋ ☌ ☍ ☎ ☏ ☐ ☑ ☒ ☓ ☚ ☛ ☜ ☝ ☞ ☟ ☠ ☡ ☢ ☣ ☤ ☥ ☦ ☧ ☨ ☩ ☪ ☫ ☬ ☭ ☮ ☯ ☰ ☱ ☲ ☳ ☴ ☵ ☶ ☷ ☸ ☹ ☺ ☻ ☼ ☽ ☾ ☿ ♀ ♁ ♂ ♃ ♄ ♅ ♆ ♇ ♈ ♉ ♊ ♋ ♌ ♍ ♎ ♏ ♐ ♑ ♒ ♓ ♔ ♕ ♖ ♗ ♘ ♙ ♚ ♛ ♜ ♝ ♞ ♟ ♠ ♡ ♢ ♣ ♤ ♥ ♦ ♧ ♨ ♩ ♪ ♫ ♬ ♭ ♮ ♯"}}

[Toshbrown]

there is an example in past bin but I can provide some more:

 see file below.

[jptmoore]

Do you have the JSON on its own I can paste into a validator? I can save that to a file and try from the client.

[Toshbrown]

Sure, here is a file full of them, separated by a new line ;-)

badTweets.txt

[jptmoore]

Are these all failing? I tried the first one and that did not give me an error.

[Toshbrown]

Yep they all failed my end.

Here is a new set of logs. The tweets in badTweets are the ones rejected by the store. storeLogs.txt contains all the output from the store with logging enabled.

badTweets.txt
storeLogs.txt

[Toshbrown]

Turned out to be an issue with https://github.com/Toshbrown/nodeZestClient calculating the length of utf8 payloads incorrectly. Fix in version 0.0.9.