Interupts download of large JSON files, declares them malformed. about jsonview

[bhollis]

Hm. I can actually get that file to work once it's in cache, but on the first load it doesn't work. Seems like something arcane in Firefox's data handling code...

[bhollis]

This pretty consistently works for me in FF 29.0.1 on OS X now.

[oliver-graetz]

I can confirm this issue for Firefox 30. I have a big JSON output that is dynamically created so there is no cache to mitigate the problem. When I am sending it using text/plain I see the complete output. Using application/json I get JSONView's "malformed" error message and the output is truncated at arbitrary positions that change on every reload.

[oliver-graetz]

The output works fine if the output of the dynamic creation is enclosed between ob_start(); and ob_end_flush();. This means that the very same output works if it is sent in one piece but fails if it reaches the browser in multiple chunks.

[bhollis]

We had a problem like that early on (FF4?) but fixed it with a lot of the complex stream code in JSONView. Looks like something has changed in how Firefox's streaming code works, but I'm not sure how to work around it.

[oliver-graetz]

Perhaps you could put my workaround using the PHP Output Buffering into the common issues section of the readme. Sometimes the size of the transfer seems not to be the actual problem, but its chunked nature.

[bhollis]

I think I'd prefer to just fix how we receive data to allow for chunked output.

[oliver-graetz]

g OK, I'll take a bug fix over a workaround anytime.

It's just that I don't expect quick fixes anymore. I'm still getting emails for updates to unfixed bugs on the Eclipse PDT bug tracker, over a year after switching to PhpStorm. Makes my day every time. Putting a shot note in a readme only takes minutes.

[bhollis]

@oliver-graetz could you send me a .php file that buffers output in a way that triggers the bug? I'm trying to reproduce it but I can't seem to.

Here's what I tried:

<?php
// This is a test of returning a JSON object slowly, over several chunks,
// in order to verify https://github.com/bhollis/jsonview/issues/54

header('Content-Type: application/json');
?>
{
<?php
  for ($i = 0; $i < 5; $i++) {
    sleep(1);
?>
  "buffered<?php echo $i ?>": "yeah",
<?php
    flush();
  }
?>
  "last": true
}
<?php flush(); ?>

[oliver-graetz]

Ouch, I completely overlooked your request. The problematic code has been refactored since and I couldn't reproduce it now. I will look into older revisions and try to extract a code sample.

[oliver-graetz]

The chunked sample was definitely too small. I meant to say that when already having using large amounts of data then sometimes the error can be avoided when using an output buffer for the whole message. Now I think it is more a thing of data size and timing because using a buffer makes the request finish faster and this sometimes removes the error. Also, actvating oder deactivating FireBug features like the network tab has an influence on whether the error occurs or not. Here is my current testing code:

<?php
header ('Content-Type: application/json');
$numRows   = 8000;
$numCols   = 25;
$colSize   = 4;
$useBuffer = false;

if ($useBuffer) ob_start();
echo '{"testdata":"something","data":['."\n";
for ($i = 0; $i < $numRows; $i++)
{
    if ($i > 0)
    {
        echo "\n,\n";
    }
    $col  = str_repeat('1234567890', $colSize);
    $data = array_fill(0, $numCols, $col);
    echo json_encode($data);
}
echo "\n]}\n";
if ($useBuffer) ob_end_flush();

You might have to adjust the variables at the top to trigger the error. The numRows might need values up to 25000. The supplied values are some where I currently can turn the error on with useBuffer=false and off with useBuffer=true.

I was using Firefox 31 and JSONView 0.8 to perform the tests. Using JSONView 0.9 would lead to other problems where JSONView doesn't finish at all (spinner keeps on spinning) for messages of this size, which is a worse problem but not the one we're dicussing here.

[Cosafina]

I don't know if it's any help to you at all, but the link to people.mozilla.org that hwine provided won't load at all with Firefox 35.0.1 and JSONView 0.9. It loads fine in Chrome, but Firefox won't play ball.
This also means we can't use it on our own application. Are there any plans to get this fixed?

[bhollis]

I'm able to reproduce this with @oliver-graetz's code (infinite spinner) with JSONView 0.9.1 on Firefox 38. I'm not able to reproduce it with the unreleased JSONView 1.0 (which hasn't changed anything about this) on Firefox Nightly (41). So I guess we just have to wait for Firefox to improve.