Coder Social home page Coder Social logo

yajl's Introduction

**********************************************************************
        This is YAJL 2.  For the legacy version of YAJL see
              https://github.com/lloyd/yajl/tree/1.x
**********************************************************************

Welcome to Yet Another JSON Library (YAJL)

## Why does the world need another C library for parsing JSON?  

Good question.  In a review of current C JSON parsing libraries I was 
unable to find one that satisfies my requirements.  Those are, 
0. written in C
1. portable
2. robust -- as close to "crash proof" as possible
3. data representation independent
4. fast
5. generates verbose, useful error messages including context of where
   the error occurs in the input text.
6. can parse JSON data off a stream, incrementally
7. simple to use
8. tiny

Numbers 3, 5, 6, and 7 were particularly hard to find, and were what 
caused me to ultimately create YAJL.  This document is a tour of some
of the more important aspects of YAJL.

## YAJL is Free.

Permissive licensing means you can use it in open source and
commercial products alike without any fees.  My request beyond the
licensing is that if you find bugs drop me a email, or better yet,
fork and fix.

Porting YAJL should be trivial, the implementation is ANSI C.  If you
port to new systems I'd love to hear of it and integrate your patches.

## YAJL is data representation independent.

BYODR!  Many JSON libraries impose a structure based data representation
on you.  This is a benefit in some cases and a drawback in others.
YAJL uses callbacks to remain agnostic of the in-memory representation.
So if you wish to build up an in-memory representation, you may do so
using YAJL, but you must bring the code that defines and populates the
in memory structure.

This also means that YAJL can be used by other (higher level) JSON
libraries if so desired.

## YAJL supports stream parsing

This means you do not need to hold the whole JSON representation in
textual form in memory.  This makes YAJL ideal for filtering projects,
where you're converting YAJL from one form to another (i.e. XML).  The
included JSON pretty printer is an example of such a filter program.

## YAJL is fast

Minimal memory copying is performed.  YAJL, when possible, returns
pointers into the client provided text (i.e. for strings that have no
embedded escape chars, hopefully the common case).  I've put a lot of
effort into profiling and tuning performance, but I have ignored a
couple possible performance improvements to keep the interface clean,
small, and flexible.  My hope is that YAJL will perform comparably to
the fastest JSON parser out there.

YAJL should impose both minimal CPU and memory requirements on your
application.

## YAJL is tiny.

Fat free.  No whip.

enjoy,
Lloyd - July, 2007 

yajl's People

Contributors

7ac avatar bluemarvin avatar bovine avatar conradirwin avatar dougm avatar emaste avatar gno avatar halostatue avatar jstamp avatar lloyd avatar mirek avatar mxcl avatar octo avatar patperry avatar plaguemorin avatar rflynn avatar sgravrock avatar shahbag avatar tjw avatar z00b avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yajl's Issues

Allow yajl_gen to be reset

Haiving the ability to reset all internal state and clear the buffer of a given yajl_gen instance would allow me to reuse it even if it were in yajl_gen_in_error_state or yajl_gen_generation_complete.

The benefit if in the yajl_gen_generation_complete state, is that I can continue to create new JSON strings with a single yajl_gen instance, skipping the need to continually malloc/free new instances.

While if in the yajl_gen_in_error_state, I could simply "start over" without needing to free/malloc a new instance.

wrong logic in yajl_tree_get (returns NULL even when val exists)

I just found a logic problem in the 2.0.2 implementation of yajl_tree_get. When searching for an element inside an object, when the element is found, you assign this new element to the "n" variable (overwriting its previous value, which was the parent object) and break the "for" loop. But then, you assert "i == n->u.object.len" as if the "n" variable was still the parent object, which is wrong, since it will be changed if the child element was found.

The code snippet below shows a simple fix for this bug:

yajl_val yajl_tree_get(yajl_val n, const char ** path, yajl_type type)
{
    if (!path) return NULL;
    while (n && *path) {
        unsigned int i, found = 0;

        if (n->type != yajl_t_object) return NULL;
        for (i = 0; i < n->u.object.len; i++) {
            if (!strcmp(*path, n->u.object.keys[i])) {
                n = n->u.object.values[i];
                found = 1;
                break;
            }
        }
        if (!found) return NULL;
        path++;
    }
    if (n && type != yajl_t_any && type != n->type) n = NULL;
    return n;
}

json_reformat should link against libm

Here is the patch:

--- a/reformatter/CMakeLists.txt 2011-12-20 01:23:22.000000000 +0100
+++ b/reformatter/CMakeLists.txt 2012-01-09 14:37:48.950202826 +0100
@@ -26,7 +26,7 @@

ADD_EXECUTABLE(json_reformat ${SRCS})

-TARGET_LINK_LIBRARIES(json_reformat yajl_s)
+TARGET_LINK_LIBRARIES(json_reformat yajl_s m)

copy the binary into the output directory

GET_TARGET_PROPERTY(binPath json_reformat LOCATION)

ensure_ascii mode

It would be useful to have a configuration flag for the encoder that causes it to \u escape non-ascii chars, resulting in an ascii output. similarly, it would be nice to have a similar flag for input validation.

malloc error checking

Currently malloc/realloc calls in yajl code (v1.0.5) are not checked for failures, which could cause problems or crash if malloc or realloc fails. Is there any plan to add those checks?

Srini

fix to enable yajl to compile using MSVC 2008 [HAZ_CODE]

On 64-bit windows & possibly other combos, the macro WIN32 is not defined - likely a compiler bug. This patch resolves this.

Index: yajl/yajl_gen.c

--- yajl/yajl_gen.c (revision 1137468)
+++ yajl/yajl_gen.c (working copy)
@@ -189,7 +189,7 @@
return yajl_gen_status_ok;
}

-#ifdef WIN32
+#if defined(_WIN32) || defined(WIN32) || defined(WIN32)
#include <float.h>
#define isnan _isnan
#define isinf !_finite

Parser gets in a bad state after parsing garbage string.

Consider this string: //.,p(.^^U^B>'S!_<@QmJR9uJDc'I.}84HZ{[5Ax6e>U6l8{y"@

It's not valid JSON, obviously. However, if yajl attempts to parse this string, it gets into a bad state and fails all subsequent parse requests. A workaround is to free the bad handle and allocate a new one.

Tests fail on 64 bit machines

Compilation is failing for me on 64 bit Ubuntu (SliceHost):

test case: 'cases/integers.json': --- cases/integers.json.gold 2009-04-24 21:32:45.000000000 +0000
+++ cases/integers.json.test 2009-05-14 22:50:49.000000000 +0000
@@ -10,5 +10,6 @@
integer: -123456789
integer: 2147483647
integer: -2147483647
-parse error: integer overflow
+integer: 2147483648
+array close ']'
memory leaks: 0
FAILURE

segfault in yajl_lex_lex on empty input

I'm seeing a segfault when the lexer receives no input. This happens in my application when curl reaches a timeout and has no response body, or more easily shown by launching json_reformat and sending an EOF. It looks like hand->lexer is never allocated and gets dereferenced as a NULL.

edit: and github's markup makes this impossible to post nicely... here's a link instead:

http://sprunge.us/MJgc

yajl_tree_get return 0 when path node name is the same

here is the JSON text:
{
"retcode": 0,
"result": {
"info": [{
"face": 180,
"flag": 8913472,
"nick": "tom",
"uin": 2951718842
}, {
"face": 252,
"flag": 512,
"nick": "jim",
"uin": 824317252
}, {
"face": 0,
"flag": 17302018,
"nick": "hanmeimei",
"uin": 1179162105
}, {
"face": 522,
"flag": 4719104,
"nick": "lilei",
"uin": 108219029
}]
}
}

below is the function to get "nick" node of the JSON text

char* getNickName()
{
char* path[20] = { "result", "info", "nick", (char *) 0 };
yajl_val v;
yajl_val node;
node = yajl_tree_parse(buffer, errbuf, sizeof(errbuf));
v = yajl_tree_get(node, path, yajl_t_string);
return YAJL_GET_STRING(v);
}

always return 0.
what's going on?

add support for validating unicode during generation

Today the generator passes through whatever bytes are handed to it directly to the output buffer, allowing invalid JSON to be generated. I'd like to see us add support for validating the input before writing to the buffer, with a config option to enable it (defaulting to off for backward compat).

Fails to build without modification on OSX (AFAIK)

I am unable to get this to compile without modifications on OSX (for iOS) with XCode. I can work around the problem, but not without modifying the source or the directory structure of the project. Neither of which I want to do.

What I've done to get it to compile:

First, I added the relative path from my project to yajl/src in my xcode project's (build settings, not project settings) 'user header search paths' and enabled 'always search user header paths'.

Note I've added the files to my project as references, and I've cloned the yajl project directly from github.

The directory structure is this:

my_projects/my_project_that_needs_to_parse_json/my_project.xcodeproject
my_projects/yajl

This all may not even be necessary, because the real problem is that the includes that are in the format <yajl/yajl_blah.h> don't work because this path is not a valid partial path anywhere in the project directory structure.

For example, #include <yajl/yajl_common.h> in yajl_parse.h. The problem here is the first yajl in the path, yajl_common.h's parent is "api" not "yajl". So this path, afaik, should resolve to this path in my setup: my_projects/yajl/yajl/yajl_common.h.

Yet there is no my_project/yajl/yajl directory, and, in fact, this path is to a file that is part of the existing directory structure. Facepalm.

So if I change all of these includes to #include <api/yajl_common.h> then it compiles, but, of course, I don't want to change the source files - I'd rather use the source as is.

I could also, theoretically, do a couple of other things:

  1. I can remove the yajl file in the src folder and replace it with a hard link to the api directory to fool XCode into thinking that 'api' is actually 'yajl'. That's just a bit too clever though, and I'll forget what I did by tomorrow's lunch.
  2. Write a script to copy all the source files to a directory structure that will work, but I still have the problem of some of the code including from yajl/api/ and some of the code including from yajl/yajl. And, I hate scripting. :-P

Perhaps there is something I'm missing in the build environment to work around this. Maybe this is one of those things that visual studio or other environments handle differently and/or better (or maybe there a way to get Xcode to handle it). I don't claim to be an expert in these type of build settings.

Maybe someone just needs to tell me which button to push to get this to work.

Anyway, obviously what I would like is to be able to use the directories and structure as is.

Note, as a quick fix, I could even do it with my project's precompiled headers if the #include <yajl/foo.h> was bracketed with the header include defines already in the code:

For example - <yajl/yajl_common.h> is included by yajl_parse.h.

instead of:

include <yajl/yajl_common.h>

do this:

ifndef YAJL_COMMON_H

define YAJL_COMMON_H

include <yajl/yajl_common.h>

endif

And then I'll do this in my precompiled header:

include "api/yajl_common.h"

This will obviously define YAJL_COMMON_H and then bypass the whole <yajl/file.h> business and not break everyone else.

Or maybe there's something better and more clever to be done here.

I'm happy to help out with any build testing to get this to work.

thanks!

Some numbers are allowed through as keys

Consider the following test code:

#include <stdio.h>
#include <yajl/yajl_parse.h>
#include <yajl/yajl_gen.h>                                                                                                                                                      

int main(int argc, char **argv)
{   
    /* Going to create a buffer for: {1 : 2, 2 : 3} */
    yajl_gen_config conf = { 0, "  " };
    yajl_gen handle;
    yajl_status status;
    const unsigned char *buffer = NULL;
    unsigned int length = 0;
    handle = yajl_gen_alloc(&conf, NULL);

    yajl_gen_map_open(handle);

    /* First key-value pair */
    status = yajl_gen_number(handle, "1", 1); 
    fprintf(stderr, "Status: %d\n", status);
    status = yajl_gen_number(handle, "2", 1); 
    fprintf(stderr, "Status: %d\n", status);

    /* Second key-value pair */
    status = yajl_gen_number(handle, "2", 1); 
    fprintf(stderr, "Status: %d\n", status);
    status = yajl_gen_number(handle, "3", 1); 
    fprintf(stderr, "Status: %d\n", status);

    yajl_gen_map_close(handle);
    yajl_gen_get_buf(handle, &buffer, &length);
    yajl_gen_free(handle);

    printf("Generated: %s\n", buffer);
    return 0;
}   

This generates:

Status: 0
Status: 0
Status: 1
Status: 1
Generated: {1:2}

It looks like the ENSURE_NOT_KEY macro does not work with the first element in the map

Memory leak with yajl_tree_parse

When calling yajl_tree_parse with invalid JSON, this called yajl_get_error which in turn called yajl_render_error_string which, at line 127 of yajl_parser.c, allocated memory but was never freed since yajl_tree_parse used it in snprintf

This patch will fix it but I haven't tested it alot:

424d423
<     char * internal_err_str;
442,446c441,445
<               internal_err_str = (char *) yajl_get_error(handle, 1,
<                     (const unsigned char *) input,
<                     strlen(input));
<             snprintf(error_buffer, error_buffer_size, "%s", internal_err_str);
<             YA_FREE(&(handle->alloc), internal_err_str);

---
>             snprintf(
>                 error_buffer, error_buffer_size, "%s",
>                 (char *) yajl_get_error(handle, 1,
>                                         (const unsigned char *) input,
>                                         strlen(input)));

Fixing locale-dependent conversions

(Sorry, I accidentally closed the original issue)

The functions strtod() (used in yajl_parser) and sprintf() (yajl_gen) are locale-dependent and thus cannot be reliably used to convert JSON values. The only reasonable solution I found to this was using setlocale() to insert the "C" locale. However, this can introduce side-effects, so I locally set and then restore it, like this:

char oldLocale; / Example for converting doubles /
oldLocale = setlocale(LC_ALL, NULL); /
saving old locale /
setlocale(LC_ALL, "C"); /
ensuring POSIX locale /
sprintf(i, "%g", number); /
now the conversion works as expected /
setlocale(LC_ALL, oldLocale); /
restoring locale */

This is, however, probably an inefficient quick fix. The locale should be set before starting the parsing/generation phases and restored at the end, if this does not affect string conversion.

yajl_tree_parse never parses integer numbers as integers, only doubles

I was trying to figure out why the YAJL library was always returning parsed numbers as DOUBLEs and not INTEGERs. I found this small bug in handle_number in yajl_tree.c. Essentially, endptr is never touched by yajl_parse_integer(), so the (endptr != NULL) check always fails as it is set to NULL a few lines above. The only indication of failure (that I can see) from yajl_parse_integer() is to set errno, so I changed it to check only that. I also added a check for non-integer characters, to stop actual doubles from being parsed as integers. This works for me, and I now get my integer numbers coming back with both YAJL_IS_INTEGER and YAJL_IS_DOUBLE. :D

I'm including a patch against the 2.0.1 source:

--- yajl_tree.c.ORIG    2012-11-20 15:46:46.906935724 +0100
+++ yajl_tree.c 2012-11-20 15:46:53.862970214 +0100
@@ -309,11 +309,10 @@ static int handle_number (void *ctx, con

     v->u.number.flags = 0;

-    endptr = NULL;
     errno = 0;
     v->u.number.i = yajl_parse_integer((const unsigned char *) v->u.number.r,
                                        strlen(v->u.number.r));
-    if ((errno == 0) && (endptr != NULL) && (*endptr == 0))
+    if (errno == 0)
         v->u.number.flags |= YAJL_NUMBER_INT_VALID;

     endptr = NULL;
--- yajl_parser.c.ORIG  2012-11-20 16:27:35.851079376 +0100
+++ yajl_parser.c   2012-11-20 16:27:47.511137195 +0100
@@ -42,6 +42,10 @@ yajl_parse_integer(const unsigned char *
     if (*pos == '+') { pos++; }

     while (pos < number + length) {
+        if ((*pos < '0') || (*pos > '9')) {
+            errno = EINVAL;
+            return sign == 1 ? LLONG_MAX : LLONG_MIN;
+        }
         if ( ret > MAX_VALUE_TO_MULTIPLY ) {
             errno = ERANGE;
             return sign == 1 ? LLONG_MAX : LLONG_MIN;

add option for escaping the '/' (solidus) character

Although this isn't technically required to be valid JSON, but it would be really awesome when using JSON in an HTML document to have the '/' character escaped for us for safety. We could just as well html-escape the resulting JSON output, but having the '/' character escaped during the encode would safe an extra pass on the output.

There's already code in place (though it's commented out) for this at https://github.com/lloyd/yajl/blob/master/src/yajl_encode.c#L61 but we'd need a way to expose the encoder config to that function so we could conditionally do the escape.

Fix json_reformat linking issue on uClibc

json_reformat calls yajl_gen_* functions, which internally use isnan() /
isinf(). On Glibc, these are provided by libc, but on uClibc you need to
link with -lm (like the spec says), so ensure we do so.

Signed-off-by: Peter Korsgaard [email protected]

reformatter/CMakeLists.txt | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/reformatter/CMakeLists.txt b/reformatter/CMakeLists.txt
index da4032b..d48e218 100644
--- a/reformatter/CMakeLists.txt
+++ b/reformatter/CMakeLists.txt
@@ -26,7 +26,7 @@ LINK_DIRECTORIES(${CMAKE_CURRENT_BINARY_DIR}/../${YAJL_DIST_NAME}/lib)

ADD_EXECUTABLE(json_reformat ${SRCS})

-TARGET_LINK_LIBRARIES(json_reformat yajl_s)
+TARGET_LINK_LIBRARIES(json_reformat yajl_s m)

copy the binary into the output directory

GET_TARGET_PROPERTY(binPath json_reformat LOCATION)

1.7.5.3

problem with decoding unicode

while using unicode data library is throwing exceptions:

In [1]: a = u'[{"data":"Podstawow\u0105 opiek\u0119 zdrowotn\u0105"}]'
In [2]: import yajl
In [3]: yajl.loads(a)
---------------------------------------------------------------------------
UnicodeEncodeError                        Traceback (most recent call last)
/home/cms/sobre-cms/<ipython console> in <module>()
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0105' in position 19: ordinal not in range(128)

In [4]: import simplejson
In [5]: simplejson.loads(a)
Out[5]: [{u'data': u'Podstawow\u0105 opiek\u0119 zdrowotn\u0105'}]

same code for different libraries (here simplejson) working fine

inline hot functions as an optimization?

Requested by Brent Miller. It may be worth a quick profile to quantify gains and see if they're meaningful... Below is Brent's original suggestion.

I was just looking at a profiled run of a binary that uses the yajl package you gave us (yajl-1.0.3.0), and I have a suggestion.

yajl_lex_lex() gets and particularly readChar() get called a LOT. We are streaming a lot of JSON and I see 30% of our CPU being spent in yajl_lex_lex(), 15% of that being spent in readChar.

Optimizing these funcs would save us hardware immediately. If nothing else, I think it's worth persuading the compiler to inline readChar().

For example, if using a C compiler, replace this:

static unsigned char
readChar(yajl_lexer lxr, const unsigned char * txt, unsigned int _off)
{
if (lxr->bufInUse && yajl_buf_len(lxr->buf) &&
lxr->bufOff < yajl_buf_len(lxr->buf))
{
return *((const unsigned char *) yajl_buf_data(lxr->buf) +
(lxr->bufOff)++);
}
return txt[(_off)++];
}

with a macro version:

define readChar(lxr, txt, off) \

    (((lxr)->bufInUse && yajl_buf_len((lxr)->buf) && lxr->bufOff < yajl_buf_len((lxr)->buf)) ?
     (*((const unsigned char *) yajl_buf_data((lxr)->buf) + ((lxr)->bufOff)++)) :
     ((txt)[(*(off))++]))

(and maybe do the same for unreadChar)

isnan, isinf missing in uClibc environment

Building yajl on a uClibc based Alpine Linux machine failed with the error:

Linking C executable json_reformat
../yajl-2.0.3/lib/libyajl_s.a(yajl_gen.c.o): In function `yajl_gen_double':
yajl_gen.c:(.text+0x565): undefined reference to `__isnan'
yajl_gen.c:(.text+0x580): undefined reference to `__isinf'
collect2: ld returned 1 exit status

I fixed the issue by linking the json_reformat target with libm:

diff --git a/reformatter/CMakeLists.txt b/reformatter/CMakeLists.txt
index da4032b..d48e218 100644
--- a/reformatter/CMakeLists.txt
+++ b/reformatter/CMakeLists.txt
@@ -26,7 +26,7 @@ LINK_DIRECTORIES(${CMAKE_CURRENT_BINARY_DIR}/../${YAJL_DIST_NAME}/lib)

 ADD_EXECUTABLE(json_reformat ${SRCS})

-TARGET_LINK_LIBRARIES(json_reformat yajl_s)
+TARGET_LINK_LIBRARIES(json_reformat yajl_s m)

 # copy the binary into the output directory
 GET_TARGET_PROPERTY(binPath json_reformat LOCATION)

OSX build installs with bad dylib name.

I found this problem installing yajl 1.0.6 from macports (and can reproduce with 1.0.9):
http://trac.macports.org/ticket/24238

It appears that the .dylib doesn't have the correct INSTALL_NAME_DIR set. I found that the following addition to src/CMakeLists.txt fixed it for me:

IF(APPLE)
  SET_TARGET_PROPERTIES(yajl PROPERTIES
                        INSTALL_NAME_DIR ${CMAKE_INSTALL_PREFIX}/lib)
ENDIF(APPLE)

pkg-config file

It would be really handy to have a yajl.pc pkg-config file installed with libyajl.

Tree API marco fixes

One more "api/yajl_tree.h" fix :)

-#define YAJL_IS_INTEGER(v) (YAJL_IS_NUMBER(v) && ((v)->u.flags & YAJL_NUMBER_INT_VALID))
-#define YAJL_IS_DOUBLE(v) (YAJL_IS_NUMBER(v) && ((v)->u.flags & YAJL_NUMBER_DOUBLE_VALID))
+#define YAJL_IS_INTEGER(v) (YAJL_IS_NUMBER(v) && ((v)->u.number.flags & YAJL_NUMBER_INT_VALID))
+#define YAJL_IS_DOUBLE(v) (YAJL_IS_NUMBER(v) && ((v)->u.number.flags & YAJL_NUMBER_DOUBLE_VALID))

new config api

I'd like to move to a config API that allows us to add new config options without breaking binary compatibility.

Maybe something similar to the libmysql C api:

yajl_set_config((yajl_gen*)gen, YAJL_GEN_INDENT_STR, "  ")

As you pointed out already, there's a similarly interesting approach going on in gno@0795e14

tree API needs tests

The tests that apply to the SAX API aren't applied to the tree api. We need to add testing for the tree parsing implementation.

Request: support to simplified json

Is it possible to have an option to use an alternative json format, a kind of simplified one, where the attribute/member name, numeric values and boolean values, doesn't require double quotes and that doesn't require a comma to separate entries (maybe just end of line).

YAJL_GET_DOUBLE(v) fails with German locale

A German user of my software reported an odd bug where floating point values are being rounded down to a whole number, for example 54.12345 is being returned as 54. I believe the cause is the use of strtod() in yajl_tree.c:

v->u.number.d = strtod(v->u.number.r, &endptr);
https://github.com/lloyd/yajl/blob/master/src/yajl_tree.c#L320

strtod() expects a comma rather than a point when used in the de_DE locale, and probably countless other variations in other locales too. I was able to reproduce this by setting the env variable LC_ALL="de_DE".

yajl_tree_get() failure

yajl_tree_get() fails to return correctly when certain conditions in the json are met. Here is an example:

node = yajl_tree_parse( "{ \"Logging\": { \"timeFormat\": \"utc\" } }", errbuf, sizeof(errbuf)); const char * path[] = { "Logging", "timeFormat", (const char *) 0 }; yajl_val v = yajl_tree_get(node, path, yajl_t_string);

v is NULL when in fact it should be a string-value containing "utc".

It looks as if the following line in yajl_tree_get() is incorrect:

if (i == n->u.object.len) return NULL;

The purpose of this line is to check and see if we're done iterating through all the objects held by the node, but in this case it compares the array iterator i (0) to the found string-value "len" field, which is always 0.

My temporay fix is this replacement for yajl_tree_get():

yajl_val yajl_tree_get(yajl_val n, const char ** path, yajl_type type) { if (!path) return NULL; while (n && *path) { unsigned int i, len; if (n->type != yajl_t_object) return NULL; len = n->u.object.len; for (i = 0; i < len ; i++) { if (!strcmp(*path, n->u.object.keys[i])) { n = n->u.object.values[i]; break; } } if (i == len) return NULL; // if (i == n->u.object.len) return NULL; path++; } if (n && type != yajl_t_any && type != n->type) n = NULL; return n; }

wrong logic in handle_number (number flags will never have YAJL_NUMBER_INT_VALID)

There is small bug in the handle_number function. Athought yajl_parse_integer succeeds when the number is a valid integer number, "v->u.number.flags" will never be ORed with YAJL_NUMBER_INT_VALID because the conditional statement to check whether the parsing succeeded is wrong.

You're currently checking if "endptr" is pointing to the end of the string (as you do when parsing a double using strtod). However, yajl_parse_integer does not use "endptr", but the string length instead. This way, "endptr" will always be NULL.

The following code snippet fixes the problem:

    endptr = NULL;
    errno = 0;
    v->u.number.i = yajl_parse_integer((const unsigned char *) v->u.number.r,
                                       strlen(v->u.number.r));
    if ((errno == 0)/* && (endptr != NULL) && (*endptr == 0)*/)
        v->u.number.flags |= YAJL_NUMBER_INT_VALID;

Double conversion does not work on some locales

The strtod() function used to convert a double value in the double callback respects the user locale. This means in some locations it uses for instance the comma as decimal separator, truncating the converted double. Since JSON does not depend on the locale, this function should not be used, or it should warn the user.
I currently don't have an easy solution for this, but I'll tell you if I find one.

Don't work with UTF-8

File aka
{
"name": "Блок стен 4x4"
}

don't parse with command line
json_reformat.exe -u < filename.json

"parse error: client cancelled parse via callback ..."

floating point handling broken in 1.0.8 on win32

"The isinf-definition for WIN32 in yajl_gen.c seem to be broken since
_finite on windows return true if a value is not-inf thus making it
imposible to output any floats on windows."

contributed by Fredrik Kihlander (thanks!)

errno not being reset

In src/yajl_parser.c errno should probably be reset before being used/checked in yajl_do_parse()

0001-added-errno-resetting-before-setting-checking-it.patch

error installing under ubuntu 12.10

hello
i have reported and issue when installing a gem script (visage) and the author asked me to report the problem here.
here is the original issue : auxesis/visage#124
and here is the installation error :

# gem install visage-app
Building native extensions.  This could take a while...
ERROR:  Error installing visage-app:
    ERROR: Failed to build gem native extension.

        /usr/bin/ruby1.9.1 extconf.rb
creating Makefile

make
compiling yajl_ext.c
yajl_ext.c: In function ‘yajl_encode_part’:
yajl_ext.c:117:21: warning: variable ‘status’ set but not used [-Wunused-but-set-variable]
yajl_ext.c: In function ‘rb_yajl_parser_parse’:
yajl_ext.c:442:17: warning: variable ‘stat’ set but not used [-Wunused-but-set-variable]
compiling yajl_lex.c
compiling yajl_encode.c
compiling yajl.c
compiling yajl_parser.c
compiling yajl_alloc.c
compiling yajl_gen.c
compiling yajl_buf.c
compiling yajl_version.c
linking shared-object yajl/yajl.so

make install
/usr/bin/install -c -m 0755 yajl.so /var/lib/gems/1.9.1/gems/yajl-ruby-1.1.0/lib/yajl
installing default yajl libraries


Gem files will remain installed in /var/lib/gems/1.9.1/gems/yajl-ruby-1.1.0 for inspection.
Results logged to /var/lib/gems/1.9.1/gems/yajl-ruby-1.1.0/ext/yajl/gem_make.out

isinf sometimes missing on solaris/open solaris

i was basically getting symbol missing errors with yajl on open solaris. it turns out the box's math.h doesn't have isinf.

i googled, and this is problematic for a lot of other projects:
http://www.google.com/search?q=open+solaris+isinf+undefined

the mysql bug thread is particularly convincing:
http://bugs.mysql.com/bug.php?id=14862

for now, i got my yajl working by hacking this in there, but i know it's not safe since it's not testing for the environment to have ieeefp.h. sigh.

diff --git a/src/yajl_gen.c b/src/yajl_gen.c
index 548d948..5a66e22 100644
--- a/src/yajl_gen.c
+++ b/src/yajl_gen.c
@@ -194,6 +194,10 @@ yajl_gen_integer(yajl_gen g, long int number)
 #define isnan _isnan
 #define isinf !_finite
 #endif
+#ifndef isinf
+#include <ieeefp.h>
+#define isinf(x) (!finite((x)) && (x)==(x))
+#endif
 
 yajl_gen_status
 yajl_gen_double(yajl_gen g, double number)

feel free to run w/ the environment checks.

fyi, i'm using yajl by way of py-yajl. this hack worked pretty well since it seems that the py-yajl build process ignores the necessity for cmake and goes straight to compiling using gcc. i'll likely file an issue with py-yajl with a fix to pass in some -Dsymbol to force the isinf setting. perhaps if the fix were to read #ifdef MISSING_ISINF based on not finding isinf in math.h, then i can do a similar test in py-yajl's setup.py to set that as an extra compiler flag.

thanks.

yajl_tree_get() fails to return when there is only one node in the object

With JSON { "Network" : "Remote" } and yajl 2.0.4, you can't obtain the string value.

  root = yajl_tree_parse((const char *)ucs.content, err, sizeof(err));                                                                                                                                           
  if(root != NULL)         
    {
      const char *node[] = { "Network", (const char *) 0 };
      yajl_val element = yajl_tree_get(root, node, yajl_t_string);                                                                                                                                               
      if(element != NULL)  
        {
          location = strdup(YAJL_GET_STRING(element));                                                                                                                                                           
        }
    }
  yajl_tree_free(root);

element ends up NULL.

Unresolved externals yajl_tree_*

When some code compiled with c++ compiler and then dynamically linked with libyajl, whole new tree parsing api becomes broken.
It happens because that in "api/yajl_tree.h" api declarations aren't wrapped into "#ifdef __cplusplus ..."
I fixed this issue locally and submit that it works.

autotools based alternate build system

I'm keeping a fork with the alternate autotools build system, since it could coexist with the cmake you maintain would be great we could merge them.

Would it be useful for you? I needed to do that since on some platforms I use cmake doesn't build and cross compiling using cmake is far from straightforward/immediate as with autotools.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.