While using rebar's dependency management was surprisingly easy it
broke apps that tried to build Jiffy as a dependency due to relative
path #includes.
This also fixes a few other issues. Most notably it removes the use of
the ECMAScript compatible encoding due to JSON's lack of support for +/-
Inf and NaN.
Floating point numbers are no longer encoded as a one to one mapping
of their binary representation, but as short as possible (while still
being acurate). The double-conversion library [1] is used to do the
hard work.
The ECMAScript compatible conversion is used.
[1] https://code.google.com/p/double-conversion/
By default Jiffy is quite strict in what it encodes. By default it will
not allow invalid UTF-8 to be produced. This can cause issues when
attempting to encode JSON that was decoded by other libraries as UTF-8
semantics are not uniformly enforced.
This patch adds an option 'force_utf8' to the encoder. If encoding hits
an error for an invalid string it will forcefully mutate the object to
contain only valid UTF-8 and return the resulting encoded JSON.
For the most part this means it will strip any garbage data from
binaries replacing it replacement codepoint U+FFFD. Although, it will
also try and the common error of encoding surrogate pairs as three-byte
sequences and reencode them into UTF-8 properly.
The encoder can now return \u escaped unicode data instead of leaving
it as UTF-8 byte sequences. This done like so:
Eshell V5.8.3 (abort with ^G)
1> jiffy:encode(<<240, 144, 129, 128>>, [uescape]).
<<"\"\\uD800\\uDC40\"">>
Any number that can't be decoded in C is now passed back
to Erlang for decoding.
Large numbers passed to the encoder will make it through
and be processed in Erlang after the main encoding
process.
jihyun
Paul J. Davis
Volker Mische
Sergey Urbanovich