In the original PR for `return_trailer` @vlm pointed out that I wasn't
using enif_consume_timeslice correctly. This fixes that by changing out
its called.
Previously we attempted to define the total number of bytes to decode or
encode in a single NIF call and then would consume as much of the
timeslice as we processed. This is wrong because we may start the NIF
call with less than an entire timeslice left.
The new approach is to define the number of bytes to encode or decode
per reduction and then iteratively call enif_consume_timeslice until it
indicates that we should return.
Previously Jiffy would throw an error about trailing data if there is
any non-whitespace character encounter after the first term had been
decoded.
This patch adds a decoder option `return_trailer` that will instead
return a sub-binary starting at the first non-whitespace character. This
allows users to be able to decode multiple terms from a single iodata()
term.
Thanks to @vlm for the original patch.
This implements the `use_nil` option as discussed on issue #64. Passing
the atom `use_nil` as an option to both encode and decode will replace
the atom `null` with `nil` when decoding and encode `nil` as `null` when
encoding values.
Fixes#64Fixes#68
This patch adds initial support for decoding/encoding to/from the new
maps data type.
I'd like to thank Jihyun Yu (yjh0502) for the initial versions of this
work.
This adds a configurable limit on the number of bytes consumed by
the decoder before yielding back to the Erlang VM. This is to avoid the
infamous scheduler collapse issues.
The `jiffy:decode/2` now takes an option `{bytes_per_iter,
pos_integer()}` that controls the yield frequency. The default value is
2048.
This is ground work to allow Jiffy to yield back to the scheduler.
Creating an encoder resource will allow for the necessary state to be
carried across NIF function invocations.
This is ground work to allow Jiffy to yield back to the scheduler.
Creating a decoder resource will allow for the necessary state to be
carried across NIF function invocations.
While using rebar's dependency management was surprisingly easy it
broke apps that tried to build Jiffy as a dependency due to relative
path #includes.
This also fixes a few other issues. Most notably it removes the use of
the ECMAScript compatible encoding due to JSON's lack of support for +/-
Inf and NaN.
Floating point numbers are no longer encoded as a one to one mapping
of their binary representation, but as short as possible (while still
being acurate). The double-conversion library [1] is used to do the
hard work.
The ECMAScript compatible conversion is used.
[1] https://code.google.com/p/double-conversion/
By default Jiffy is quite strict in what it encodes. By default it will
not allow invalid UTF-8 to be produced. This can cause issues when
attempting to encode JSON that was decoded by other libraries as UTF-8
semantics are not uniformly enforced.
This patch adds an option 'force_utf8' to the encoder. If encoding hits
an error for an invalid string it will forcefully mutate the object to
contain only valid UTF-8 and return the resulting encoded JSON.
For the most part this means it will strip any garbage data from
binaries replacing it replacement codepoint U+FFFD. Although, it will
also try and the common error of encoding surrogate pairs as three-byte
sequences and reencode them into UTF-8 properly.
The encoder can now return \u escaped unicode data instead of leaving
it as UTF-8 byte sequences. This done like so:
Eshell V5.8.3 (abort with ^G)
1> jiffy:encode(<<240, 144, 129, 128>>, [uescape]).
<<"\"\\uD800\\uDC40\"">>
Any number that can't be decoded in C is now passed back
to Erlang for decoding.
Large numbers passed to the encoder will make it through
and be processed in Erlang after the main encoding
process.