This adds a configurable limit on the number of bytes produced by
the encoder before yielding back to the Erlang VM. This is to avoid the
infamous scheduler collapse issues.
The `jiffy:encode/2` now takes an option `{bytes_per_iter,
pos_integer()}` that controls the yield frequency. The default value is
2048.
This adds a configurable limit on the number of bytes consumed by
the decoder before yielding back to the Erlang VM. This is to avoid the
infamous scheduler collapse issues.
The `jiffy:decode/2` now takes an option `{bytes_per_iter,
pos_integer()}` that controls the yield frequency. The default value is
2048.
This is ground work to allow Jiffy to yield back to the scheduler.
Creating an encoder resource will allow for the necessary state to be
carried across NIF function invocations.
This is ground work to allow Jiffy to yield back to the scheduler.
Creating a decoder resource will allow for the necessary state to be
carried across NIF function invocations.
A single quote input was causing segfaults due to sneaking past the
string termination logic. This patch corrects that lapse in conditional
by only parsing strings where a closing quote was found. All other
strings are rejected as invalid.
Big thanks to Jean-Charles Campagne (@jccampagne) for reporting the
issue.
While using rebar's dependency management was surprisingly easy it
broke apps that tried to build Jiffy as a dependency due to relative
path #includes.
This also fixes a few other issues. Most notably it removes the use of
the ECMAScript compatible encoding due to JSON's lack of support for +/-
Inf and NaN.
Floating point numbers are no longer encoded as a one to one mapping
of their binary representation, but as short as possible (while still
being acurate). The double-conversion library [1] is used to do the
hard work.
The ECMAScript compatible conversion is used.
[1] https://code.google.com/p/double-conversion/
After a few requests and some reflection I've decided to stop escaping
forward slashes in strings while still accepting the escaped version
through the decoder. This appears to mimic the behavior of other popular
JSON libraries I've checked (Ruby and Python).
Apparently I missed this when refactoring a do/while loop into a
standard while loop. It was a harmless extra invocation to check if the
stack was empty.
This prevents applications that depend on Jiffy from requiring PropEr as
a dependency just to run Jiffy's full test suite. If you want to run the
full test suite you'll have to run Jiffy's Makefile directly which
creates a `.jiffy.dev` marker that enables the full test suite.
I've had reports that HTTPS seems to fail a bit more often than the Git
URLs. I've seen this before as well so I've gone ahead and switched
protocols for the clone.
As it turns out I did not understand the documenation for
load/upgrade/unload correctly. load/upgrade are called conditionally if
there's code in the VM for the NIF. Ie, no code means load is called,
where as if code exists, upgrade is called.
unload is called regardless once per load/unload. This means that
load/upgrade in Jiffy's case will each create a state object and unload
will free it each time. I was missing the fact that unload is called
every time and hence I don't need to clear the state in upgrade.
The docs aren't entirely clear on the order of calls for upgrades so
this is mostly just in case old_priv ever happens to not be what load
returned in priv.
By default Jiffy is quite strict in what it encodes. By default it will
not allow invalid UTF-8 to be produced. This can cause issues when
attempting to encode JSON that was decoded by other libraries as UTF-8
semantics are not uniformly enforced.
This patch adds an option 'force_utf8' to the encoder. If encoding hits
an error for an invalid string it will forcefully mutate the object to
contain only valid UTF-8 and return the resulting encoded JSON.
For the most part this means it will strip any garbage data from
binaries replacing it replacement codepoint U+FFFD. Although, it will
also try and the common error of encoding surrogate pairs as three-byte
sequences and reencode them into UTF-8 properly.
* Update rebar to a more recent version
* Update rebar.config for deprecation warnings
* Allow people to export the REBAR environment variable to use
their own version of rebar
* Fixed `make clean`
I had a silly direction mistake in a bit shift that was causing the high
portion of all combining characters to be printed as \uD800 which is
obviously wrong. This bug only affects people using the non-default
uescape option during encoding.
Numbers like 1.0 were being encoded as <<"1">> which can lead to a bit
of confusion. This merely checks if a decimial point exists and if not
it appends ".0" to the value.
I need to fix this so that object sizes don't explode when generating
larger values. Basically, as the type generator recurses is should be
adjusting the size value.
Paul J. Davis
Paul Guyot
Nobuhiro Iwamatsu
Volker Mische
Dave Cottlehuber
Benoit Chesneau
Dave Cottlehuber
Sergey Urbanovich
Jeremy Raymond