Apparently hex only includes some files by default but not others
rather than a sane default of anything except for VCS directories. So
now I also get to maintain a list of files in my .app.src.
Apparently rebar3 can't generate a sane version number from Git tags and
adds a bunch of BS build number and ref tags which hex.pm then rejects.
Given that I can't find any documentation on how that works and reading
code is an exercise in over engineered abstractionism I've succumbed to
the demoralizing conclusion that I'll just have to go back to the dark
ages and maintain my versions both in source and in version control.
A call to `jiffy:encode(Term, [use_nil]).` will currently cause dialyzer
to issue a warning because `use_nil` is not included in `encode_option`.
This appears to just be an omission.
Previously if a key was malformed UTF-8 and the user specified the
`force_utf8` option we would fail to try and encode a fixed up version
of the object. This was due to missing a clause to catch the
`invalid_object_member_key` exception. This adds the clause and a couple
tests to ensure it works.
In the original PR for `return_trailer` @vlm pointed out that I wasn't
using enif_consume_timeslice correctly. This fixes that by changing out
its called.
Previously we attempted to define the total number of bytes to decode or
encode in a single NIF call and then would consume as much of the
timeslice as we processed. This is wrong because we may start the NIF
call with less than an entire timeslice left.
The new approach is to define the number of bytes to encode or decode
per reduction and then iteratively call enif_consume_timeslice until it
indicates that we should return.
Previously Jiffy would throw an error about trailing data if there is
any non-whitespace character encounter after the first term had been
decoded.
This patch adds a decoder option `return_trailer` that will instead
return a sub-binary starting at the first non-whitespace character. This
allows users to be able to decode multiple terms from a single iodata()
term.
Thanks to @vlm for the original patch.
The `val` variable is a register value that we need to be able to return
at any time from `decode_iter`. If it happened that a yield was
triggered while processing trailing whitespace the lack of persistance
caused decode to return a term intialized from a random integer value.
Obviously the Erlang VM did not enjoy this.
Thanks to @michalpalka for the report.
Fixes#66
This patch adds initial support for decoding/encoding to/from the new
maps data type.
I'd like to thank Jihyun Yu (yjh0502) for the initial versions of this
work.
This adds a configurable limit on the number of bytes produced by
the encoder before yielding back to the Erlang VM. This is to avoid the
infamous scheduler collapse issues.
The `jiffy:encode/2` now takes an option `{bytes_per_iter,
pos_integer()}` that controls the yield frequency. The default value is
2048.
This adds a configurable limit on the number of bytes consumed by
the decoder before yielding back to the Erlang VM. This is to avoid the
infamous scheduler collapse issues.
The `jiffy:decode/2` now takes an option `{bytes_per_iter,
pos_integer()}` that controls the yield frequency. The default value is
2048.
This is ground work to allow Jiffy to yield back to the scheduler.
Creating an encoder resource will allow for the necessary state to be
carried across NIF function invocations.
This is ground work to allow Jiffy to yield back to the scheduler.
Creating a decoder resource will allow for the necessary state to be
carried across NIF function invocations.
By default Jiffy is quite strict in what it encodes. By default it will
not allow invalid UTF-8 to be produced. This can cause issues when
attempting to encode JSON that was decoded by other libraries as UTF-8
semantics are not uniformly enforced.
This patch adds an option 'force_utf8' to the encoder. If encoding hits
an error for an invalid string it will forcefully mutate the object to
contain only valid UTF-8 and return the resulting encoded JSON.
For the most part this means it will strip any garbage data from
binaries replacing it replacement codepoint U+FFFD. Although, it will
also try and the common error of encoding surrogate pairs as three-byte
sequences and reencode them into UTF-8 properly.
The encoder can now return \u escaped unicode data instead of leaving
it as UTF-8 byte sequences. This done like so:
Eshell V5.8.3 (abort with ^G)
1> jiffy:encode(<<240, 144, 129, 128>>, [uescape]).
<<"\"\\uD800\\uDC40\"">>
The encode and decode functions now return the value directly without
being wrapped in a tuple on success. If there is an error, it is
thrown. This is to more closely match the semantics of term_to_binary
and binary_to_term.
Any number that can't be decoded in C is now passed back
to Erlang for decoding.
Large numbers passed to the encoder will make it through
and be processed in Erlang after the main encoding
process.