This is done through 3 main change groups:
- replacing `~s` by `~ts` in format strings, so that strings that
contain unicode are properly printed rather than crashing
- adding the `unicode` argument to all function of the `re` module to
ensure transformations on strings containing unicode data are valid
instead of crashing (see issue #1302)
- replacing `ec_cnv:to_binary/1` and `ec_cnv:to_list/1` with matching
functions in `rebar_utils`.
The last point has been done, rather than modifying and updating erlware
commons, because binary and list conversions can be a contentious
subject. For example, if what is being handled is actually bytes from a
given binary stream, then forcing a byte-oriented interpretation of the
data can corrupt it. As such, it does not appear safe to modify erlware
commons' conversion functions since it may not be safe for all its
users.
Instead, rebar3 reimplements a subset of them (only converting
atoms and chardata, ignoring numbers) with the explicit purpose of
handling unicode string data.
Tests were left as unchanged as possible. This may impact the ability to
run rebar3's own suites in a unicode path, but respects a principle of
least change for such a large patch.
This uses the very risky '$handle_undefined_function'/2 export from the
r3 and rebar_agent modules to allow meta-calls that can support plugins
and all other rebar3 extensions.
This is nasty but very tempting. Currently we only support:
- r3:do(Command)
- r3:do(Namespace, Command)
There is currently no way to pass arguments to the function such that we
can, for example, run cover analysis or tests on a subset of suites.
With the new abuse of '$handle_undefined_function'/2, we can detect the
unused commands (since they are not exported) and re-route them:
- r3:Command()
- r3:Command("--args=as a string")
- r3:Command(Namespace, "--args=as a string")
Of course, in doing so, we make it impossible to use the 'do' provider
(as in 'rebar3 do ct -c, cover') since the 'do' function is already
required for things to work.
Since the previous function had very strict guards, we can, without
conflict, add manual overrides that simulate the meta-calls fine.
Sample run:
https://gist.github.com/ferd/2c06d59c7083c146d25e4ee301de0073
Includes improvments and function documentation for all modules (in
alphabetical order) up to rebar_core, and may have included more in
other modules as I saw fit to dig and understand more of the internals.
When an app contains one of the modules we can't afford to get upgraded
without breakage (i.e. erlware_commons, providers, getopt), skip
reloading that application from the rebar3 agent, such that it can keep
on working in subsequent calls.
Blacklisted:
- erlware_commons (broke on ec_cmd_log)
- providers (core to functionality)
- cf (useful to not explode on colors)
- cth_readable (only used for CT suites, but incompatibilities may break
whole runs)
This tries to reduce memory usage when running `rebar3 shell` by running
the agent in the current process (and avoiding to copy state
cross-boundaries), and using frequent hibernation after each run to
force a full GC and compaction of the current process.
The old run didn't necessarily apply all profiles well, and would only
reload modules in the app file. However, since adding extra_src_dirs,
modules can be compiled without ending up in the app file; this lets
the rebar3 shell agent handle that case.
Fred Hebert
vans163
Tristan Sloughter
Viacheslav Kovalev