Although fairly irregular, the presence of a subdirectory in ebin/ while
running rebar3 escriptize would crash the run and generate a dump.

Since ebin subdirectories are generally not supported (adding a path for
an application only adds ebin/ and no subdirectories to the VM's path
set), we just silently ignore the directory altogether.

Fixes #1693

Rather than the caller having to think of what to purge or not, use
erlang:check_process_code/2 to detect if the caller (rebar3) may die
because of the operation.

If so, do a soft purge with a conditional delete instead of a hard purge
with a mandatory delete.

Prevents the killing of a plugin with itself

- Uses the code at https://github.com/tsloughter/rebar_alias and brings
  it within rebar3
- adds safety checks to prevent redefining built-in commands or obvious
  circular dependencies between commands (indirect circular deps are
  still possible)
- adds tests
- adds a systest to ensure no clash with the existing plugin

Has no ill effect as tested on Win10

The path reloading of plugins had been fixed properly, but the problem
is that the paths it was using to re-load only considered the current
compile step, rather than the overall state of plugins. As such, the
reloaded paths after plugin compilation only reloaded the *latest*
plugin and not the other ones.

This fix forces the addition of all built plugin paths to the code paths
after a plugin compile job is run. This ensures that the path is clean
for initial plugin deps (only add those that are required), and is
re-made total after the fact (add all the plugins possible).

This commit also includes a system tests suite that can be run
optionally; the problem with this plugin mechanism was impossible to
find through mocked dependencies, and a working counterexample was
provided to us. The systest suite can be run against real projects
without conflict to make sure no regressions are hit.

Adds an option (-m, --min_coverage, or {cover_opts, {min_coverage,X}})
to the 'cover' command, where the value is an integer between 0 and 100.

If the total coverage during the analysis is below the value received,
the command will fail with output like:

    ===> Requiring 64% coverage to pass. Only 62% obtained

If the rate is correct, the command silently passes as it currently
does.

This feature allows to enforce code coverage standards in a project if
desired.

This commit adds a common test hook along with the cth_readable stuff
whose role is to track failing test cases, and create a test
specification out of them.

The test specification is dumped on disk at
_build/<profile>/logs/retry.spec and can be accessed by calling
'rebar3 ct --retry'. This will auto-load the spec file if it can be
found and re-run the failing cases.

If any other argument is found on the list specifying tests, the
'--retry' argument is ignored.

All code for this is marked as experimental in case we end up (keeping
and then) dropping the feature.

This commit adds a common test hook along with the cth_readable stuff
whose role is to track failing test cases, and create a test
specification out of them.

The test specification is dumped on disk at
_build/<profile>/logs/retry.spec and can be accessed by calling
'rebar3 ct --retry'. This will auto-load the spec file if it can be
found and re-run the failing cases.

If any other argument is found on the list specifying tests, the
'--retry' argument is ignored.

All code for this is marked as experimental in case we end up (keeping
and then) dropping the feature.

The Erlang compiler runs based on a global state built from currently
loaded libraries and the configured code path that is available. For
this reason, the rebar3 compiler job unloads all plugin paths before
calling the Erlang compiler.

However, this causes a problem when an application uses a custom
resource handler with a dynamic version in their .app.src file since the
plugin that can be used to find the version has been unloaded.

Fortunately, the compile phase that runs the version handling is
distinct from the phase that uses the Erlang compiler. This patch fixes
the problem by re-loading the plugins' paths in memory before generating
the .app file, and before unloading them afterwards.

It appears that unloading them is unnecessary because the hooks after
that will re-load them, but it is likely better to play it safe with
that global state and clean up after ourselves. It offers better
protection for future changes.

Fixes #1657

Current rebar3 uses debug_info rules where debug_info is added by
default, and no_debug_info prevents the default from being added, and
removes any explicit debug_info, if any. The problem is that if
'no_debug_info' is anywhere in the config for a run, it cannot be
removed even with other profiles. additionally, no_debug_info ignores
special tuples like {debug_info, {Mod, Data}} and {debug_info_key, Key},
which can be used to add debug info and encrypt it (in lieu of plain
debug_info) respectively.

This patch makes it so that the following rules are in place:

- the last option seen takes priority, allowing profile overrides by
  the ordering rules the compiler expects
- the overriden options shall be explicitly deleted to avoid confusing
  the compiler
- any option related to debug info seen last cancels any no_debug_info
  that preceded it
- any no_debug_info option seen last cancels all of the other
  debug_info options
- if debug_info is seen last, it cancels out {debug_info_key, Key}
- if {debug_info_key, Key} is seen last, it cancels out debug_info
- All other options are left untouched in that context (defines can
  still be expanded and so on)

This should allow proper profile rules to be followed. Note that erl_opt
profile merging puts precedence on the *last* element of a list, to
match the compiler.

When fetching deps, if this is a clean repo there will be extensive
messages warning that dependencies which have already been fetched are
being skipped. For large projects being built and tested in a clean
environment this significantly increases the noise level of the build.

This modification adds an additional rebar option
(deps_warning_on_conflict) that will allow disabling these warning
messages. If deps_error_on_conflict is set, an error will still be
thrown. This will not change default behaviour of rebar.

There is a similar outstanding issue:
https://github.com/erlang/rebar3/issues/1105
However this seems to be a push for not outputting warnings when the dep
version is the same, rather than disabling warnings altogether.

Updates relx (windows fixes) and erlware commons (strings)

- OTP-21 readiness, Full Unicode support, massive dep upgrade
- fixed handling of proxy username and password when fetching registry
- git versions from tag made consistent and all strip 'v' prefix
- Prevent hard crash on duplicate plugin paths
- Fix include paths in profile multiapp edge case
- Fix unlock state carry, which broke do sequences with unlock in them.
- Avoid guessing on utf8 decoding of app files
- Various fixes related to .app files
- Warn user when an unsupported local git or hg resource is used
- Corrects a fix to src_dir values
- Update eunit_formatters to latest version
- Changes in wording of warnings for more accuracy

This replaces all deprecated function usage by alternative ones based on
a version switch enacted at compile time, preventing all warnings.

This will likely introduce some possible runtime errors in using a
Rebar3 compiled on OTP-20 or OTP-21 back in versions 19 and earlier, but
we can't really work around that.

A bunch of dependencies have been updated to support OTP-21 without
warnings as well.

The compiling of OTP applications is done by first topographically
sorting them according to their dependencies, deps-first. This allows
all compilation to take place in order. In the current code, the same
logic extends to top-level applications in an umbrella project.

Unfortunately, there are cases where this is not going to be true: when
an application has extra_src_dirs entries (or additional directories or
files) to conditionally compile under some profiles, it may start
depending on another top-level application dedicated to that profile for
include files.

However, such an app will never make it to production and neither will
the compilation artifacts that create the dependency. Under that
scenario, current rebar3 is unusable.

This patch makes it so that the compilation provider instead changes the
logic for top-level apps: rather than copying their directories one by
one and compiling them in order, it:

1. copies all top-level apps to the build directory so the files are in
   their proper locations
2. adds the top-level apps to the path (after the global hooks have run,
   so the existing scope and env has not changed)
3. runs the compilation as usual.

Fixes #1651

When a global plugin is used both locally and within the project, there
are cases when the rebar3 program will hard crash (killed in do_boot).
This has been traced to plugin-handling in compilation, where the same
code path may be purged twice in a row without further reloading for the
compile operation.

This of course yields the result where the code handling on the VM kills
all processes holding references to the module in memory, in this case
the rebar3 process itself.

By deduplicating the paths first, we ensure at most one purge before
reloading plugins and paths, and this prevents a hard crash.

When composed with 'do', not carrying the unlocks in state may create
problems.

Rather than trying one method and then the other, allow the caller to
specify the encoding of the expected file. All other schemes are risky
and won't work well.

Rollback the function's default interface to the binary format in case
any plugin used it for non-unicode content, preserving backwards compat.

Those aren't supported and so a warning should be output. Fixes #1003

Same default value as used in relx and other environments, but as
reported in #979 some tools don't like having no description available.

The parsing functions were used inconsistently, and the returned values
were bad in some clauses; things only worked because they are never used
within rebar3. This ensures the return types are consistent on all
clauses.

was still hardcoded to 3.4.4

The previous patch at #7c959cc fixed the usage of duplicate values
for directories through relative paths, but mistakenly went overboard
and dropped the `./` path, which is still fairly common. Similarly for
`../".

The code is modified to special-case such values and keep the code
working.

- fix sys config merging
- Fix relative src_dir specifications to avoid
  double .app.src file detection
- Recompile when include files change in
  non-default directories

When fetching src_dir values, some relative paths can be inserted. When
deduplicating the paths on the fetch, this fact means that logically
duplicate (but literally different) directories can be returned at once.

By normalizing the names, duplication bugs can be resolved.

Specifically, this impacts profiles. It appears that relx as a whole
requires its configuration to be merged in one tuple order (New takes
precedence over Old), whereas the overlays require the opposite (Old
takes precedence over New) since the operation order on disk is
important to work well.

This patch reorders overlay values such that the overlay of a profile
takes place *after* the basic overlay, ensuring that the profile actions
take place after the basic ones; this allows profiles to properly
overwrite files as expected (see #1609)

This is done while adequately maintaining the order of operations that
were required as part of #1563

Overlay vars of profiles are also checked to be working fine, along with
a test.

This fixes #1247 and #1609

When a config file exists at the root of a project, defines a given
configuration value for a given profile, and that a sub-application
(umbrella app) also has the same profile defined with the same key (but
different values), the configuration values of the sub-application's
profile would get silently dropped.

The problem being that when the function to merge profiles is applied
recursively, it is applied to each profile (so it will merge on the keys
test, prod, etc.) rather than to each of the values of each profile.

This patch reworks the profile merging so that the current behaviour is
respected overall (a profile cannot be cancelled by a subdep's
non-existant profile since its value should have been ignored), but
ensures that sub-deps' profiles are otherwise applied recursively with
the proper rules:

- dependencies favor prior values
- plugins favor new values
- erl_first_files combine the lists
- relx uses the tuple merge algorithm
- erl_opts has its own custom merge as well
- otherwise the new value takes precedence

A test has also been added.

There is a risk of breakage in some applications that may have relied on
the buggy behaviour to work, though at this time we are aware of none of
them.

During the building of escripts, multiple passes are done. Two of them
may end up duplicating content: one that gathers all of the beam files
that will be needed for the app to work, and a second one that goes over
the ebin/ directory of the root application to grab all the stuff in
there, prior to the run.

This allows to grab whatever could be required for runtime without
breaking the rest (or so I think), and sticks them at the front of the
archive, where it needs to sit for things to work fine.

Whenever the ebin/ directory contains a pre-compile .beam file, it gets
fetched both from the first pass described and the latter one. This
results in duplicate entries in the resulting zip files used for the
escript and makes the executable larger than it needs to be.

The patch is a simple 1:1 removal of duplicate values. Since
large pre-populated ebin/ directories are pretty rare, this should not
be too costly for the vast majority of users.

Fixes #1577

The code for symlink handling failed whenever a win32 platform with no
symlink capability would be detected. This patch is provided by
William H from the support ticket at
http://www.rebar3.org/v3/discuss/598225c90365bb00144bc07f, which
adds detection of failures in non-NTFS scenarios on Win32, and then
copies files instead of bailing out.

The end result should be appropriate support for such a platform.