Allows override of apps specified in other on-disk options. Equivalent
to `--apps=""`

Fixes #1785

Also small output fix in rebar3 shell

Based off a macro by @okeuday at https://github.com/erlang/otp/pull/1783

This patch makes it so that whenever the rebar3 shell has a new
configuration for an application that is already running and would be
restarted (without risking the stability of the node or functionality of
rebar_agent), we stop and restart the app.

This replaces all deprecated function usage by alternative ones based on
a version switch enacted at compile time, preventing all warnings.

This will likely introduce some possible runtime errors in using a
Rebar3 compiled on OTP-20 or OTP-21 back in versions 19 and earlier, but
we can't really work around that.

A bunch of dependencies have been updated to support OTP-21 without
warnings as well.

I noticed this when trying to include entop in a release. Entop uses
cecho which takes over the terminal, so you do not want it loaded or
started.  According to http://erlang.org/doc/man/rel.html, when you
specify a Type of none it should not load or start, but the code for
it's modules should be loaded.  This patch ensures the code is not
loaded or started, but doesn't do anything with the code paths.
At the very least this allows me to start a shell in the case where
I have an application of type none, and the application is neither
loaded nor started.

Like for instance: {app, "0.1.0", '='}

Some tricky changes in there but should be okay

Somehow swapping the tty handler once more fixes everything. I guess we
were missing a step somehow.

The sys.config loading in the shell command did not implement recursion
and thus processed the file different from a `erl -config cfg/path`.

This commit moves the handling of distribution config and starting out
of rebar_prv_shell and into rebar_dist_utils. The module is able to
handle standard config options and boot a distributed node mode. This
could be used in plugins (once it is exposed) and other providers like
CT.

Configuration is also expanded so that options like:

    {dist, [{sname, atom()}, {name, atom()}, {setcookie, term()}]}

can be used and will be handled as a default. The config handler
supports similar terms from the command line being parsed in if the
calling provider supports them.

A test suite is added for configuration handling.

Following suggestions from @psyeugenic, this code terminates and waits
for the termination of the port handling IO before booting our own,
which should get rid of annoying warnings.

We also allow for the failure to shutdown the user worker under
kernel_sup, since it is likely not there anymore in many scenarios,
preventing crashes.

Replicates `erl` behaviour.

If a bad configuration file is submitted to rebar3 shell, display the
following error:

    ===> The configuration file submitted could not be read and will be
    ignored.

And keep going otherwise rather than silently failing.

While crash-fast is usually a good mechanism, the shell so far is very
tolerant of failures from apps to boot and whatnot, so this feels
appropriate.

Fixes #1019

This reuses the trick used within OTP to pick within old and new shell.

The 'user' structure is the same for all cases (escript, escript + dumb
TERM, unstable install, unstable install + dumb TERM), so we take it
down first.

Then we boot the TTY driver, which fails if TERM=dumb, in which case we
boot the retro-style usr. If it worked, we shut down the driver again,
and boot a modern shell structure.

This avoids all warnings and seems to work in all cases.

This is quite the hack.

This requires to detect the current shell running; if it's the new
shell, business as usual.

However, if it's the old shell, we have to find a way to take over it
and drive IO. This requires a few steps because:

- the old shell does not let you be supervised intelligently (it uses
  supervisor_bridge, so killing the child is not a supported operation
  from the supervisor)
- the old shell ignores all trappable exit signals except those coming
  from the Port in charge of stdio ({fd, 0, 1})
- the old shell shuts down on all exit signals from the stdio Port
  except for badsig, and replicates the shutdown reason otherwise
- An escript does not tolerate the `user` process dying (old shell) for
  any non-normal reason without also taking the whole escript down
- Booting in an escript has an implicit 'noshell' argument interpreted
  by the old shell as a way to boot the stdio Port with only stdout
  taken care of

Because of all these points, we have to kill the old `user` process by
sending it a message pretending to be the Stdio port dying of reason
`normal`, which lets it die without triggering the ire of its
supervision tree and keeping the escript alive. This, in turn, kills the
old stdio port since its parent (user.erl) has died.

Then we have to boot our copy of user.erl (rebar_user.erl) which
conveniently ignores the possibility of running the stdio port on stdout
only -- always using stdin *and* stdout, giving us a bona fide old-style
shell.

A known issue introduced is that running r3:do(ct) seems to then kill
the shell, and r3:do(dialyzer) appears to have an odd failure, but
otherwise most other commands appear to work fine.

Application masters are booted at the root of apps, and take over the
group leader role to redirect IO. To cut the chain short and properly
have their role inherited, they are their own leader, and keep a
reference to the old leader in their internal state, which we cannot
change.

This is done so process ownership to a given application can be
established, and allows to properly clean up resources outside the
supervision tree when an app is shut down.

This patch goes around and finds all processes whose group leaders are
application masters older than the new `user' process booted by the
shell providers, and swaps them with that new `user'. This lets the
application masters survive, and fixes the blocking IO issue
(resolving issue #899)

This may mean an incomplete clean up is down on application shutdown,
but that seems like a fair compromise.

Adds the ability to run an escript before starting the apps and
interactive shell for a project. This is intended to improve the
local development experience for projects by providing an easy way to
run companion services (mock rest APIs, databases etc) that the
project relies on.

This patch also adds {shell, Defaults} to the rebar config file so
that a project can supply default values for many of the new or
improved 'rebar3 shell' options:
* {apps, OTPApps}
* {script_file, EscriptFileName}
* {config, ConfigFileName}

The order of option precedence is command line, rebar.config, relx.

The list of applications in the relx config section could contain
tuples. The tuple will either contain a version constraint for the app,
the start type of the app or both.

This fix silently expands `{shell_apps, [Apps]}` to support the same
format.

This tries to reduce memory usage when running `rebar3 shell` by running
the agent in the current process (and avoiding to copy state
cross-boundaries), and using frequent hibernation after each run to
force a full GC and compaction of the current process.

Whenever the old shell got killed and an app got loaded prior, the whole
thing would silently drop output as the old 'user' process was replaced
while application master processes would keep the old one's pid in
their internal state.

To work around this limitation, make sure the apps are booted only after
the shell is replaced so that only the new `user` pid is used.

When swapping handlers, if a custom shell has been installed, it's
possible it'll take over the tty and other options. This may break
common operations that work on an otherwise regular shell, so we ignore
failures and let things work with the custom shell only.

- Crashes in providers lib when no providers in a namespace are bare
- Making sure bareness matches semantics; i.e. a bare provider is
  visible, a non-bare provider is hidden.

When setting up a shell some time elapses between listing the pids and setting
the group_leader. If the process exited during that time then
erlang:group_leader/2 will crash with badarg.

Helps with integration efforts, but unfortunately can't support the
'-sname' and '-name' options, only '--sname' and '--name'.

Config files from the command line would expect the wrong format:

    {app1, [...]}.
    {app2, [...]}.

Instead of the correct sys.config format:

    [{app1, [...]},
     {app2, [...]}]

Not supported yet: the recursive file references documented in
http://www.erlang.org/doc/man/config.html

Precedence still goes to shell_apps.

Change the order from load-config -> start-apps to load-apps ->
load-config -> start-apps

This allows proper checking of new configurations, deps, or plugins, and
makes sure they are detected during an active shell session.

The shell agent allows to run rebar3 commands and autoload compiled
modules when that is done.

- moved path addition, config loading and app boot to before the shell
  is available
- apps successfully booting are in an INFO message, failed to boot into
  an ERROR message
- A warning is printed when apps are booted informing to please use
  releases for actual deployment, and is omitted otherwise.
- Some minor refactorings otherwise.

Adds a `--config` flag to the shell command to specify a config file.
If the flag is not specified, attempts to read the sys_config
defined for relx.