The sys.config loading in the shell command did not implement recursion
and thus processed the file different from a `erl -config cfg/path`.

This commit moves the handling of distribution config and starting out
of rebar_prv_shell and into rebar_dist_utils. The module is able to
handle standard config options and boot a distributed node mode. This
could be used in plugins (once it is exposed) and other providers like
CT.

Configuration is also expanded so that options like:

    {dist, [{sname, atom()}, {name, atom()}, {setcookie, term()}]}

can be used and will be handled as a default. The config handler
supports similar terms from the command line being parsed in if the
calling provider supports them.

A test suite is added for configuration handling.

Following suggestions from @psyeugenic, this code terminates and waits
for the termination of the port handling IO before booting our own,
which should get rid of annoying warnings.

We also allow for the failure to shutdown the user worker under
kernel_sup, since it is likely not there anymore in many scenarios,
preventing crashes.

Replicates `erl` behaviour.

If a bad configuration file is submitted to rebar3 shell, display the
following error:

    ===> The configuration file submitted could not be read and will be
    ignored.

And keep going otherwise rather than silently failing.

While crash-fast is usually a good mechanism, the shell so far is very
tolerant of failures from apps to boot and whatnot, so this feels
appropriate.

Fixes #1019

This reuses the trick used within OTP to pick within old and new shell.

The 'user' structure is the same for all cases (escript, escript + dumb
TERM, unstable install, unstable install + dumb TERM), so we take it
down first.

Then we boot the TTY driver, which fails if TERM=dumb, in which case we
boot the retro-style usr. If it worked, we shut down the driver again,
and boot a modern shell structure.

This avoids all warnings and seems to work in all cases.

This is quite the hack.

This requires to detect the current shell running; if it's the new
shell, business as usual.

However, if it's the old shell, we have to find a way to take over it
and drive IO. This requires a few steps because:

- the old shell does not let you be supervised intelligently (it uses
  supervisor_bridge, so killing the child is not a supported operation
  from the supervisor)
- the old shell ignores all trappable exit signals except those coming
  from the Port in charge of stdio ({fd, 0, 1})
- the old shell shuts down on all exit signals from the stdio Port
  except for badsig, and replicates the shutdown reason otherwise
- An escript does not tolerate the `user` process dying (old shell) for
  any non-normal reason without also taking the whole escript down
- Booting in an escript has an implicit 'noshell' argument interpreted
  by the old shell as a way to boot the stdio Port with only stdout
  taken care of

Because of all these points, we have to kill the old `user` process by
sending it a message pretending to be the Stdio port dying of reason
`normal`, which lets it die without triggering the ire of its
supervision tree and keeping the escript alive. This, in turn, kills the
old stdio port since its parent (user.erl) has died.

Then we have to boot our copy of user.erl (rebar_user.erl) which
conveniently ignores the possibility of running the stdio port on stdout
only -- always using stdin *and* stdout, giving us a bona fide old-style
shell.

A known issue introduced is that running r3:do(ct) seems to then kill
the shell, and r3:do(dialyzer) appears to have an odd failure, but
otherwise most other commands appear to work fine.

Application masters are booted at the root of apps, and take over the
group leader role to redirect IO. To cut the chain short and properly
have their role inherited, they are their own leader, and keep a
reference to the old leader in their internal state, which we cannot
change.

This is done so process ownership to a given application can be
established, and allows to properly clean up resources outside the
supervision tree when an app is shut down.

This patch goes around and finds all processes whose group leaders are
application masters older than the new `user' process booted by the
shell providers, and swaps them with that new `user'. This lets the
application masters survive, and fixes the blocking IO issue
(resolving issue #899)

This may mean an incomplete clean up is down on application shutdown,
but that seems like a fair compromise.

Adds the ability to run an escript before starting the apps and
interactive shell for a project. This is intended to improve the
local development experience for projects by providing an easy way to
run companion services (mock rest APIs, databases etc) that the
project relies on.

This patch also adds {shell, Defaults} to the rebar config file so
that a project can supply default values for many of the new or
improved 'rebar3 shell' options:
* {apps, OTPApps}
* {script_file, EscriptFileName}
* {config, ConfigFileName}

The order of option precedence is command line, rebar.config, relx.

The list of applications in the relx config section could contain
tuples. The tuple will either contain a version constraint for the app,
the start type of the app or both.

This fix silently expands `{shell_apps, [Apps]}` to support the same
format.

This tries to reduce memory usage when running `rebar3 shell` by running
the agent in the current process (and avoiding to copy state
cross-boundaries), and using frequent hibernation after each run to
force a full GC and compaction of the current process.

Whenever the old shell got killed and an app got loaded prior, the whole
thing would silently drop output as the old 'user' process was replaced
while application master processes would keep the old one's pid in
their internal state.

To work around this limitation, make sure the apps are booted only after
the shell is replaced so that only the new `user` pid is used.

When swapping handlers, if a custom shell has been installed, it's
possible it'll take over the tty and other options. This may break
common operations that work on an otherwise regular shell, so we ignore
failures and let things work with the custom shell only.

- Crashes in providers lib when no providers in a namespace are bare
- Making sure bareness matches semantics; i.e. a bare provider is
  visible, a non-bare provider is hidden.

When setting up a shell some time elapses between listing the pids and setting
the group_leader. If the process exited during that time then
erlang:group_leader/2 will crash with badarg.

Helps with integration efforts, but unfortunately can't support the
'-sname' and '-name' options, only '--sname' and '--name'.

Config files from the command line would expect the wrong format:

    {app1, [...]}.
    {app2, [...]}.

Instead of the correct sys.config format:

    [{app1, [...]},
     {app2, [...]}]

Not supported yet: the recursive file references documented in
http://www.erlang.org/doc/man/config.html

Precedence still goes to shell_apps.

Change the order from load-config -> start-apps to load-apps ->
load-config -> start-apps

This allows proper checking of new configurations, deps, or plugins, and
makes sure they are detected during an active shell session.

The shell agent allows to run rebar3 commands and autoload compiled
modules when that is done.

- moved path addition, config loading and app boot to before the shell
  is available
- apps successfully booting are in an INFO message, failed to boot into
  an ERROR message
- A warning is printed when apps are booted informing to please use
  releases for actual deployment, and is omitted otherwise.
- Some minor refactorings otherwise.

Adds a `--config` flag to the shell command to specify a config file.
If the flag is not specified, attempts to read the sys_config
defined for relx.

Removed separate compilers
Resolves apps to build
Finds avail deps before pulling/building
Includes relx
Simplifies build commands

ensure any processes with a reference to an old user process as their
group leader are updated to use the new user process. this introduces a
slight delay at startup as the system must wait for the new processes
to be registered. there is a max wait period of three seconds (before
the shell command gives up and throws a timeout error)

fixes #314 ("rebar shell" somehow blocks using io:format in gen_server
handle_call)

attempt to emulate the behavior of
`erl -pa ebin -pa deps/*/ebin`

fix error messages and formatting issues of `rebar shell` by
shutting down and restarting the user subsystem in a mode more
hospitable to the shell than the simple user started when run
as an escript. emulate `error_logger` behaviour when the shell
is run via `erl`

add documentation of the shell command

limitations:

the erlang interrupt handler is not enabled when running as an
escript and there is no interface to re-enable it via erlang code.
this means `ctrl-c` will immediately exit the running process
unlike when running the shell via `erl`. `ctrl-g` is, however,
unaffected

the user subsystem is killed and restarted but not supervised. if
your code somehow relies on the user subsystem crashing and
restarting `rebar shell` may interfere with it's operation