The app data, when set by `rebar3_app_discover` basically does not
include the modules if it was derived from the .app.src file, even if at
a later point the modules were compiled.

Since there is currently no clear way to add that information reliably
across all compiler versions in all types of compilers, we simply clear
the cache and re-derive the information when required.

This might have a small performance cost, but is required for
correctness whenever an application's first build is run. Specifically,
this bug was detected when a plugin included lager's parse transform,
and the application itself also required it, but two distinct versions
were needed.

This patch finalizes fixing this issue.

Compile time with parse transforms needs to have the deps first in scope
before the plugins

Due to building dependencies with potential artifacts such as parse
transforms or macros, project builder plugins should be included in the
path, but _after_ deps are loaded.

Doing otherwise means that if any of the dependencies is required at
compile time, those of a plugin might get used first.

The old path handling routine would force loading a bunch of
applications, but it did not necessarily unload them aside from specific
calls that the other test handlers used to need to do, but no longer
needed under the updated path management of 3.7.0.

The leftover calls in the cover compilation bits caused some apps to be
loaded and to remain loaded before tests started.

Signed-off-by: Fred Hebert <mononcqc@ferd.ca>

This is mostly useful for tests, where a test suite of any kind can be
interrupted halfway through so that the user can probe the running
system to see what is happening.

This is done as follows:

1. the user must call `r3:break()` in a test suite
2. the user runs the task as `r3:async_do(ct)`
3. the test holds up and the user can do whatever
4. the user calls `r3:resume()` and the test proceeds as normal

A safeguard is added so that breakpoints are only triggered in the shell
in async mode

Sample session:

    $ rebar3 shell
    ...
    1> rebar_agent:async_do(ct).
    ok
    ...
    Running Common Test suites...
    %%% rebar_alias_SUITE: .
    === BREAK ===

    2> % <do some checks>
    2> r3:resume().
    ok
    3> .....
    %%% rebar_as_SUITE: ...........
    %%% rebar_compile_SUITE: ......
    ...

This PR is a simpler and mergeable version of #1853 by @shamis, since
the provider changed format enough to make merging difficult.

Compared to #1853, this also puts the responsibility on each resource to
check rather than adding a new optional callback.

The process dictionary is use as a warning/check cache.

In some cases, such as when the global rebar.config file contains typoes
and invalid terms, the rebar3 executable fails when trying to log the
error since it hasn't been set yet, such as in #1792

This patch fixes that by going for a fallback mechanism.

Dollar Variable expansion (`$VAR`) was inadvertently disabled for
windows variables, although %VARIABLES% already worked. This reduced the
portability of hooks in general.

Additionally, tests would fail on windows due to bad quoting of paths:
the path C:/a/b/c would fail when passed to the command
`cmd /q /c C:/a/b/c` because it would interpret /a /b and /c as 3
options. Using quotes makes the tests pass.

app_info was turning any vsn into a binary which the vcs_vsn
function interprets as being an actual version and not something
like <<"git">>. original_vsn shouldn't be converted as it is then
not longer "original".

- Only set paths that need to be put as a priority
- Clean up paths before leaving API mode

The first point accounted for some performance cost, but the latter one
explains the 40% overhead in test runs: since rebar3 calls rebar3 a lot
with a bunch of fake apps, and that the new mechanism for path handling
by default does not _remove_ paths, it just _orders_ them, we would end
up in a situation where as the tests ran, more and more fake paths would
get added to the VM.

By the time the run was over, all path handling would take longer since
more paths needed filtering every time. By resetting paths at the end of
an API run, we prevent a given 'project' from polluting another one's
runtime and performance once the API successfully returns.

Some finishing touch to that code

Also handle some formatting

Move path management out of rebar_utils manual code path function
handling (which we leave there for backwards compat), and centralize
them to allow easier coordination of paths between plugins and deps.

On top of path handling, do a check of loaded modules to only purge and
reload those that actually need it done in order to prevent all kinds of
weird interaction and accidental purge kills. It also allows the
possible cohabitation of both at once, with a "in case of conflict pick
X" as a policy

Changing path handling in providers also highlighted a bunch of bugs in
some tests and appears to fix some in other providers, specifically
around plugins.

However, we add a once-printed deprecation warning for the new
interface.

* add compile type for dynamic project compilation

* new rebar_compiler abstraction for running multiple compilers

rebar_compiler is a new behaviour that a plugin can implement to
be called on any ues of the compile provider to compile source
files and keep track of their dependencies.

* fix check that modules in .app modules list are from src_dirs

* use project_type to find module for building projects

* allow plugins to add project builders and compilers