Closer to what it was initially, and reduces some extra operations too.

This was a long-standing pending issue with plugins having a weird code
path handling. The work done in the previous commit to do version
detection for the compiler in dependencies along with prior PRs that
added a deps-only mode to the `compile` provider means we can now
re-introduce plugins into the 3.14 compilation mechanism.

This makes compiler version detection a compiler task concern and
removes annoying manual checks, but also solves an unreported issue
(brought up on slack only) where a plugin would no longer have its priv/
or include/ dir included when used as a checkout dependency. Essentially
the latter bug was caused by being out-of-sync with application
preparation for a build task, but by re-folding it into the main flow,
we know we have the proper support as used for deps.

Rather than doing a check on each single file in a dep set (which can
grow quite large), do a dynamic rebuild of deps based on whether the DAG
mentions the files have been modified. To avoid reloading the whole DAG
more than once and then having to still map files, the compiler version
is added to rebar_compiler_erl's CritMeta for the DAG.

This comparison is only set in place to impact the
rebar_prv_install_deps call that handles pruncing and culling deps by
the compiler provider. Intra-DAG comparisons still need to be done by
rebar_compiler_erl for things to work, and the hope is that since most
built-in compiler mods end up generating erl files in deps, that will be
transparent enough to work well.

A DAG call to check the status of the dag (vsn, critmeta, existing,
etc.) has been added to do the faster check there and allowing proper
dispatch.

Plugins couldn't be updated because they don't really use the standard
dep-style facilities for things due to an early filtering.

This fix is done in three parts:

1. Add the compiler version to the data tracked in the DAG, and extract
   it from source file (when the DAG isn't around) for comparisons. Then
   on each build job, check for the compiler version as a build option
   like every other one.

   A change in compiler version represents a change in build options,
   which triggers a rebuild
2. Make it work for plugins. This requires more work because plugins
   that can find their ebin/ dir are assumed to be pre-built. Rather
   than undoing this, poke at 1 single beam file in their ebin/ dir
   and check if their compiler version matches ours to do an additional
   filter round. This is likely to be faster than reanalyzing the whole
   app, but a bit more brittle in the long run.
3. Make it work for deps. This currently is done with copy/pasted
   plugin code that needs to be reworked later if this is shown
   to be an acceptable extra step.

This patch also includes a fix on the path pruning for plugins, which
mistakenly subtracted AppInfo records from file paths; the intent was
to use the ebin dir as a comparison, which this patch fixes.

No tests have been added at this point.

fixes tsloughter/rebar3_hex#184

Mostly changing the format of debug messages, fixing one URL that was
bad in a warning.

Since we switched to DIAGNOSTIC=1 in the crashdump, the only way to make
DEBUG=1 discoverable (outside of docs) is in the help provider.

Didn't handle tuple-based dep definitions right

Upgrading dependencies already checked for package updates before running
the upgrade task. Plugins didn't have that mechanism in place and that
made upgrading plugins over hex trickier since users could be stuck a
long time without running an update manually.

This fix mostly readapts existing dep code to the context of plugins to
look them up one at a time and run the update as part of the upgrade.

This should cover most cases, including hooks.
What this specifically does is change some less useful messages for more
contextualized ones, including tying specific configurations to actions
taking place. This is particularly visible in hooks, and hopefully
provides some amount of discoverability.

Also of note:
- ebin/ directory copying is called explicitly when done since people
  have issues with that
- paths for compiled files are made relative since the analysis list
  shows the full paths already
- hooks are a bit more verbose but also more useful.

Move most stuff that have to do with effectiveness to the debug
level, and some stuff that has to do with implementation details to
diagnostics

The change was possible since R13B04 when macros with many arities
became supported. We never really cleaned that stuff up before so I'm
picking it up while I'm updating that code

Diagnostic mode is intended to be used for rebar3 maintainers to debug
and understand issues. It shows all the things and all the data and
works with the assumption that people has intent to use the source code
to debug things.

Debug mode is promoted to user-oriented debugging, based on the idea
that what they care about is figuring out their interactions with the
tool. A major improvement in usability would be to directly tie the user
config to the interpreted config (ie merged profiles) to clearly hint at
seen rebar3’s behaviour.  If the debug output can show that and even tie
some behaviours to the config value that impacts things, we could get
some discoverability out of it too, something missing from declarative
tools like this.

Co-authored-by: Fred Hebert <mononcqc@ferd.ca>

This commit changes rebar3 to check for name clashes of the
modules created from its templates with existing Erlang modules
to produce a non-blocking warning for each clashing module.

This is required for compile order to be maintained in a post-3.14
world.

... and e.g. an unknown type is detected

This commit changes mainly the `rebar_paths` module by adding a
new `runtime` target. When using that target, `rebar_paths:get_apps/2`
will, for each project app, find all their `applications` and
`included_applicactions` and filter those that could be found
in rebar3's state.

This was due to a subtle mistake in the new code in
https://github.com/erlang/rebar3/pull/2322 which optimized the search
behaviour of files not found by EPP. The patch mistakenly always kept
the top-level src/ directory for an app even if it did a lookup in
recursive ones.

This patch tracks and maintains the proper subdirectory hierarchy. To
keep performance adequate, the lists:keyfind/3 function is used, which
is also now a NIF to the same extent as lists:member/2; manual lookups
would likely end up reducing performance, particularly in deep
hierarchies.

A test case has been added to track the regression.
Reported by @elbrujohalcon

Saves a few seconds when loading DAG with thousands of files.
Unfortunately, uses internal knowledge about digraph implementation.

Instead of sequentially running *.erl (or any other) files
one-by-one, spawn a separate process, in a scatter-gather way,
then wait for all processes to complete.
Combined with the previous change it introduces a significant
speed-up for large repositories, up to a number of cores that
the host has.