Rather than doing a check on each single file in a dep set (which can
grow quite large), do a dynamic rebuild of deps based on whether the DAG
mentions the files have been modified. To avoid reloading the whole DAG
more than once and then having to still map files, the compiler version
is added to rebar_compiler_erl's CritMeta for the DAG.

This comparison is only set in place to impact the
rebar_prv_install_deps call that handles pruncing and culling deps by
the compiler provider. Intra-DAG comparisons still need to be done by
rebar_compiler_erl for things to work, and the hope is that since most
built-in compiler mods end up generating erl files in deps, that will be
transparent enough to work well.

A DAG call to check the status of the dag (vsn, critmeta, existing,
etc.) has been added to do the faster check there and allowing proper
dispatch.

Plugins couldn't be updated because they don't really use the standard
dep-style facilities for things due to an early filtering.

The change was possible since R13B04 when macros with many arities
became supported. We never really cleaned that stuff up before so I'm
picking it up while I'm updating that code

Saves a few seconds when loading DAG with thousands of files.
Unfortunately, uses internal knowledge about digraph implementation.

Instead of sequentially running *.erl (or any other) files
one-by-one, spawn a separate process, in a scatter-gather way,
then wait for all processes to complete.
Combined with the previous change it introduces a significant
speed-up for large repositories, up to a number of cores that
the host has.

We run the analysis for an extra_src_dir by making a fake app for it,
but send in the app alone for analysis.

In there, the DAG pruning routine looks at the DAG and files submitted
and goes "this right here is a project with 99% of its apps deleted". It
then tries to prune the whole DAG except for some test files. The code
"works" simply because there's a false-positive check that makes sure
the file is on disk before removing it for the DAG.

This ends up making extra runs where ~80% of the time is spent
double-checking the false positives for file deletions.

This commit fixes this by merging in all extra_src fake apps and making them
run in a single analysis phase, meaning we only pay the cost of the DAG
pruning once for the whole project, making it faster than any sparse
repo.

There's also a small patch needed for the root-level extra src dirs;
turns out that since the context-handling in the `rebar_compiler` uses a
map to store content, running single-pass analysis clobbered entries for
a given app if they had more than one extra_src_dir in there.

I also took the time to clean up the ordering of that file.

Into a standalone module where it can be reused for optimisation work
later on.

Cleans up some annoying broken abstraction.

Also fix erl_first_file artifact tracking, which will avoid seeking to
disk by finding the opts in the DAG

- topsort of an app does a full analysis of all apps to come up with
  their app-level DAG
- each extra_src_dir has its own analysis run
- each extra_src_dir run is simulated as being an app
- we therefore end up doing a lot of useless analysis for single app
  runs

This patch notes runs with 1 or 0 apps and skips doing the topsort
analysis on them, which speeds things up further.

Also fixes artifacts left on filesystem, and produces less
filelib:last_modified() checks.

The propagation was confusing source files and artifacts; the artifact
ordering was flipped, and the tagging non-mandatory (aside from as
edges), which made things hard to identify when plugins for compilers
are used.

This allows to do quicker re-compile option validation by not requiring
to access the disk and check all candidate erlang files for option
changes. This also opens up the way to compilers that produce more than
one artifact per file.

THat was code written on a bus and rarely executed, but still the
function called didn't exist for real.

Reduces dependency verification time for large codebases.

This moves the old DAG stuff out from the main code paths and
substitutes it with the new epp-based DAG work.

Also fixes previous work for integration tests:

- consider ptrans from erl_opts in app ordering
- display proper app compilation info messages due to reordering

While leaving the old one in place, prep the ground for new analysis
phases for the DAG work. The new DAG functions are added, but not hooked
in yet.

Fixes to the EPP handling have also been added due to issues in
resolving include_lib information

This commit is a transition point that makes some assumptions about new
callbacks for the compiler modules, which will likely not hold when it
comes to making epp be able to read and handle multiple applications at
once (so it can resolve parse transforms and behaviours across OTP
apps).

As such, a follow-up commit is going to be completing that one and
changing its API in incompatible ways; if you find this commit during a
bisect, it's probably not a good one in which to run a bisection.

This is another tricky commit towards replacing the current module
analysis with EPP. The compiler DAG is now being shared across multiple
applications being compiled, rather than a per-application basis, which
promises to allow better ordering, parallelism, and more thorough
invalidation of runtime dependencies when they are modified.

This however required changes:
- The compiler DAG is no longer private to `rebar_compiler`, and has
  been extracted to the `rebar_compiler_dag` module
- The compiler DAG is now started by the `rebar_prv_compile` module,
  which oversees the calls to `rebar_compiler` for each OTP application
- The compiler DAG has been refactored to use a "dirty flag" to know if
  it was modified, rather than just tracking modifications in a
  functional manner, since the scope change (going multi-app) makes it
  impossible to cleanly use the functional approach without much larger
  changes
- The DAG used to be cached within each OTP application. This is no
  longer possible since it is shared. Instead the DAG is stored in the
  state's deps_dir, which allows to cleanly split caches between regular
  apps for the user's project and plugins
- The DAG supported a "label" mode that was used to store distinct DAGs
  for extra_src_dir runs and regular modules; this label is now used
  (and extended to `rebar_prv_compile` internals) to distinguish between
  "compile runs", such as "project_apps", or just "apps" (deps). The
  label is optional (i.e. not used by plugins which have no such need)
- The extra_src_dirs for each app is now compiled using the main app's
  DAG, but the run takes place later in the compilation process. This
  may need changing to detect and prevent dependencies from src_dirs
  into extra_src_dirs, but this should not technically be a problem for
  runtime anyway.
- Reworked the support for extra_src_dirs that are at the root of an
  umbrella project (and therefore do not belong to any single app) to
  use the new structure, also as part of the project_apps DAG.

All tests keep passing, and this puts us in a better place to use EPP
with cross-app support in the near-future.