The propagation was confusing source files and artifacts; the artifact
ordering was flipped, and the tagging non-mandatory (aside from as
edges), which made things hard to identify when plugins for compilers
are used.
This allows to do quicker re-compile option validation by not requiring
to access the disk and check all candidate erlang files for option
changes. This also opens up the way to compilers that produce more than
one artifact per file.
This moves the old DAG stuff out from the main code paths and
substitutes it with the new epp-based DAG work.
Also fixes previous work for integration tests:
- consider ptrans from erl_opts in app ordering
- display proper app compilation info messages due to reordering
While leaving the old one in place, prep the ground for new analysis
phases for the DAG work. The new DAG functions are added, but not hooked
in yet.
Fixes to the EPP handling have also been added due to issues in
resolving include_lib information
This commit is a transition point that makes some assumptions about new
callbacks for the compiler modules, which will likely not hold when it
comes to making epp be able to read and handle multiple applications at
once (so it can resolve parse transforms and behaviours across OTP
apps).
As such, a follow-up commit is going to be completing that one and
changing its API in incompatible ways; if you find this commit during a
bisect, it's probably not a good one in which to run a bisection.
This is another tricky commit towards replacing the current module
analysis with EPP. The compiler DAG is now being shared across multiple
applications being compiled, rather than a per-application basis, which
promises to allow better ordering, parallelism, and more thorough
invalidation of runtime dependencies when they are modified.
This however required changes:
- The compiler DAG is no longer private to `rebar_compiler`, and has
been extracted to the `rebar_compiler_dag` module
- The compiler DAG is now started by the `rebar_prv_compile` module,
which oversees the calls to `rebar_compiler` for each OTP application
- The compiler DAG has been refactored to use a "dirty flag" to know if
it was modified, rather than just tracking modifications in a
functional manner, since the scope change (going multi-app) makes it
impossible to cleanly use the functional approach without much larger
changes
- The DAG used to be cached within each OTP application. This is no
longer possible since it is shared. Instead the DAG is stored in the
state's deps_dir, which allows to cleanly split caches between regular
apps for the user's project and plugins
- The DAG supported a "label" mode that was used to store distinct DAGs
for extra_src_dir runs and regular modules; this label is now used
(and extended to `rebar_prv_compile` internals) to distinguish between
"compile runs", such as "project_apps", or just "apps" (deps). The
label is optional (i.e. not used by plugins which have no such need)
- The extra_src_dirs for each app is now compiled using the main app's
DAG, but the run takes place later in the compilation process. This
may need changing to detect and prevent dependencies from src_dirs
into extra_src_dirs, but this should not technically be a problem for
runtime anyway.
- Reworked the support for extra_src_dirs that are at the root of an
umbrella project (and therefore do not belong to any single app) to
use the new structure, also as part of the project_apps DAG.
All tests keep passing, and this puts us in a better place to use EPP
with cross-app support in the near-future.