The propagation was confusing source files and artifacts; the artifact
ordering was flipped, and the tagging non-mandatory (aside from as
edges), which made things hard to identify when plugins for compilers
are used.
This allows to do quicker re-compile option validation by not requiring
to access the disk and check all candidate erlang files for option
changes. This also opens up the way to compilers that produce more than
one artifact per file.
This moves the old DAG stuff out from the main code paths and
substitutes it with the new epp-based DAG work.
Also fixes previous work for integration tests:
- consider ptrans from erl_opts in app ordering
- display proper app compilation info messages due to reordering
While leaving the old one in place, prep the ground for new analysis
phases for the DAG work. The new DAG functions are added, but not hooked
in yet.
Fixes to the EPP handling have also been added due to issues in
resolving include_lib information
This commit is a transition point that makes some assumptions about new
callbacks for the compiler modules, which will likely not hold when it
comes to making epp be able to read and handle multiple applications at
once (so it can resolve parse transforms and behaviours across OTP
apps).
As such, a follow-up commit is going to be completing that one and
changing its API in incompatible ways; if you find this commit during a
bisect, it's probably not a good one in which to run a bisection.
This patch does two things:
1. it broadens the interface for the compiler module so that
non-first-file modules can possibly be parallelized. This is done by
dynamically switching on `[ListOfFiles]`, which remains sequential as
before, or `{[SeqPriority], [Parallel]}`, which divides regular files
between higher priority ones and those that can be in parallel
2. implements this mechanism in the rebar compiler, based on the erl
file digraph. If a file has an in-neighbour, it is depended on by
another file. The mechanism therefore makes it so all files that have
dependants get compiled in their strict relative sequential order
first, and then the undepended-on files get compiled together in
parallel.
By running:
./rebar3 ct --suite test/rebar_compile_SUITE.erl --case \
recompile_when_parse_transform_inline_changes --repeat 50
the previous iteration of this would rapidly fail, and this one succeeds
every time.
When the commit at
8c4a74a3ed
introduced a recursive search for build elements, it accidentally did so
using a function that accepts a regex for its match.
In doing so, if a behaviour or include had a name such as
`common_prefix` and that multiple other modules used the name
`common_prefix_<rest_of_name>`, all the other files would be seen as
dependencies to be checked in a directed graph.
This resulted in continuous re-checking of files that kept depending on
each others and blew the compile time up exponentially.
By using a delimitation on the regex (^<modulename>$), the build comes
back down to finding only the right files and becomes fast again.
* add compile type for dynamic project compilation
* new rebar_compiler abstraction for running multiple compilers
rebar_compiler is a new behaviour that a plugin can implement to
be called on any ues of the compile provider to compile source
files and keep track of their dependencies.
* fix check that modules in .app modules list are from src_dirs
* use project_type to find module for building projects
* allow plugins to add project builders and compilers