Deps command shows an * if the local state of the dependencies do not
match the config file, highlighting the differences between the lock
file and the config file if there are any.
This patch does two things:
1. it broadens the interface for the compiler module so that
non-first-file modules can possibly be parallelized. This is done by
dynamically switching on `[ListOfFiles]`, which remains sequential as
before, or `{[SeqPriority], [Parallel]}`, which divides regular files
between higher priority ones and those that can be in parallel
2. implements this mechanism in the rebar compiler, based on the erl
file digraph. If a file has an in-neighbour, it is depended on by
another file. The mechanism therefore makes it so all files that have
dependants get compiled in their strict relative sequential order
first, and then the undepended-on files get compiled together in
parallel.
By running:
./rebar3 ct --suite test/rebar_compile_SUITE.erl --case \
recompile_when_parse_transform_inline_changes --repeat 50
the previous iteration of this would rapidly fail, and this one succeeds
every time.
Support for parallel compilation of *.erl file was dropped before 3.0 release.
However, our tests for a project containing ~500 source files show substantial gain, lowering compilation time from 58 seconds to 18 on a MacBook Pro 15" (4 cores, 8 threads), and to just 10 seconds on Xeon-D machine.
Previously all escriptize hooks from all apps in the project were
executed, which could lead to issue in case of dependencies on artifacts
which are created by these hooks.
Now the behaviour is similar to the handling of other non-global hooks.
We currently have a problem in the handling of locks in the `upgrade
<app>` command when a transitive dependency switches supervision trees.
If we start with:
A
/ \
B1 C
|
D
|
E
And upgrade to:
A
/ \
B2 C
| |
E D
by calling `rebar3 upgrade b`, we would expect the ugprade command to do
so directly. However we are currently left with the following lock file:
A
/ \
B2 C
|
D
|
E
This is _not_ critical since on the next run, the lock file is fixed
automatically through an install-deps sequence. However, this is a
jarring state in which to leave a project.
The reason is that we proceed the following way to handle an upgrade:
1. pick the name of the chosen app, and unlock it if it's at the level 0
2. grab a list of all its current children (and their own) and unlock
them
3. do the same for all the other unlocked dependencies
4. pass this new lock internal format to the get-deps provider in
upgrade mode
5. take the result and save it to the lock file.
What's interesting is that right now this yields invalid locks at
step 3, because we have removed the lock on B1, but not on E
(which we don't yet know is going to be a new child of the next
B version until we fetch it)
In step 4, we fetch dependencies, and in there we compare the
pre-upgrade locks and fetch structure with those expected in the current
lock state, and rebuild everything.
The fix in this commit adds a check there where if the app is in the
current lock set at a level deeper than the current one, the lock is
forced to be refreshed as if it were not there. This lock would get
refreshed on a blank run anyway since the current lock set would start
empty (rather than pre-populated by `upgrade`), and the level-order
traversal ensures that the locks remain safe in all cases.
Fixes#2030
Fred Hebert
Pierre Fenoll
Tino Breddin
Justin Wood
Tristan Sloughter
Jamu Kakar
Benedikt Reinartz
Jose M Perez
Maxim Fedorov
Piotr Bober