Deps command shows an * if the local state of the dependencies do not
match the config file, highlighting the differences between the lock
file and the config file if there are any.
This patch does two things:
1. it broadens the interface for the compiler module so that
non-first-file modules can possibly be parallelized. This is done by
dynamically switching on `[ListOfFiles]`, which remains sequential as
before, or `{[SeqPriority], [Parallel]}`, which divides regular files
between higher priority ones and those that can be in parallel
2. implements this mechanism in the rebar compiler, based on the erl
file digraph. If a file has an in-neighbour, it is depended on by
another file. The mechanism therefore makes it so all files that have
dependants get compiled in their strict relative sequential order
first, and then the undepended-on files get compiled together in
parallel.
By running:
./rebar3 ct --suite test/rebar_compile_SUITE.erl --case \
recompile_when_parse_transform_inline_changes --repeat 50
the previous iteration of this would rapidly fail, and this one succeeds
every time.
Support for parallel compilation of *.erl file was dropped before 3.0 release.
However, our tests for a project containing ~500 source files show substantial gain, lowering compilation time from 58 seconds to 18 on a MacBook Pro 15" (4 cores, 8 threads), and to just 10 seconds on Xeon-D machine.
Previously all escriptize hooks from all apps in the project were
executed, which could lead to issue in case of dependencies on artifacts
which are created by these hooks.
Now the behaviour is similar to the handling of other non-global hooks.
We currently have a problem in the handling of locks in the `upgrade
<app>` command when a transitive dependency switches supervision trees.
If we start with:
A
/ \
B1 C
|
D
|
E
And upgrade to:
A
/ \
B2 C
| |
E D
by calling `rebar3 upgrade b`, we would expect the ugprade command to do
so directly. However we are currently left with the following lock file:
A
/ \
B2 C
|
D
|
E
This is _not_ critical since on the next run, the lock file is fixed
automatically through an install-deps sequence. However, this is a
jarring state in which to leave a project.
The reason is that we proceed the following way to handle an upgrade:
1. pick the name of the chosen app, and unlock it if it's at the level 0
2. grab a list of all its current children (and their own) and unlock
them
3. do the same for all the other unlocked dependencies
4. pass this new lock internal format to the get-deps provider in
upgrade mode
5. take the result and save it to the lock file.
What's interesting is that right now this yields invalid locks at
step 3, because we have removed the lock on B1, but not on E
(which we don't yet know is going to be a new child of the next
B version until we fetch it)
In step 4, we fetch dependencies, and in there we compare the
pre-upgrade locks and fetch structure with those expected in the current
lock state, and rebuild everything.
The fix in this commit adds a check there where if the app is in the
current lock set at a level deeper than the current one, the lock is
forced to be refreshed as if it were not there. This lock would get
refreshed on a blank run anyway since the current lock set would start
empty (rather than pre-populated by `upgrade`), and the level-order
traversal ensures that the locks remain safe in all cases.
Fixes#2030
- vendor in hex_core at v0.5.0
- Change where repo_name should be the org and not the parent
- Changed rebar_utils:url_append_path/2 to not explicitly add a '?',
this is returned in the Query chunk by http_uri:parse/1 (e.g., "?foo=bar")
- update organization_merging test to expect the sub-repo as the repo_name
- Add tests for rebar_utils:url_append_path/2
- Stop referencing/setting "organization" in config and use new
organization settings (api_repository and repo_organization)
- Do not set (assume) the read key is valid for any/every repo
- Set repo_organization and api_repository to org
- Update tests to check for new config opts
- Modified rebar_hex_repos:auth_config/1 to abort on error except in
the case of enoent, in which we return an empty map.
- added auth_config_errors test to test/rebar_pkg_repos_SUITE
by default, all apps except internal rebar3 ones can be blacklisted.
Some months ago, someone added support for configurable lists within
rebar3 itself using the .app env.
This PR re-exports that functionality from the rebar.config file, so
that one can set something like:
{shell, [
{app_reload_blacklist, [cowboy, ranch]}
]}.
Which will allow to prevent applications that often crash when being
reloaded from doing so. For example, cowboy and ranch processes can be
stuck in an accept call for multiple reloads in dev, which ends up
causing large failures.
This change fixes cases where changes in .hrl files would not be picked
up in .erl files that are in extra source directories (such as those
defined with `extra_src_dirs` or modules in the test/ directory during a
CT or Eunit run).
The problem was due to the way the Directed Acyclic Graph (DAG) of
dependencies between files was being loaded and stored by the compiler
modules.
Prior to this fix, a single DAG would be used for all runs. On a regular
run, the prior DAG is loaded from disk, re-checked, and if changed, it
would get re-written to disk with the changes deciding what to
re-compile. However, whenever extra source directories were specified, a
second run would be done which swaps target directories around in the
compiler modules.
Bug 1: this second run was done without properly tracking the private .hrl
files (in src/), so the changes were invisible. This has been fixed by
re-adding the paths.
The problem is that the DAG handling is self-contained; just invoking it
was sufficient to get it loaded and rewritten to disk. But since runs
with extra src dirs were done on different sets, the compilation of
extra src dirs would be done with bad historical data (all the modules
in src/ are dropped, all those in test/ are re-added); this DAG was then
written to disk once again, polluting the next non-extra run.
This is bug 2, and it is fixed by adding an optional label to each run
so that a regular or extra compile round can be distinguished, each
tracking their own files in their own DAG.
A single test (and a lot of diffing) were sufficient for this.
Allows to select one of many releases conditionally.
Default behaviour should be unchanged from existing one.
Can filter on only the version or only the name, or both.
Starting with OTP-21.2.3, some output appears to be sent to the shell
while we're taking down the TTY, which ends up stalling the whole log
flow for the default handler. I don't have proof but suspect this is due
to the system logging going directly from the ERTS runtime to the
logger, which may be trigger when we kill the TTY driver.
This patch makes it so whenever we detect that logger is active, we save
the default config, disable the default logger, and only then we can
kill the TTY driver. Once the driver is started and all the group
leaders have been rewritten, the default logger is re-added.
This all takes place _before_ the logger configuration is updated as
part of booting apps, and the change should be fully backwards
compatible.
When the commit at
8c4a74a3ed
introduced a recursive search for build elements, it accidentally did so
using a function that accepts a regex for its match.
In doing so, if a behaviour or include had a name such as
`common_prefix` and that multiple other modules used the name
`common_prefix_<rest_of_name>`, all the other files would be seen as
dependencies to be checked in a directed graph.
This resulted in continuous re-checking of files that kept depending on
each others and blew the compile time up exponentially.
By using a delimitation on the regex (^<modulename>$), the build comes
back down to finding only the right files and becomes fast again.
Benedikt Reinartz
Fred Hebert
Jose M Perez
Maxim Fedorov
Tino Breddin
Justin Wood
Piotr Bober
Tristan Sloughter
Bryan Paxton
Bryan Paxton
Josh Soref
Amelia Bellamy-Royds