When a config file exists at the root of a project, defines a given
configuration value for a given profile, and that a sub-application
(umbrella app) also has the same profile defined with the same key (but
different values), the configuration values of the sub-application's
profile would get silently dropped.
The problem being that when the function to merge profiles is applied
recursively, it is applied to each profile (so it will merge on the keys
test, prod, etc.) rather than to each of the values of each profile.
This patch reworks the profile merging so that the current behaviour is
respected overall (a profile cannot be cancelled by a subdep's
non-existant profile since its value should have been ignored), but
ensures that sub-deps' profiles are otherwise applied recursively with
the proper rules:
- dependencies favor prior values
- plugins favor new values
- erl_first_files combine the lists
- relx uses the tuple merge algorithm
- erl_opts has its own custom merge as well
- otherwise the new value takes precedence
A test has also been added.
There is a risk of breakage in some applications that may have relied on
the buggy behaviour to work, though at this time we are aware of none of
them.
This is done through 3 main change groups:
- replacing `~s` by `~ts` in format strings, so that strings that
contain unicode are properly printed rather than crashing
- adding the `unicode` argument to all function of the `re` module to
ensure transformations on strings containing unicode data are valid
instead of crashing (see issue #1302)
- replacing `ec_cnv:to_binary/1` and `ec_cnv:to_list/1` with matching
functions in `rebar_utils`.
The last point has been done, rather than modifying and updating erlware
commons, because binary and list conversions can be a contentious
subject. For example, if what is being handled is actually bytes from a
given binary stream, then forcing a byte-oriented interpretation of the
data can corrupt it. As such, it does not appear safe to modify erlware
commons' conversion functions since it may not be safe for all its
users.
Instead, rebar3 reimplements a subset of them (only converting
atoms and chardata, ignoring numbers) with the explicit purpose of
handling unicode string data.
Tests were left as unchanged as possible. This may impact the ability to
run rebar3's own suites in a unicode path, but respects a principle of
least change for such a large patch.
This can be reproduced by running xref analysis against a rebar3
plugin project which doesn't list rebar3 as an explicit dependency
-- calls to certain bundled modules ('rebar_state', 'rebar_api',
'ec_cnv', ...) will result in a failed pattern match as these modules
appear to have had their abstract code stripped.
The use case has been described in issue #1478 where a local application
can exist while being declared as a dependency as well. This allows, for
example, to work on a release where all applications may require to be
published independently, or to provide some form of 'vendoring' with a
local app.
The fix is done by decoupling the dependency source resolution form the
dependency parsing. The reason for this being that the discovery phase
needs to parse apps for their top-level deps, and dep installation needs
to resolve the packages with accuracy. In the current implementation,
both code paths call to the same function.
This patch splits up the precise discovery and makes it happen *only*
when installing dependencies, and only if a top-level app does not
already define the application needing resolving.
One weakness of this fix is that it necessarily breaks cycle detection
in dependencies that involve a root application depending on itself
since its own version as a dep will not be expanded. There appears to be
no possible way to prevent this, but should be rare enough to be worth
the tradeoff for the common case.
Similarly to PATH env variable, this allows to pass paths to bare
compiler which do not fit nicely into a single wildcard structure.
Colon (":") is used as separator.
This provides more flexibility when rebar is run in offline/hermetic
environment, e.g. #958 and #1281.
I noticed this when trying to include entop in a release. Entop uses
cecho which takes over the terminal, so you do not want it loaded or
started. According to http://erlang.org/doc/man/rel.html, when you
specify a Type of none it should not load or start, but the code for
it's modules should be loaded. This patch ensures the code is not
loaded or started, but doesn't do anything with the code paths.
At the very least this allows me to start a shell in the case where
I have an application of type none, and the application is neither
loaded nor started.
When a given behaviour module does not exist, rebar3 brutally crashes.
This patch makes it so instead, there is a warning output menitoning the
missing behaviour, and this one is omitted from the unused function
calls check. This means that unused calls will instead be shown for the
module implementing a non-existing behaviour, as if no behaviour were
declared in the first place.
If xref analysis is being run by a dependency during its compilation
phase, the xref provider will try to add that deps' parents to the check
job while the ebin/ directories for them do not exist. This causes a
crash.
This patch makes it so directories are only added if they are existing,
preventing failure on any compile hook for dependencies and allowing
successful compilation as a best effort.
Patches:
- Try to start epmd when distribution fails
- Allow non-default profile deps upgrade
- Switch default license in templates to Apache 2.0, and hook it up in
the .app file
- Fix git SHA comparisons for dep update
- Let relx know about rebar-specified erl_opts
- Use rm_rf instead of ec_file:delete, fixing some deps-related issues
on Windows
- Handle internal CT failures
- Fix profile merging of erl_first_files options
- Fix CT output in skipped test cases and failing end of function
hooks
- Officially drop R15 support, since its SSL libs can no longer talk
to hex.pm
- REBAR_COLOR supports all ec_cmd_log intensity options, allowing the
none value to disable all colors without dropping termcap support
- Bump relx to 3.22.4
The one new feature supported here is for the interactive shell:
- Support commands in the shell such as r3:Command(),
r3:Command("--args=as a string"), or r3:Command(Namespace, "--args=as a
string"), instead of just r3:do(Command).
Prevents showing false upgrade messages when upgrading in a non-default
profile. Also shows an info message explaining default deps don't get
updated in a non-default upgrade.
In contrast to `ec_file:delete`, `rebar_file_utils:rm_rf` will also delete
write-protected files on Windows which is needed for git object files.
Fixes#1483.
This is based on issue #1517 where out of nowhere, CT has returned a
user's error code directly. This in turn caused a crashdump in rebar3
itself.
This patch handles the unexpected cases by:
a) not trying to format them
b) converting them to an error whenever they happen
The execution flow is still interrupted, but we should fail with a
clearer error than a crashdump.
The order of the files listed in "erl_files_first" option is
important and should not be sorted. By handling the merge
similarly to mib_files_first, the sort order is preserved.
It can't even fetch packages from Hex anymore because of old SSL/TLS
libraries, and so it can't bootstrap anymore either. Plus R20 comes out
soon, and 5 major versions is quite enough.
This uses the very risky '$handle_undefined_function'/2 export from the
r3 and rebar_agent modules to allow meta-calls that can support plugins
and all other rebar3 extensions.
This is nasty but very tempting. Currently we only support:
- r3:do(Command)
- r3:do(Namespace, Command)
There is currently no way to pass arguments to the function such that we
can, for example, run cover analysis or tests on a subset of suites.
With the new abuse of '$handle_undefined_function'/2, we can detect the
unused commands (since they are not exported) and re-route them:
- r3:Command()
- r3:Command("--args=as a string")
- r3:Command(Namespace, "--args=as a string")
Of course, in doing so, we make it impossible to use the 'do' provider
(as in 'rebar3 do ct -c, cover') since the 'do' function is already
required for things to work.
Since the previous function had very strict guards, we can, without
conflict, add manual overrides that simulate the meta-calls fine.
Sample run:
https://gist.github.com/ferd/2c06d59c7083c146d25e4ee301de0073
86e883b8d8 always returns the full length SHA,
therefore when using a dependency having the short SHA,
it'll always consider that the SHAs are different,
hence it'll alway return true for .
The option {spec,Specs} is allowed in ct_opts, but
rebar_prv_common_test:test_dirs did not take into account that Specs
could also be a string only, i.e. not a list of strings.
rebar_file_utils:cp_r uses rebar_utils:escape_chars to ensure that the file names are safe to use, but it doesn't escape double and single quotes. If the file name includes those characters, they disappear when the shell processes them and we get "file not found" errors.
The main culprit here is eunit, that creates reports whose names are `TEST-file_"myfile.app".xml`, and I wish it didn't but I think escape_chars should still do its job all the way.
Fred Hebert
Guilherme Andrade
Mikhail Kalashnikov
Sam Sawan
Tristan Sloughter
Gleb Peregud
Anthony Molinaro
getong
Benedikt Reinartz
Luis Rascao
Thomas O'Dowd
Michiel Beijen
Alin Popa
Vlad Dumitrescu
Siri Hansen
Carlo Carraro
Juan Facorro