The parsing functions were used inconsistently, and the returned values
were bad in some clauses; things only worked because they are never used
within rebar3. This ensures the return types are consistent on all
clauses.
The previous patch at #7c959cc fixed the usage of duplicate values
for directories through relative paths, but mistakenly went overboard
and dropped the `./` path, which is still fairly common. Similarly for
`../".
The code is modified to special-case such values and keep the code
working.
When fetching src_dir values, some relative paths can be inserted. When
deduplicating the paths on the fetch, this fact means that logically
duplicate (but literally different) directories can be returned at once.
By normalizing the names, duplication bugs can be resolved.
Specifically, this impacts profiles. It appears that relx as a whole
requires its configuration to be merged in one tuple order (New takes
precedence over Old), whereas the overlays require the opposite (Old
takes precedence over New) since the operation order on disk is
important to work well.
This patch reorders overlay values such that the overlay of a profile
takes place *after* the basic overlay, ensuring that the profile actions
take place after the basic ones; this allows profiles to properly
overwrite files as expected (see #1609)
This is done while adequately maintaining the order of operations that
were required as part of #1563
Overlay vars of profiles are also checked to be working fine, along with
a test.
This fixes#1247 and #1609
When a config file exists at the root of a project, defines a given
configuration value for a given profile, and that a sub-application
(umbrella app) also has the same profile defined with the same key (but
different values), the configuration values of the sub-application's
profile would get silently dropped.
The problem being that when the function to merge profiles is applied
recursively, it is applied to each profile (so it will merge on the keys
test, prod, etc.) rather than to each of the values of each profile.
This patch reworks the profile merging so that the current behaviour is
respected overall (a profile cannot be cancelled by a subdep's
non-existant profile since its value should have been ignored), but
ensures that sub-deps' profiles are otherwise applied recursively with
the proper rules:
- dependencies favor prior values
- plugins favor new values
- erl_first_files combine the lists
- relx uses the tuple merge algorithm
- erl_opts has its own custom merge as well
- otherwise the new value takes precedence
A test has also been added.
There is a risk of breakage in some applications that may have relied on
the buggy behaviour to work, though at this time we are aware of none of
them.
During the building of escripts, multiple passes are done. Two of them
may end up duplicating content: one that gathers all of the beam files
that will be needed for the app to work, and a second one that goes over
the ebin/ directory of the root application to grab all the stuff in
there, prior to the run.
This allows to grab whatever could be required for runtime without
breaking the rest (or so I think), and sticks them at the front of the
archive, where it needs to sit for things to work fine.
Whenever the ebin/ directory contains a pre-compile .beam file, it gets
fetched both from the first pass described and the latter one. This
results in duplicate entries in the resulting zip files used for the
escript and makes the executable larger than it needs to be.
The patch is a simple 1:1 removal of duplicate values. Since
large pre-populated ebin/ directories are pretty rare, this should not
be too costly for the vast majority of users.
Fixes#1577
The code for symlink handling failed whenever a win32 platform with no
symlink capability would be detected. This patch is provided by
William H from the support ticket at
http://www.rebar3.org/v3/discuss/598225c90365bb00144bc07f, which
adds detection of failures in non-NTFS scenarios on Win32, and then
copies files instead of bailing out.
The end result should be appropriate support for such a platform.
This is done through 3 main change groups:
- replacing `~s` by `~ts` in format strings, so that strings that
contain unicode are properly printed rather than crashing
- adding the `unicode` argument to all function of the `re` module to
ensure transformations on strings containing unicode data are valid
instead of crashing (see issue #1302)
- replacing `ec_cnv:to_binary/1` and `ec_cnv:to_list/1` with matching
functions in `rebar_utils`.
The last point has been done, rather than modifying and updating erlware
commons, because binary and list conversions can be a contentious
subject. For example, if what is being handled is actually bytes from a
given binary stream, then forcing a byte-oriented interpretation of the
data can corrupt it. As such, it does not appear safe to modify erlware
commons' conversion functions since it may not be safe for all its
users.
Instead, rebar3 reimplements a subset of them (only converting
atoms and chardata, ignoring numbers) with the explicit purpose of
handling unicode string data.
Tests were left as unchanged as possible. This may impact the ability to
run rebar3's own suites in a unicode path, but respects a principle of
least change for such a large patch.
This can be reproduced by running xref analysis against a rebar3
plugin project which doesn't list rebar3 as an explicit dependency
-- calls to certain bundled modules ('rebar_state', 'rebar_api',
'ec_cnv', ...) will result in a failed pattern match as these modules
appear to have had their abstract code stripped.
The use case has been described in issue #1478 where a local application
can exist while being declared as a dependency as well. This allows, for
example, to work on a release where all applications may require to be
published independently, or to provide some form of 'vendoring' with a
local app.
The fix is done by decoupling the dependency source resolution form the
dependency parsing. The reason for this being that the discovery phase
needs to parse apps for their top-level deps, and dep installation needs
to resolve the packages with accuracy. In the current implementation,
both code paths call to the same function.
This patch splits up the precise discovery and makes it happen *only*
when installing dependencies, and only if a top-level app does not
already define the application needing resolving.
One weakness of this fix is that it necessarily breaks cycle detection
in dependencies that involve a root application depending on itself
since its own version as a dep will not be expanded. There appears to be
no possible way to prevent this, but should be rare enough to be worth
the tradeoff for the common case.
Fred Hebert
Tristan Sloughter
Sean Cribbs
Jimmy Zöger
suexcxine
getong
patrick cieplak
alisdair sullivan
Vitor Enes Duarte
William H
Guilherme Andrade