This will allow project with larger dependencies sets to clean only the
apps they want to when testing or changing small things, rather than
forcing a rebuild of the whole dep set.
Also allows cleaning up apps, not just deps.
When in umbrella mode (or through multiple profiles), users can specify
macros for EDocs based on either the {def, ...} or the {macros, ...}
arguments.
This patch replaces the prior options merging for umbrellas to use the
rebar3 tup_umerge utils to remove identical duplicates while preserving
correct ordering, and manually merges the {macros, ...} definitions
while ke eping the correct precedence rules since these appear (given
their behaviour) to be all individually extracted and passed as `{d,
...}` to the compiler so that epp expands them. This compiler
function freaks out on any re-defined macros and explodes.
Do note that the macros with `{def, ...}` are edoc macros and do not
suffer from that issue, safely deduplicating multiple definitions.
This adds an additional loading and merging of options for EDoc using
the values from the top-level along with those specified in the
rebar.config of an umbrella application.
The app-specific config values are prepended to the global ones; this
can likely cause some problems with manual path handling, but is
unlikely to happen in practice and the rest seems to work fine based on
order
Fixes the issue in #2114
This allows to reduce the number of noise to only checking deps' app
files when they're built, rather than on every run.
Since main apps and checkouts are still compiled every time, the linting
takes place there and then with a higher frequency.
Overrides should apply to a layer below where they are declared. This
patch makes it so if the project root is an application (i.e. it isn't
'root' and therefore not an umbrella project), we omit applying
overrides in rebar_app_discover.
This in turn required changing a bunch of tests, because all the tests
worked with the idea that all overrides applied to all apps to validate
that they get inherited properly. The changes re-structure the cases so
they are written with an umbrella app, demonstrating that the changes
stick.
Deps command shows an * if the local state of the dependencies do not
match the config file, highlighting the differences between the lock
file and the config file if there are any.
We currently have a problem in the handling of locks in the `upgrade
<app>` command when a transitive dependency switches supervision trees.
If we start with:
A
/ \
B1 C
|
D
|
E
And upgrade to:
A
/ \
B2 C
| |
E D
by calling `rebar3 upgrade b`, we would expect the ugprade command to do
so directly. However we are currently left with the following lock file:
A
/ \
B2 C
|
D
|
E
This is _not_ critical since on the next run, the lock file is fixed
automatically through an install-deps sequence. However, this is a
jarring state in which to leave a project.
The reason is that we proceed the following way to handle an upgrade:
1. pick the name of the chosen app, and unlock it if it's at the level 0
2. grab a list of all its current children (and their own) and unlock
them
3. do the same for all the other unlocked dependencies
4. pass this new lock internal format to the get-deps provider in
upgrade mode
5. take the result and save it to the lock file.
What's interesting is that right now this yields invalid locks at
step 3, because we have removed the lock on B1, but not on E
(which we don't yet know is going to be a new child of the next
B version until we fetch it)
In step 4, we fetch dependencies, and in there we compare the
pre-upgrade locks and fetch structure with those expected in the current
lock state, and rebuild everything.
The fix in this commit adds a check there where if the app is in the
current lock set at a level deeper than the current one, the lock is
forced to be refreshed as if it were not there. This lock would get
refreshed on a blank run anyway since the current lock set would start
empty (rather than pre-populated by `upgrade`), and the level-order
traversal ensures that the locks remain safe in all cases.
Fixes#2030
- vendor in hex_core at v0.5.0
- Change where repo_name should be the org and not the parent
- Changed rebar_utils:url_append_path/2 to not explicitly add a '?',
this is returned in the Query chunk by http_uri:parse/1 (e.g., "?foo=bar")
- update organization_merging test to expect the sub-repo as the repo_name
- Add tests for rebar_utils:url_append_path/2
- Stop referencing/setting "organization" in config and use new
organization settings (api_repository and repo_organization)
- Do not set (assume) the read key is valid for any/every repo
- Set repo_organization and api_repository to org
- Update tests to check for new config opts
- Modified rebar_hex_repos:auth_config/1 to abort on error except in
the case of enoent, in which we return an empty map.
- added auth_config_errors test to test/rebar_pkg_repos_SUITE
This change fixes cases where changes in .hrl files would not be picked
up in .erl files that are in extra source directories (such as those
defined with `extra_src_dirs` or modules in the test/ directory during a
CT or Eunit run).
The problem was due to the way the Directed Acyclic Graph (DAG) of
dependencies between files was being loaded and stored by the compiler
modules.
Prior to this fix, a single DAG would be used for all runs. On a regular
run, the prior DAG is loaded from disk, re-checked, and if changed, it
would get re-written to disk with the changes deciding what to
re-compile. However, whenever extra source directories were specified, a
second run would be done which swaps target directories around in the
compiler modules.
Bug 1: this second run was done without properly tracking the private .hrl
files (in src/), so the changes were invisible. This has been fixed by
re-adding the paths.
The problem is that the DAG handling is self-contained; just invoking it
was sufficient to get it loaded and rewritten to disk. But since runs
with extra src dirs were done on different sets, the compilation of
extra src dirs would be done with bad historical data (all the modules
in src/ are dropped, all those in test/ are re-added); this DAG was then
written to disk once again, polluting the next non-extra run.
This is bug 2, and it is fixed by adding an optional label to each run
so that a regular or extra compile round can be distinguished, each
tracking their own files in their own DAG.
A single test (and a lot of diffing) were sufficient for this.
consider A-MIB imports from B-MIB:
rebar compile MUST FAIL if mib_first_files is ["mibs/A-MIB.mib"]
rebar compile MUST SUCCEED if mib_first_files is ["mibs/B-MIB.mib"]
- Added support to rebar_prv_xref for filtering out whole module. A
module may now be added to the list of xref_ignores in rebar.config or
via -ignore_xref() module attribute
- Added unit test for whole module filtering support to xref CT suite
Dollar Variable expansion (`$VAR`) was inadvertently disabled for
windows variables, although %VARIABLES% already worked. This reduced the
portability of hooks in general.
Additionally, tests would fail on windows due to bad quoting of paths:
the path C:/a/b/c would fail when passed to the command
`cmd /q /c C:/a/b/c` because it would interpret /a /b and /c as 3
options. Using quotes makes the tests pass.
- Only set paths that need to be put as a priority
- Clean up paths before leaving API mode
The first point accounted for some performance cost, but the latter one
explains the 40% overhead in test runs: since rebar3 calls rebar3 a lot
with a bunch of fake apps, and that the new mechanism for path handling
by default does not _remove_ paths, it just _orders_ them, we would end
up in a situation where as the tests ran, more and more fake paths would
get added to the VM.
By the time the run was over, all path handling would take longer since
more paths needed filtering every time. By resetting paths at the end of
an API run, we prevent a given 'project' from polluting another one's
runtime and performance once the API successfully returns.
Move path management out of rebar_utils manual code path function
handling (which we leave there for backwards compat), and centralize
them to allow easier coordination of paths between plugins and deps.
On top of path handling, do a check of loaded modules to only purge and
reload those that actually need it done in order to prevent all kinds of
weird interaction and accidental purge kills. It also allows the
possible cohabitation of both at once, with a "in case of conflict pick
X" as a policy
Changing path handling in providers also highlighted a bunch of bugs in
some tests and appears to fix some in other providers, specifically
around plugins.
* add compile type for dynamic project compilation
* new rebar_compiler abstraction for running multiple compilers
rebar_compiler is a new behaviour that a plugin can implement to
be called on any ues of the compile provider to compile source
files and keep track of their dependencies.
* fix check that modules in .app modules list are from src_dirs
* use project_type to find module for building projects
* allow plugins to add project builders and compilers
retired packages are now used the same as any other but
a warning will be printed when it is resolved.
prerelease versions are skipped unless explicitly given
as the version in the constraint or lock file.
When REBAR_CONFIG was set it would not effect the top level app's
configuration because app_discover was rereading the top level
rebar.config which ignored REBAR_CONFIG. Instead this patch has
it use the existing configuration from REBAR_CONFIG.
* update to hex_core for hex-v2 repo support (#1865)
* update to hex_core for hex-v2 repo support
This patch adds only single repo hex-v2 support through hex_core.
Packages no longer filtered out by buildtool metadata and the
package index is updated per-package instead of fetched as one
large ets dump.
* tell travis to also build hex_core branch
* support list of repos for hex packages (#1866)
* support list of repos for hex packages
repos are defined under the hex key in rebar configs. They can be
defined at the top level of a project or globally, but not in
profiles and the repos configured in dependencies are also ignored.
Searching for packages involves first checking for a match in the
local repo index cache, in the order repos are defined. If not found
each repo is checked through the hex api for any known versions of
the package and the first repo with a version that fits the constraint
is used.
* add {repos, replace, []} for overriding the global & default repos
* add hex auth handling for repos (#1874)
auth token are kept in a hex.config file that is modified by the
rebar3 hex plugin.
Repo names that have a : separating a parent and child are considered
organizations. The parent repo's auth will be included with the child.
So an organization named hexpm:rebar3_test will include any hexpm
auth tokens found in the rebar3_test organization's configuration.
* move packages to top level of of hexpm cache dir (#1876)
* move packages to top level of of hexpm cache dir
* append organization name to parent's repo_url when parsing repos
* only eval config scripts and apply overrides once per app (#1879)
* only eval config scripts and apply overrides once per app
* move new resource behaviour to rebar_resource_v2 and keep v1
* cleanup use of rebar_resource module and unused functions
* cleanup error messages and unused code
* when discovering apps support mix packages as unbuilt apps (#1882)
* use hex_core tarball unpacking support in pkg resource (#1883)
* use hex_core tarball unpacking support in pkg resource
* ignore etag if package doesn't exist and delete if checksum fails
* add back tests for bad package checksums
* improve bad registry checksum error message
This would cause crashes on linux and force people to have a src_dirs
config that is strictly matching what is on the file system rather than
acting as a specification of those that are valid.
To compare, if lib_dirs worked the same, then any repo that did not both
have apps/ and lib/ would crash, as the spec mentions both options as
valid.
Sergey Yelin
Fred Hebert
Guilherme Andrade
Tristan Sloughter
Bryan Paxton
Pierre Fenoll
Tino Breddin
Justin Wood
Jose M Perez
Bryan Paxton
Eric Meadows-Jönsson
mopp
Stefan Grundmann
tothlac
Bryan Paxton
Alexander Petrovsky
Dmitry