Co-authored-by: Fred Hebert <mononcqc@ferd.ca>

This commit changes rebar3 to check for name clashes of the
modules created from its templates with existing Erlang modules
to produce a non-blocking warning for each clashing module.

This is required for compile order to be maintained in a post-3.14
world.

... and e.g. an unknown type is detected

This commit changes mainly the `rebar_paths` module by adding a
new `runtime` target. When using that target, `rebar_paths:get_apps/2`
will, for each project app, find all their `applications` and
`included_applicactions` and filter those that could be found
in rebar3's state.

This was due to a subtle mistake in the new code in
https://github.com/erlang/rebar3/pull/2322 which optimized the search
behaviour of files not found by EPP. The patch mistakenly always kept
the top-level src/ directory for an app even if it did a lookup in
recursive ones.

This patch tracks and maintains the proper subdirectory hierarchy. To
keep performance adequate, the lists:keyfind/3 function is used, which
is also now a NIF to the same extent as lists:member/2; manual lookups
would likely end up reducing performance, particularly in deep
hierarchies.

A test case has been added to track the regression.
Reported by @elbrujohalcon

Saves a few seconds when loading DAG with thousands of files.
Unfortunately, uses internal knowledge about digraph implementation.

Instead of sequentially running *.erl (or any other) files
one-by-one, spawn a separate process, in a scatter-gather way,
then wait for all processes to complete.
Combined with the previous change it introduces a significant
speed-up for large repositories, up to a number of cores that
the host has.

This commit makes rebar_compiler_erl to start a gen_server that acts
as a filesystem cache, avoiding re-scanning it over and over.

rebar3 would crash if the option `{keep_logs, N}` was present and
there were more than N run directories in the log dir, so if the
option is found we keep the N-1 newest directories and delete the
rest.

While we are at it, as this should fix the crash, let's limit the
number of logs runs to keep from test runs in rebar3 to 5.

Modify also the load_file function by adding a
way to check an error, and abort the program
smoothly.

Co-authored-by: Fred Hebert <mononcqc@ferd.ca>

this fixes the issue where using 'rebar3 plugins upgrade P'
would result in a plugin's .app file having an empty
modules list. The code this replaces hadn't been touched
since rebar3 3.0.0-beta.3 and a lot of improvements have
gone into dep handling and the compiler since then. This
change should take advantage of those changes.

This patch contains two behaviour changes and reasserts other behaviours
that now line things up with user and documentation expectations:

1. The src directories remain recursive. We turned it on by accident at
   some point in the past and now people rely on it, so we're stuck with
   it. However a new test ensures that the feature can be turned off
   on-demand as documented on the website.

2. The test directories are no longer recursive by default. The fix is
   done by properly fixing how rebar3.erl does its feature injection by
   mandating the default value there.
   I'm somewhat nervous that this change could negatively impact some
   users and older compiler module versions, but if users stick to the
   rebar_dir interface, everything should keep working transparently.

3. The test directories' configuration is no longer silently dropped.
   Due to how rebar3.erl injected test state without looking for what
   the user may have specified, multiple extra_src_dirs entries existed
   at once and were run; one with the recursion set to true and one with
   whatever the user specified. If the user disabled recursion of the
   "test" extra_src_dir, then the injected value still ran it once...

4. The handling of extra files in the compiler module is fixed to
   actually use the rebar_dir interface properly, and reinjects
   non-default directory recursion settings into the swapped options
   for the shimmed extra apps. Not doing this annotation step resulted
   in the write for swapped opts to actually drop the configured
   recursion value and make everything recursive all the time.

A single new test actually validates all of that behaviour and seems to
work fine.

This commit changes the location where rebar3 will extract
its libs upon calling a `local` command from `~/.cache/rebar3/lib`
to `~/.cache/rebar3/vsns/VERSION/lib`, with `VERSION` being
rebar3 current version.

It also updates the bin script to reflect these changes.

validate_app in rebar_otp_app does not need to set the vsn and
original_vsn because they are set from the app details during
discovery.

The original_vsn was set here before becuase when there is no
.app.src or .app.src.script the app file does not go through
preprocess/3 and preprocess/3 is where vsn is set based on evaluating
the vsn from .app.src. But if neither of these files exist it
means the discovery vsn was the correct version and there is
no need to update.

As reported by Alexey Pronin, this would let people potentially pass in
URLs that can call to command execution.

While running arbitrary code is kind of our bread and butter, it makes
sense to at least play nicely and escape more characters:

- the pipe (`|`) which can be used to run subcommands
- the tab (`\t`) which the shell interprets as a space.
- the home directory (`~`) just because the expansion could behave funny
- redirection (`<` and `>`) to avoid problems with data setting up
  streams

Before, running `rebar3 ct` on such an empty project would crash with an
internal badmatch since no directory to run CT on could be found.

Now this case will terminate with a sensible error message and won't try
to run any further.

Commit 029cbcc7 changed the return value of `rebar_git_resource:lock_/2`

None of the version checks are considered safe enough by mix
maintainers, and so the agreed upon mechanism is to just set this
environment variable, which will be supported implicitly by rebar3
versions that can handle it.

This lets build artifacts (aside from some priv/ issues with plugins) to
properly be located within their build folder rather than the deps
folder, as the --outdir switch would otherwise have done.