Modify also the load_file function by adding a
way to check an error, and abort the program
smoothly.

Co-authored-by: Fred Hebert <mononcqc@ferd.ca>

this fixes the issue where using 'rebar3 plugins upgrade P'
would result in a plugin's .app file having an empty
modules list. The code this replaces hadn't been touched
since rebar3 3.0.0-beta.3 and a lot of improvements have
gone into dep handling and the compiler since then. This
change should take advantage of those changes.

This patch contains two behaviour changes and reasserts other behaviours
that now line things up with user and documentation expectations:

1. The src directories remain recursive. We turned it on by accident at
   some point in the past and now people rely on it, so we're stuck with
   it. However a new test ensures that the feature can be turned off
   on-demand as documented on the website.

2. The test directories are no longer recursive by default. The fix is
   done by properly fixing how rebar3.erl does its feature injection by
   mandating the default value there.
   I'm somewhat nervous that this change could negatively impact some
   users and older compiler module versions, but if users stick to the
   rebar_dir interface, everything should keep working transparently.

3. The test directories' configuration is no longer silently dropped.
   Due to how rebar3.erl injected test state without looking for what
   the user may have specified, multiple extra_src_dirs entries existed
   at once and were run; one with the recursion set to true and one with
   whatever the user specified. If the user disabled recursion of the
   "test" extra_src_dir, then the injected value still ran it once...

4. The handling of extra files in the compiler module is fixed to
   actually use the rebar_dir interface properly, and reinjects
   non-default directory recursion settings into the swapped options
   for the shimmed extra apps. Not doing this annotation step resulted
   in the write for swapped opts to actually drop the configured
   recursion value and make everything recursive all the time.

A single new test actually validates all of that behaviour and seems to
work fine.

This commit changes the location where rebar3 will extract
its libs upon calling a `local` command from `~/.cache/rebar3/lib`
to `~/.cache/rebar3/vsns/VERSION/lib`, with `VERSION` being
rebar3 current version.

It also updates the bin script to reflect these changes.

validate_app in rebar_otp_app does not need to set the vsn and
original_vsn because they are set from the app details during
discovery.

The original_vsn was set here before becuase when there is no
.app.src or .app.src.script the app file does not go through
preprocess/3 and preprocess/3 is where vsn is set based on evaluating
the vsn from .app.src. But if neither of these files exist it
means the discovery vsn was the correct version and there is
no need to update.

As reported by Alexey Pronin, this would let people potentially pass in
URLs that can call to command execution.

While running arbitrary code is kind of our bread and butter, it makes
sense to at least play nicely and escape more characters:

- the pipe (`|`) which can be used to run subcommands
- the tab (`\t`) which the shell interprets as a space.
- the home directory (`~`) just because the expansion could behave funny
- redirection (`<` and `>`) to avoid problems with data setting up
  streams

Before, running `rebar3 ct` on such an empty project would crash with an
internal badmatch since no directory to run CT on could be found.

Now this case will terminate with a sensible error message and won't try
to run any further.

Commit 029cbcc7 changed the return value of `rebar_git_resource:lock_/2`

None of the version checks are considered safe enough by mix
maintainers, and so the agreed upon mechanism is to just set this
environment variable, which will be supported implicitly by rebar3
versions that can handle it.

This lets build artifacts (aside from some priv/ issues with plugins) to
properly be located within their build folder rather than the deps
folder, as the --outdir switch would otherwise have done.

Not all of the changes to the git resource were reverted when
the sparse checkout support was moved to a new resource,
git_subdir_resource. This fixes issues caused by these changes
that were accidentally left.

Since the compiler refactor changed the order of steps and all hooks get
applied at once, knowing which app is being escriptized isn't really
obvious, particularly when many apps individually define their own 'main
app' and they get run at once.

This adds the name of the main app being run when escriptizing it to make
it clear what goes on, especially if there are failures.

This was reported in https://github.com/erlang/rebar3/pull/2211#issuecomment-615937628

Fixes #2273

This patch aims to make rebar3 delete a dependency by first looking for
its name (fixing `del` on hex dependencies) and if that isn't found
remove it as always, keeping its current behaviour with the `del` override.

Starting with OTP-22, Erlang started changing how volume names are handled in
windows: filename:join/1 drops drive letters, filename:split/2 and
file:absname/1 use c:/ as a form instead of c:\\

We adjust the file_utils and their tests accordingly.

Rebar3.14 has new tests that were not windows ready and those are fixed.

Relx has two failing tests that are not covered in this branch.

Tested in Windows 10 with Powershell.

This reverts commit b7cfa757dc.

We run the analysis for an extra_src_dir by making a fake app for it,
but send in the app alone for analysis.

In there, the DAG pruning routine looks at the DAG and files submitted
and goes "this right here is a project with 99% of its apps deleted". It
then tries to prune the whole DAG except for some test files. The code
"works" simply because there's a false-positive check that makes sure
the file is on disk before removing it for the DAG.

This ends up making extra runs where ~80% of the time is spent
double-checking the false positives for file deletions.

This commit fixes this by merging in all extra_src fake apps and making them
run in a single analysis phase, meaning we only pay the cost of the DAG
pruning once for the whole project, making it faster than any sparse
repo.

There's also a small patch needed for the root-level extra src dirs;
turns out that since the context-handling in the `rebar_compiler` uses a
map to store content, running single-pass analysis clobbered entries for
a given app if they had more than one extra_src_dir in there.

I also took the time to clean up the ordering of that file.

Into a standalone module where it can be reused for optimisation work
later on.

Cleans up some annoying broken abstraction.

Also fix erl_first_file artifact tracking, which will avoid seeking to
disk by finding the opts in the DAG

This reverts commit f0c8c202af.

Because the resource doesn't absolutely require using a sparse
checkout, it only matters that rebar3 knows the correct path of the
application, the resource is named git_subdir.

It has to be git specific because the reason this subdir support is
special is the need to keep the code checkout as a usable git clone
to run commands against. This means the resource can't just copy
the content of the subdirectory to the application out directory.

End result should allow to look like this:

    $ head -n 3 rebar3
    #!/usr/bin/env escript
    %% Rebar3 3.14.0-rc1
    %%! +sbtu +A1

This will allow tools like mix to read the top of the file to know the
version being used in the current path (when using features that aren't
forwards-compatible) without having to invoke the tool itself.

The version (full dynamic one) is also added to the local install:

    $ head -n 2 /home/ferd/.cache/rebar3/bin/rebar3
    #!/usr/bin/env sh
    ## Rebar3 3.14.0-rc1+build.4599.ref6b0f90d

The `##` comments are used to keep a fixed width over the other form.

To be backported to 3.13.2 as well.