- update bootstrap script to account for hex_core in src/vendored

This was a long-standing pending issue with plugins having a weird code
path handling. The work done in the previous commit to do version
detection for the compiler in dependencies along with prior PRs that
added a deps-only mode to the `compile` provider means we can now
re-introduce plugins into the 3.14 compilation mechanism.

This makes compiler version detection a compiler task concern and
removes annoying manual checks, but also solves an unreported issue
(brought up on slack only) where a plugin would no longer have its priv/
or include/ dir included when used as a checkout dependency. Essentially
the latter bug was caused by being out-of-sync with application
preparation for a build task, but by re-folding it into the main flow,
we know we have the proper support as used for deps.

This reverts commit b7cfa757dc.

- Can't deal with := in map type declarations
- maps:update_with didn't exist back then so we reinvent it for the few
  months until we deprecate OTP 18

This commit is a transition point that makes some assumptions about new
callbacks for the compiler modules, which will likely not hold when it
comes to making epp be able to read and handle multiple applications at
once (so it can resolve parse transforms and behaviours across OTP
apps).

As such, a follow-up commit is going to be completing that one and
changing its API in incompatible ways; if you find this commit during a
bisect, it's probably not a good one in which to run a bisection.

Solves #2071, IPV6 support for proxy, by trying to resolve proxy
host name without specific IP address family. If it fails, it
attempts a different IP family, and in case that succeeds, it
sets up HTTP client profile to use IP family that worked.

When recompiling rebar3 itself, the path swaps and cleaning removes the
modules loaded from _build/bootstrap, but the VM still manages to
discover those in _build/prod from previous builds of the escript when
building the new escript, which causes weird failures when compilers get
modified between releases.

The fix is to just clear _build/prod as part of the init of the
bootstrap script, the same way we already clear _build/bootstrap.

Otherwise we only get silent failures. This has been fixed a while ago
in mainline rebar3 but never in the bootstrap script.

- vendor in hex_core at v0.5.0
  - Change where repo_name should be the org and not the parent
  - Changed rebar_utils:url_append_path/2 to not explicitly add a '?',
    this is returned in the Query chunk by http_uri:parse/1 (e.g., "?foo=bar")
  - update organization_merging test to expect the sub-repo as the repo_name
  - Add tests for rebar_utils:url_append_path/2
  - Stop referencing/setting "organization" in config and use new
     organization settings (api_repository and repo_organization)
  - Do not set (assume) the read key is valid for any/every repo
  - Set repo_organization and api_repository to org
  - Update tests to check for new config opts

This should avoid common problems when upgrading across versions where
old code kind of messes up a lot of things and clearing _build keeps
being required.

Signed-off-by: Fred Hebert <mononcqc@ferd.ca>

* update to hex_core for hex-v2 repo support (#1865)

* update to hex_core for hex-v2 repo support

This patch adds only single repo hex-v2 support through hex_core.
Packages no longer filtered out by buildtool metadata and the
package index is updated per-package instead of fetched as one
large ets dump.

* tell travis to also build hex_core branch

* support list of repos for hex packages (#1866)

* support list of repos for hex packages

repos are defined under the hex key in rebar configs. They can be
defined at the top level of a project or globally, but not in
profiles and the repos configured in dependencies are also ignored.

Searching for packages involves first checking for a match in the
local repo index cache, in the order repos are defined. If not found
each repo is checked through the hex api for any known versions of
the package and the first repo with a version that fits the constraint
is used.

* add {repos, replace, []} for overriding the global & default repos

* add hex auth handling for repos (#1874)

auth token are kept in a hex.config file that is modified by the
rebar3 hex plugin.

Repo names that have a : separating a parent and child are considered
organizations. The parent repo's auth will be included with the child.
So an organization named hexpm:rebar3_test will include any hexpm
auth tokens found in the rebar3_test organization's configuration.

* move packages to top level of of hexpm cache dir (#1876)

* move packages to top level of of hexpm cache dir

* append organization name to parent's repo_url when parsing repos

* only eval config scripts and apply overrides once per app (#1879)

* only eval config scripts and apply overrides once per app

* move new resource behaviour to rebar_resource_v2 and keep v1

* cleanup use of rebar_resource module and unused functions

* cleanup error messages and unused code

* when discovering apps support mix packages as unbuilt apps (#1882)

* use hex_core tarball unpacking support in pkg resource (#1883)

* use hex_core tarball unpacking support in pkg resource

* ignore etag if package doesn't exist and delete if checksum fails

* add back tests for bad package checksums

* improve bad registry checksum error message

Includes an update to parse_trans on an OTP-21 friendly version

Based off a macro by @okeuday at https://github.com/erlang/otp/pull/1783

We got funny interactions since PR #1656:

- the last `debug_info`-related option seen in a list of options after
  profile merge is kept, allowing later profiles from overtaking earlier
  ones
- if you go `rebar3 as a,b,c compile`, the options from profile A come
  before B, which come before C, so C's options win
- overrides are applied in order of profile as well, giving a priority
  to a later profile than an earlier one
- The values in overrides are prepended rather than suffixed to the
  existing list
- this means if we have to overrides adding options, such as `default`
  adding `no_debug_info', and `dialyze` adding `debug_info`, the
  results are `[debug_info]` as `dialyze` is applied first, and then
  `[no_debug_info, debug_info]` as `default` overrides are applied
- the final result is `no_debug_info` always winning when erl_opts are
  overriden specifically.

only `debug_info` options are going to suffer this, and in the context of
overrides. Other `erl_opts` should be fine. I'm not sure how that can be
fixed at all.

In the meanwhile, we can add support back while leaving the default to
not having debug information. This is done by:

- moving all `no_debug_info` options to the `prod` profile
- forcing `prod` to be called by `./bootstrap` so that most people keep
  getting no debug info
- anyone calling `rebar3 clean -a`  and then rebuilding with `rebar3
  escriptize` (i.e. rebar3 devs) get debug info going

This is up for review and discussion.

Updating the bootstrap script in #1689 made it so symlinks that already
exist return `exist` rather than ok, in line with the regular code.
However the bootstrap module wouldn't handle this case.

This fixes the problem by ensuring that `exists` is as valid as `ok`.