* update to hex_core for hex-v2 repo support (#1865)

* update to hex_core for hex-v2 repo support

This patch adds only single repo hex-v2 support through hex_core.
Packages no longer filtered out by buildtool metadata and the
package index is updated per-package instead of fetched as one
large ets dump.

* tell travis to also build hex_core branch

* support list of repos for hex packages (#1866)

* support list of repos for hex packages

repos are defined under the hex key in rebar configs. They can be
defined at the top level of a project or globally, but not in
profiles and the repos configured in dependencies are also ignored.

Searching for packages involves first checking for a match in the
local repo index cache, in the order repos are defined. If not found
each repo is checked through the hex api for any known versions of
the package and the first repo with a version that fits the constraint
is used.

* add {repos, replace, []} for overriding the global & default repos

* add hex auth handling for repos (#1874)

auth token are kept in a hex.config file that is modified by the
rebar3 hex plugin.

Repo names that have a : separating a parent and child are considered
organizations. The parent repo's auth will be included with the child.
So an organization named hexpm:rebar3_test will include any hexpm
auth tokens found in the rebar3_test organization's configuration.

* move packages to top level of of hexpm cache dir (#1876)

* move packages to top level of of hexpm cache dir

* append organization name to parent's repo_url when parsing repos

* only eval config scripts and apply overrides once per app (#1879)

* only eval config scripts and apply overrides once per app

* move new resource behaviour to rebar_resource_v2 and keep v1

* cleanup use of rebar_resource module and unused functions

* cleanup error messages and unused code

* when discovering apps support mix packages as unbuilt apps (#1882)

* use hex_core tarball unpacking support in pkg resource (#1883)

* use hex_core tarball unpacking support in pkg resource

* ignore etag if package doesn't exist and delete if checksum fails

* add back tests for bad package checksums

* improve bad registry checksum error message

This is done through 3 main change groups:

- replacing `~s` by `~ts` in format strings, so that strings that
  contain unicode are properly printed rather than crashing
- adding the `unicode` argument to all function of the `re` module to
  ensure transformations on strings containing unicode data are valid
  instead of crashing (see issue #1302)
- replacing `ec_cnv:to_binary/1` and `ec_cnv:to_list/1` with matching
  functions in `rebar_utils`.

The last point has been done, rather than modifying and updating erlware
commons, because binary and list conversions can be a contentious
subject. For example, if what is being handled is actually bytes from a
given binary stream, then forcing a byte-oriented interpretation of the
data can corrupt it. As such, it does not appear safe to modify erlware
commons' conversion functions since it may not be safe for all its
users.

Instead, rebar3 reimplements a subset of them (only converting
atoms and chardata, ignoring numbers) with the explicit purpose of
handling unicode string data.

Tests were left as unchanged as possible. This may impact the ability to
run rebar3's own suites in a unicode path, but respects a principle of
least change for such a large patch.

Some tricky changes in there but should be okay

- the internal representation for package locks moves from `{Name, {pkg,
  PkgName, Vsn}, Lvl}` to `{Name, {pkg, PkgName, Vsn, Hash}, Lvl}`
- the internal representation for packages moves from `{pkg, PkgName,
  Vsn}` to `{pkg, PkgName, Vsn, Hash}`
- the hash can be `undefined`, meaning no check will be done
- no checking is done yet.

- Crashes in providers lib when no providers in a namespace are bare
- Making sure bareness matches semantics; i.e. a bare provider is
  visible, a non-bare provider is hidden.

Whenever files were being locked *and* provided in rebar.config
duplicates could be printed. This is taken care of.

Also whenever atom packages were being used, they would cause the merge
step to fail. The new format does proper name-based deduplication of
resources, giving priority to locks to work.

ModificatioN/needs-update support is still in.

This may also act as a fix for #341, if automation isn't required too
much.

The 'element(1, ..)' guard would always fail and cause crashes.

Removed separate compilers
Resolves apps to build
Finds avail deps before pulling/building
Includes relx
Simplifies build commands

suggested by @tuncer

This calls the 'p4' command-line tool to checkout and sync Perforce
trees. It involves significantly more special code in Rebar than
using 'git p4', but it eliminates the indirection of
Rebar->Git->Python->Perforce

* update files
* fix Dialyzer warning
* unconditionally enable info fil
* clean-up inconsistencies
* use term_to_binary compression
* use try...catch instead of case...catch...of
* do not write build info file if the graph is unmodified
* store info file as <base_dir>/.rebarinfo
* properly support list of compile directives
* fix regressions:
 - Fix a bug in handling of files to compile first.
 - If a file that is depended upon itself depends on other files, make sure
   those are compiled first. While at it, rename variables for correctness.
   Reported-by: David Robakowski
 - Make sure that FirstFiles has no dupes and preserves the proper order.
 - headers referenced via -include_lib() were not properly resolved to absolute
   filenames
 - .erl files found in sub dirs of src_dirs were not properly resolved to
   absolute filenames

As discussed with Martin Schut, remove support for
{app_name, "vsn", {fossil, "url", latest}.

Sometimes tags like 1.1-3-g3af5478 or d20b53f0 are encountered. The
first is the output of 'git describe', and the second is just a regular
git SHA. git fetch --tags will not pull these down, so do a full git
fetch instead.

Because rebar_core handles skipping apps, we had to specialcase the
handling in the case of update-deps because it has to do its own dep
handling. The way this was done is not particularly clean, but there
currently does not exist another way for a command to signal rebar_core
that it doesn't want rebar_core to pay attention to skip_apps.

With this change, however, you can update-deps even with local
conflicting changes/commits by simply skipping the deps you don't wish
to update, or whitelisting he ones you do wish to update.

Previously, update-deps on a dep tagged as {branch, ...} would do the
following:

  git fetch
  git checkout -q origin/<branch>

If you were already on that branch, the repo would end up in detached
head state. This is kind of annoying if you're doing local development.

This patch changes the behaviour to be

  git fetch
  git checkout -q <branch>
  git pull --ff-only --no-rebase -q <branch>

The intent of this is to move the branch's HEAD forward to match
upstream without destroying any local commits or changes, and without
accidentally causing merges or rebases. It will fail if the operation
can not be performed without losing history, merging or rebasing.

The previous behaviour has been around a very long time:

064195dc5a (L0R308)

It also exactly mirrors the download_source case, which is not really
true. With git tags and SHAs, one can assume that they don't change, but
branches move all the time.