The change was possible since R13B04 when macros with many arities
became supported. We never really cleaned that stuff up before so I'm
picking it up while I'm updating that code

* add compile type for dynamic project compilation

* new rebar_compiler abstraction for running multiple compilers

rebar_compiler is a new behaviour that a plugin can implement to
be called on any ues of the compile provider to compile source
files and keep track of their dependencies.

* fix check that modules in .app modules list are from src_dirs

* use project_type to find module for building projects

* allow plugins to add project builders and compilers

Turns output like this:

    ===> Compiling apps/c/src/c.erl failed
    apps/c/src/c.erl:3: can't find include lib "parse_trans/include/codegen.hrl"

Into:

    ===> Compiling apps/c/src/c.erl failed
    apps/c/src/c.erl:3: can't find include lib
    "parse_trans/include/codegen.hrl"; Make sure parse_trans is in your app
    file's 'applications' list

Which is likely going to help newcomers encountering issues.

This is done through 3 main change groups:

- replacing `~s` by `~ts` in format strings, so that strings that
  contain unicode are properly printed rather than crashing
- adding the `unicode` argument to all function of the `re` module to
  ensure transformations on strings containing unicode data are valid
  instead of crashing (see issue #1302)
- replacing `ec_cnv:to_binary/1` and `ec_cnv:to_list/1` with matching
  functions in `rebar_utils`.

The last point has been done, rather than modifying and updating erlware
commons, because binary and list conversions can be a contentious
subject. For example, if what is being handled is actually bytes from a
given binary stream, then forcing a byte-oriented interpretation of the
data can corrupt it. As such, it does not appear safe to modify erlware
commons' conversion functions since it may not be safe for all its
users.

Instead, rebar3 reimplements a subset of them (only converting
atoms and chardata, ignoring numbers) with the explicit purpose of
handling unicode string data.

Tests were left as unchanged as possible. This may impact the ability to
run rebar3's own suites in a unicode path, but respects a principle of
least change for such a large patch.

This allows to reuse the code for any provider that formats source files
out to the user.

The option to configure it does remain compiler-centric for backwards
compatibility

The regex mistakenly matched too many files (any character followed by
an underscore) rather than only files starting in '._'

This properly escapes the expressions to work in all cases.

Includes improvments and function documentation for all modules (in
alphabetical order) up to rebar_core, and may have included more in
other modules as I saw fit to dig and understand more of the internals.

rebar_base_compiler allows to be called with two types of options: a
dictionary, or a rebar_state record. In the later case, the options are
taken out with a call from rebar_opts, which fetches options that have
been inserted in the application via rebar_app_info as part of the
app_discovery phase, and are a list.

This yields a possibility that options used when formatting warnings can
either be a list of a dict, and we only used lists when making checks.
This ended up breaking 3rd party compiler users (i.e. LFE compile
plugin) since they were calling us with a dict rather than our own
internal records.

This patch supports both types of lookups to avoid issues.

This required moving the reporting functions to rebar_base_compiler but
since this was already done for error_tuple, this seems to make sense.

Paths are also reformatted for warnings in erlc files.

By default rebar3 displays compiler sources as absolute paths in their
original location, which is under the build dir.

This change introduces an option 'compiler_source_format' to format
sources in two alternative ways:

    relative
    absolute

When either 'relative' or 'absolute' are specified, the file is
resolved to its original location when it is a link. When 'relative'
is specified, the path is displayed relative to the current working
directory. When 'absolute' is specified, the path is absolute.

The default value is 'unchaged' which leaves the compiler source
unchanged.

This is arguably too flexible as I suspect most people would opt for
'relative' all the time - it's the most compact representation of the
file and is sufficient to find the source given cwd. The change
however is meant to introduce the change gradually, preserving
existing behavior and giving users a choice for formats.

In time perhaps the default can be changed to 'relative' - but still
allowing users to revert to the other two options ('absolutel' and
'unchanged') as needed.

If you happen to fetch a zip archive of the git repo and try to build
from that, you may, for example, ask erlc to build src/._rebar.erl.
._* are OS X resource forks and not real .erl files. This may also
happen with network filesystems on OS X. To fix that, limit the
files compiled by rebar to include only those which start with
a letter or a digit.

If the syntax error is in a .hrl file, then the reported error message
is not as useful because it's not clear which .erl file was being
compiled. We can fix that easily by first printing what source file was
being processed. We don't change the actual error message, so this will
still work with your editor of choice for jumping to the right line.

Before
------

Success:
Compiled src/foo.erl

Failure:
include/foo.hrl:10: syntax error [...]

After
-----

Success:
Compiled src/foo.erl

Failure:
Compiling src/foo.erl failed:
include/foo.hrl:10: syntax error [...]

rebar throws away some error messages, e.g. the ones generated if the yecc compiler is broken.

Handle the case where the error didn't occur in the file being
compiled.  That is, if there is an error on line 9 of bar.hrl,
instead of:

/path/to/foo.erl:9: type foo() already defined

print:

/path/to/bar.hrl:9: type foo() already defined

Added a backward compartible feature to specify `erlydtl_opts' options
for the DTL template compiler to allow inclusion of templates in different
directories with different compilation settings for each. E.g.:

    {erlydtl_opts, [
          [{doc_root, "src/view"}, {module_ext, "_dtl_vw"}]
        , [{doc_root, "src"},      {module_ext, ""}, {recursive, false}]

        , {out_dir,   "ebin"}
        , {compiler_options, [verbose, debug_info]}
    ]}.

The definition above is identical to this (the last two options
are duplicated in each list):

    {erlydtl_opts, [
          [{doc_root,   "src/view"}
          ,{module_ext, "_dtl_vw"}
          ,{out_dir,    "ebin"}
          ,{compiler_options, [verbose, debug_info]}]

        , [{doc_root,   "src"}
          ,{module_ext, ""}
          ,{out_dir,    "ebin"}
          ,{compiler_options, [verbose, debug_info]}
          ,{recursive, false}]
    ]}.

In this case "src/view" and "src" directories containing template files
will be compiled.  A new `recursive' option tells rebar_erlydtl_compiler
to search files recursively from a given doc_root.  In the example above
the "src" directory won't be scanned recursively, and the target template
name for target beam modules won't have "_dtl_vw" suffix.