The change was possible since R13B04 when macros with many arities
became supported. We never really cleaned that stuff up before so I'm
picking it up while I'm updating that code

Also fixes artifacts left on filesystem, and produces less
filelib:last_modified() checks.

However, we add a once-printed deprecation warning for the new
interface.

* add compile type for dynamic project compilation

* new rebar_compiler abstraction for running multiple compilers

rebar_compiler is a new behaviour that a plugin can implement to
be called on any ues of the compile provider to compile source
files and keep track of their dependencies.

* fix check that modules in .app modules list are from src_dirs

* use project_type to find module for building projects

* allow plugins to add project builders and compilers

When REBAR_CONFIG was set it would not effect the top level app's
configuration because app_discover was rereading the top level
rebar.config which ignored REBAR_CONFIG. Instead this patch has
it use the existing configuration from REBAR_CONFIG.

- resolves #1645

This is done through 3 main change groups:

- replacing `~s` by `~ts` in format strings, so that strings that
  contain unicode are properly printed rather than crashing
- adding the `unicode` argument to all function of the `re` module to
  ensure transformations on strings containing unicode data are valid
  instead of crashing (see issue #1302)
- replacing `ec_cnv:to_binary/1` and `ec_cnv:to_list/1` with matching
  functions in `rebar_utils`.

The last point has been done, rather than modifying and updating erlware
commons, because binary and list conversions can be a contentious
subject. For example, if what is being handled is actually bytes from a
given binary stream, then forcing a byte-oriented interpretation of the
data can corrupt it. As such, it does not appear safe to modify erlware
commons' conversion functions since it may not be safe for all its
users.

Instead, rebar3 reimplements a subset of them (only converting
atoms and chardata, ignoring numbers) with the explicit purpose of
handling unicode string data.

Tests were left as unchanged as possible. This may impact the ability to
run rebar3's own suites in a unicode path, but respects a principle of
least change for such a large patch.

on 19.x forward the compiler should now take into consideration the value
of the environment variable `ERL_COMPILER_OPTIONS` when deciding whether
or not to recompile a module

The regex mistakenly matched too many files (any character followed by
an underscore) rather than only files starting in '._'

This properly escapes the expressions to work in all cases.

The option {recursive,boolean()} can now be set pr directory in
'src_dirs' and 'extra_src_dirs', and on top level in the new
'erlc_compiler' option. Example config:

{erlc_compiler,[{recursive,false}]}.
{src_dirs,[{"src",[{recursive,true}]}]}.

This will cause recursive compilation within the "src" directory, but
not in any other directoires.

Some tricky changes in there but should be okay

Helps cover extra cases.

When an include file is set in a private path (i.e. src/), the rebar3
compiler would not add them to the {i, Path} params -- only include/ and
the project root were being added.

This meant that when some extra source directories were added to the
compile job, such as test/ when running under the test profile, the
private include paths could not be shared with the test module.

This patch fixes the issues (and adds tests) for such a specific case by
adding all the configured include paths to the {i, Path} erl_opts
arguments, yielding successful compile runs.

there's no way to detect which files actually rely on a parse transform
passed to the compiler via the options (as opposed to `-compile(..)`
so if any parse transforms are in modules that need recompiling just
recompile the world

fixes #1328

changed include files were not properly picked up by `erlc_compiler`
in cases where they were in directories relative to the application
source and not the current working dir of rebar3

fixes #1199

partially addresses #1184

This required moving the reporting functions to rebar_base_compiler but
since this was already done for error_tuple, this seems to make sense.

Paths are also reformatted for warnings in erlc files.

This will allow us to eventually change the default type without it
looking ridiculous.

By default rebar3 displays compiler sources as absolute paths in their
original location, which is under the build dir.

This change introduces an option 'compiler_source_format' to format
sources in two alternative ways:

    relative
    absolute

When either 'relative' or 'absolute' are specified, the file is
resolved to its original location when it is a link. When 'relative'
is specified, the path is displayed relative to the current working
directory. When 'absolute' is specified, the path is absolute.

The default value is 'unchaged' which leaves the compiler source
unchanged.

This is arguably too flexible as I suspect most people would opt for
'relative' all the time - it's the most compact representation of the
file and is sufficient to find the source given cwd. The change
however is meant to introduce the change gradually, preserving
existing behavior and giving users a choice for formats.

In time perhaps the default can be changed to 'relative' - but still
allowing users to revert to the other two options ('absolutel' and
'unchanged') as needed.

Several projects use an include path relative
to the project's root.
file:compile will look in three places for the include
files:
The current working directory
The directory where the module is being compiled
The directories given by the include option

this restores compatibility with rebar2 and erlang.mk

When compiling a dependency with a MIB file the generated hrl file is left in
the root project directory in a file called "include". This has the perverse
effect of messing up the search path for include files causing any dependencies
with files in their "include" directory to fail to build after that.

In some cases (nested includes?) paths end up in such a way that joining
them breaks up and hard-crashes rebar3. This patch specifically handles
this scenario to fix things by avoiding passing empty lists to
filename:join.

* modify compiler interface to work on either application objects or
  directories containing source files
* compile all sources in `src_dirs` to the application `ebin` dir and
  all sources in `extra_src_dirs` to a directory mirroring it's
  position in the app's `_build` directory. for example, `apps/foo/more`
  would compile to `_build/default/lib/foo/more`
  for `extra_src_dirs` in the root of a project with multiple
  applications (so orphan directories that don't "belong" to an
  application) compile to `_build/default/extras/more`
* copy directories specified in `extra_src_dirs` into the `_build`
  directory so tools like `ct` and `xref` that expect source to be
  in a particular location still work
* clean compiled artifacts from all `extra_src_dirs`
* alter `eunit`, `ct` and `cover` to work with the new directory
  structure
* billions of new tests

This reverts commit 9d82215296.

This passes the directory to the option to ensure it works with more
than configs at the root of the project.

to `rebar_erlc_compiler`

note that now ALL `src_dirs` across included profiles are
compiled. previously only the last included profile's `src_dirs`
were used

avoid repeated loading and unloading of modules