The compiling of OTP applications is done by first topographically
sorting them according to their dependencies, deps-first. This allows
all compilation to take place in order. In the current code, the same
logic extends to top-level applications in an umbrella project.

Unfortunately, there are cases where this is not going to be true: when
an application has extra_src_dirs entries (or additional directories or
files) to conditionally compile under some profiles, it may start
depending on another top-level application dedicated to that profile for
include files.

However, such an app will never make it to production and neither will
the compilation artifacts that create the dependency. Under that
scenario, current rebar3 is unusable.

This patch makes it so that the compilation provider instead changes the
logic for top-level apps: rather than copying their directories one by
one and compiling them in order, it:

1. copies all top-level apps to the build directory so the files are in
   their proper locations
2. adds the top-level apps to the path (after the global hooks have run,
   so the existing scope and env has not changed)
3. runs the compilation as usual.

Fixes #1651

on 19.x forward the compiler should now take into consideration the value
of the environment variable `ERL_COMPILER_OPTIONS` when deciding whether
or not to recompile a module

The option {recursive,boolean()} can now be set pr directory in
'src_dirs' and 'extra_src_dirs', and on top level in the new
'erlc_compiler' option. Example config:

{erlc_compiler,[{recursive,false}]}.
{src_dirs,[{"src",[{recursive,true}]}]}.

This will cause recursive compilation within the "src" directory, but
not in any other directoires.

When an include file is set in a private path (i.e. src/), the rebar3
compiler would not add them to the {i, Path} params -- only include/ and
the project root were being added.

This meant that when some extra source directories were added to the
compile job, such as test/ when running under the test profile, the
private include paths could not be shared with the test module.

This patch fixes the issues (and adds tests) for such a specific case by
adding all the configured include paths to the {i, Path} erl_opts
arguments, yielding successful compile runs.

there's no way to detect which files actually rely on a parse transform
passed to the compiler via the options (as opposed to `-compile(..)`
so if any parse transforms are in modules that need recompiling just
recompile the world

fixes #1328

changed include files were not properly picked up by `erlc_compiler`
in cases where they were in directories relative to the application
source and not the current working dir of rebar3

fixes #1199

partially addresses #1184

Several projects use an include path relative
to the project's root.
file:compile will look in three places for the include
files:
The current working directory
The directory where the module is being compiled
The directories given by the include option

this restores compatibility with rebar2 and erlang.mk

* modify compiler interface to work on either application objects or
  directories containing source files
* compile all sources in `src_dirs` to the application `ebin` dir and
  all sources in `extra_src_dirs` to a directory mirroring it's
  position in the app's `_build` directory. for example, `apps/foo/more`
  would compile to `_build/default/lib/foo/more`
  for `extra_src_dirs` in the root of a project with multiple
  applications (so orphan directories that don't "belong" to an
  application) compile to `_build/default/extras/more`
* copy directories specified in `extra_src_dirs` into the `_build`
  directory so tools like `ct` and `xref` that expect source to be
  in a particular location still work
* clean compiled artifacts from all `extra_src_dirs`
* alter `eunit`, `ct` and `cover` to work with the new directory
  structure
* billions of new tests

This reverts commit 9d82215296.

This passes the directory to the option to ensure it works with more
than configs at the root of the project.

to `rebar_erlc_compiler`

Requires a rework of other test suites using the same dep-handling
mechanism.

The test works by using a parse transform that stamps modules with an
attribute as it runs them. It then compiles everything, loads the
module, and makes sure the stamps respect the defined order.

get correct compile paths included in module info

fixes #216 and #220

pre-purging before checking to facilitate long do chains