You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

655 lines
26 KiB

Support old-style shell for rebar3 shell This is quite the hack. This requires to detect the current shell running; if it's the new shell, business as usual. However, if it's the old shell, we have to find a way to take over it and drive IO. This requires a few steps because: - the old shell does not let you be supervised intelligently (it uses supervisor_bridge, so killing the child is not a supported operation from the supervisor) - the old shell ignores all trappable exit signals except those coming from the Port in charge of stdio ({fd, 0, 1}) - the old shell shuts down on all exit signals from the stdio Port except for badsig, and replicates the shutdown reason otherwise - An escript does not tolerate the `user` process dying (old shell) for any non-normal reason without also taking the whole escript down - Booting in an escript has an implicit 'noshell' argument interpreted by the old shell as a way to boot the stdio Port with only stdout taken care of Because of all these points, we have to kill the old `user` process by sending it a message pretending to be the Stdio port dying of reason `normal`, which lets it die without triggering the ire of its supervision tree and keeping the escript alive. This, in turn, kills the old stdio port since its parent (user.erl) has died. Then we have to boot our copy of user.erl (rebar_user.erl) which conveniently ignores the possibility of running the stdio port on stdout only -- always using stdin *and* stdout, giving us a bona fide old-style shell. A known issue introduced is that running r3:do(ct) seems to then kill the shell, and r3:do(dialyzer) appears to have an odd failure, but otherwise most other commands appear to work fine.
9 years ago
support for hex v2, multiple repository fetching, private organizations (#1884) * update to hex_core for hex-v2 repo support (#1865) * update to hex_core for hex-v2 repo support This patch adds only single repo hex-v2 support through hex_core. Packages no longer filtered out by buildtool metadata and the package index is updated per-package instead of fetched as one large ets dump. * tell travis to also build hex_core branch * support list of repos for hex packages (#1866) * support list of repos for hex packages repos are defined under the hex key in rebar configs. They can be defined at the top level of a project or globally, but not in profiles and the repos configured in dependencies are also ignored. Searching for packages involves first checking for a match in the local repo index cache, in the order repos are defined. If not found each repo is checked through the hex api for any known versions of the package and the first repo with a version that fits the constraint is used. * add {repos, replace, []} for overriding the global & default repos * add hex auth handling for repos (#1874) auth token are kept in a hex.config file that is modified by the rebar3 hex plugin. Repo names that have a : separating a parent and child are considered organizations. The parent repo's auth will be included with the child. So an organization named hexpm:rebar3_test will include any hexpm auth tokens found in the rebar3_test organization's configuration. * move packages to top level of of hexpm cache dir (#1876) * move packages to top level of of hexpm cache dir * append organization name to parent's repo_url when parsing repos * only eval config scripts and apply overrides once per app (#1879) * only eval config scripts and apply overrides once per app * move new resource behaviour to rebar_resource_v2 and keep v1 * cleanup use of rebar_resource module and unused functions * cleanup error messages and unused code * when discovering apps support mix packages as unbuilt apps (#1882) * use hex_core tarball unpacking support in pkg resource (#1883) * use hex_core tarball unpacking support in pkg resource * ignore etag if package doesn't exist and delete if checksum fails * add back tests for bad package checksums * improve bad registry checksum error message
6 years ago
Support old-style shell for rebar3 shell This is quite the hack. This requires to detect the current shell running; if it's the new shell, business as usual. However, if it's the old shell, we have to find a way to take over it and drive IO. This requires a few steps because: - the old shell does not let you be supervised intelligently (it uses supervisor_bridge, so killing the child is not a supported operation from the supervisor) - the old shell ignores all trappable exit signals except those coming from the Port in charge of stdio ({fd, 0, 1}) - the old shell shuts down on all exit signals from the stdio Port except for badsig, and replicates the shutdown reason otherwise - An escript does not tolerate the `user` process dying (old shell) for any non-normal reason without also taking the whole escript down - Booting in an escript has an implicit 'noshell' argument interpreted by the old shell as a way to boot the stdio Port with only stdout taken care of Because of all these points, we have to kill the old `user` process by sending it a message pretending to be the Stdio port dying of reason `normal`, which lets it die without triggering the ire of its supervision tree and keeping the escript alive. This, in turn, kills the old stdio port since its parent (user.erl) has died. Then we have to boot our copy of user.erl (rebar_user.erl) which conveniently ignores the possibility of running the stdio port on stdout only -- always using stdin *and* stdout, giving us a bona fide old-style shell. A known issue introduced is that running r3:do(ct) seems to then kill the shell, and r3:do(dialyzer) appears to have an odd failure, but otherwise most other commands appear to work fine.
9 years ago
Support old-style shell for rebar3 shell This is quite the hack. This requires to detect the current shell running; if it's the new shell, business as usual. However, if it's the old shell, we have to find a way to take over it and drive IO. This requires a few steps because: - the old shell does not let you be supervised intelligently (it uses supervisor_bridge, so killing the child is not a supported operation from the supervisor) - the old shell ignores all trappable exit signals except those coming from the Port in charge of stdio ({fd, 0, 1}) - the old shell shuts down on all exit signals from the stdio Port except for badsig, and replicates the shutdown reason otherwise - An escript does not tolerate the `user` process dying (old shell) for any non-normal reason without also taking the whole escript down - Booting in an escript has an implicit 'noshell' argument interpreted by the old shell as a way to boot the stdio Port with only stdout taken care of Because of all these points, we have to kill the old `user` process by sending it a message pretending to be the Stdio port dying of reason `normal`, which lets it die without triggering the ire of its supervision tree and keeping the escript alive. This, in turn, kills the old stdio port since its parent (user.erl) has died. Then we have to boot our copy of user.erl (rebar_user.erl) which conveniently ignores the possibility of running the stdio port on stdout only -- always using stdin *and* stdout, giving us a bona fide old-style shell. A known issue introduced is that running r3:do(ct) seems to then kill the shell, and r3:do(dialyzer) appears to have an odd failure, but otherwise most other commands appear to work fine.
9 years ago
Support old-style shell for rebar3 shell This is quite the hack. This requires to detect the current shell running; if it's the new shell, business as usual. However, if it's the old shell, we have to find a way to take over it and drive IO. This requires a few steps because: - the old shell does not let you be supervised intelligently (it uses supervisor_bridge, so killing the child is not a supported operation from the supervisor) - the old shell ignores all trappable exit signals except those coming from the Port in charge of stdio ({fd, 0, 1}) - the old shell shuts down on all exit signals from the stdio Port except for badsig, and replicates the shutdown reason otherwise - An escript does not tolerate the `user` process dying (old shell) for any non-normal reason without also taking the whole escript down - Booting in an escript has an implicit 'noshell' argument interpreted by the old shell as a way to boot the stdio Port with only stdout taken care of Because of all these points, we have to kill the old `user` process by sending it a message pretending to be the Stdio port dying of reason `normal`, which lets it die without triggering the ire of its supervision tree and keeping the escript alive. This, in turn, kills the old stdio port since its parent (user.erl) has died. Then we have to boot our copy of user.erl (rebar_user.erl) which conveniently ignores the possibility of running the stdio port on stdout only -- always using stdin *and* stdout, giving us a bona fide old-style shell. A known issue introduced is that running r3:do(ct) seems to then kill the shell, and r3:do(dialyzer) appears to have an odd failure, but otherwise most other commands appear to work fine.
9 years ago
  1. %% -*- erlang-indent-level: 4;indent-tabs-mode: nil -*-
  2. %% ex: ts=4 sw=4 et
  3. %% -------------------------------------------------------------------
  4. %%
  5. %% rebar: Erlang Build Tools
  6. %%
  7. %% Copyright (c) 2011 Trifork
  8. %%
  9. %% Permission is hereby granted, free of charge, to any person obtaining a copy
  10. %% of this software and associated documentation files (the "Software"), to deal
  11. %% in the Software without restriction, including without limitation the rights
  12. %% to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  13. %% copies of the Software, and to permit persons to whom the Software is
  14. %% furnished to do so, subject to the following conditions:
  15. %%
  16. %% The above copyright notice and this permission notice shall be included in
  17. %% all copies or substantial portions of the Software.
  18. %%
  19. %% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  20. %% IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  21. %% FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  22. %% AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  23. %% LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  24. %% OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  25. %% THE SOFTWARE.
  26. %% -------------------------------------------------------------------
  27. -module(rebar_prv_shell).
  28. -author("Kresten Krab Thorup <krab@trifork.com>").
  29. -author("Fred Hebert <mononcqc@ferd.ca>").
  30. -behaviour(provider).
  31. -export([init/1,
  32. do/1,
  33. format_error/1]).
  34. -include("rebar.hrl").
  35. -include_lib("providers/include/providers.hrl").
  36. -define(PROVIDER, shell).
  37. -define(DEPS, [compile]).
  38. -dialyzer({nowarn_function, rewrite_leaders/2}).
  39. %% ===================================================================
  40. %% Public API
  41. %% ===================================================================
  42. -spec init(rebar_state:t()) -> {ok, rebar_state:t()}.
  43. init(State) ->
  44. State1 = rebar_state:add_provider(
  45. State,
  46. providers:create([
  47. {name, ?PROVIDER},
  48. {module, ?MODULE},
  49. {bare, true},
  50. {deps, ?DEPS},
  51. {example, "rebar3 shell"},
  52. {short_desc, "Run shell with project apps and deps in path."},
  53. {desc, info()},
  54. {opts, [{config, undefined, "config", string,
  55. "Path to the config file to use. Defaults to "
  56. "{shell, [{config, File}]} and then the relx "
  57. "sys.config file if not specified."},
  58. {name, undefined, "name", atom,
  59. "Gives a long name to the node."},
  60. {sname, undefined, "sname", atom,
  61. "Gives a short name to the node."},
  62. {setcookie, undefined, "setcookie", atom,
  63. "Sets the cookie if the node is distributed."},
  64. {script_file, undefined, "script", string,
  65. "Path to an escript file to run before "
  66. "starting the project apps. Defaults to "
  67. "rebar.config {shell, [{script_file, File}]} "
  68. "if not specified."},
  69. {apps, undefined, "apps", string,
  70. "A list of apps to boot before starting the "
  71. "shell. (E.g. --apps app1,app2,app3) Defaults "
  72. "to rebar.config {shell, [{apps, Apps}]} or "
  73. "relx apps if not specified."},
  74. {relname, $r, "relname", atom,
  75. "Name of the release to use as a template for the "
  76. "shell session"},
  77. {relvsn, $v, "relvsn", string,
  78. "Version of the release to use for the shell "
  79. "session"},
  80. {start_clean, undefined, "start-clean", boolean,
  81. "Cancel any applications in the 'apps' list "
  82. "or release."},
  83. {env_file, undefined, "env-file", string,
  84. "Path to file of os environment variables to setup "
  85. "before expanding vars in config files."},
  86. {user_drv_args, undefined, "user_drv_args", string,
  87. "Arguments passed to user_drv start function for "
  88. "creating custom shells."}]}
  89. ])
  90. ),
  91. {ok, State1}.
  92. -spec do(rebar_state:t()) -> {ok, rebar_state:t()} | {error, string()}.
  93. do(Config) ->
  94. shell(Config),
  95. {ok, Config}.
  96. -spec format_error(any()) -> iolist().
  97. format_error(Reason) ->
  98. io_lib:format("~p", [Reason]).
  99. %% NOTE:
  100. %% this is an attempt to replicate `erl -pa ./ebin -pa deps/*/ebin`. it is
  101. %% mostly successful but does stop and then restart the user io system to get
  102. %% around issues with rebar being an escript and starting in `noshell` mode.
  103. %% it also lacks the ctrl-c interrupt handler that `erl` features. ctrl-c will
  104. %% immediately kill the script. ctrl-g, however, works fine
  105. shell(State) ->
  106. setup_name(State),
  107. setup_paths(State),
  108. ShellArgs = debug_get_value(shell_args, rebar_state:get(State, shell, []), undefined,
  109. "Found user_drv args from command line option."),
  110. setup_shell(ShellArgs),
  111. maybe_run_script(State),
  112. %% apps must be started after the change in shell because otherwise
  113. %% their application masters never gets the new group leader (held in
  114. %% their internal state)
  115. maybe_boot_apps(State),
  116. simulate_proc_lib(),
  117. true = register(rebar_agent, self()),
  118. {ok, GenState} = rebar_agent:init(State),
  119. %% Hack to fool the init process into thinking we have stopped and the normal
  120. %% node start process can go on. Without it, init:get_status() always return
  121. %% '{starting, started}' instead of '{started, started}'
  122. init ! {'EXIT', self(), normal},
  123. gen_server:enter_loop(rebar_agent, [], GenState, {local, rebar_agent}, hibernate).
  124. info() ->
  125. "Start a shell with project and deps preloaded similar to~n'erl -pa ebin -pa deps/*/ebin'.~n".
  126. setup_shell(ShellArgs) ->
  127. LoggerState = maybe_remove_logger(),
  128. OldUser = kill_old_user(),
  129. %% Test for support here
  130. NewUser = try erlang:open_port({spawn,"tty_sl -c -e"}, []) of
  131. Port when is_port(Port) ->
  132. true = port_close(Port),
  133. setup_new_shell(ShellArgs)
  134. catch
  135. error:_ ->
  136. setup_old_shell()
  137. end,
  138. rewrite_leaders(OldUser, NewUser),
  139. maybe_reset_logger(LoggerState).
  140. %% @private starting with OTP-21.2.3, there's an oddity where the logger
  141. %% likely tries to handle system logs while we take down the TTY, which
  142. %% ends up hanging the default logger. This function (along with
  143. %% `maybe_reset_logger/1') removes and re-adds the default logger before and
  144. %% after the TTY subsystem is taken offline, which prevents such hanging.
  145. maybe_remove_logger() ->
  146. case erlang:function_exported(logger, module_info, 0) of
  147. false ->
  148. ignore;
  149. true ->
  150. {ok, Cfg} = logger:get_handler_config(default),
  151. logger:remove_handler(default),
  152. {restart, Cfg}
  153. end.
  154. maybe_reset_logger(ignore) ->
  155. ok;
  156. maybe_reset_logger({restart, #{module := Mod, config := Cfg}}) ->
  157. logger:add_handler(default, Mod, Cfg).
  158. kill_old_user() ->
  159. OldUser = whereis(user),
  160. %% terminate the current user's port, in a way that makes it shut down,
  161. %% but without taking down the supervision tree so that the escript doesn't
  162. %% fully die
  163. [P] = [P || P <- element(2,process_info(whereis(user), links)), is_port(P)],
  164. user ! {'EXIT', P, normal}, % pretend the port died, then the port can die!
  165. exit(P, kill),
  166. wait_for_port_death(1000, P),
  167. OldUser.
  168. wait_for_port_death(N, _) when N < 0 ->
  169. %% This risks displaying a warning!
  170. whatever;
  171. wait_for_port_death(N, P) ->
  172. case erlang:port_info(P) of
  173. undefined ->
  174. ok;
  175. _ ->
  176. timer:sleep(10),
  177. wait_for_port_death(N-10, P)
  178. end.
  179. setup_new_shell(ShellArgs) ->
  180. %% terminate the current user supervision structure, if any
  181. _ = supervisor:terminate_child(kernel_sup, user),
  182. %% start a new shell (this also starts a new user under the correct group)
  183. case ShellArgs of
  184. undefined ->
  185. _ = user_drv:start();
  186. _ ->
  187. _ = user_drv:start(ShellArgs)
  188. end,
  189. %% wait until user_drv and user have been registered (max 3 seconds)
  190. ok = wait_until_user_started(3000),
  191. whereis(user).
  192. setup_old_shell() ->
  193. %% scan all processes for any with references to the old user and save them to
  194. %% update later
  195. NewUser = rebar_user:start(), % hikack IO stuff with fake user
  196. NewUser = whereis(user),
  197. NewUser.
  198. rewrite_leaders(OldUser, NewUser) ->
  199. %% set any process that had a reference to the old user's group leader to the
  200. %% new user process. Catch the race condition when the Pid exited after the
  201. %% liveness check.
  202. _ = [catch erlang:group_leader(NewUser, Pid)
  203. || Pid <- erlang:processes(),
  204. [_|_] = Info <- [erlang:process_info(Pid)],
  205. proplists:get_value(group_leader, Info) == OldUser,
  206. is_process_alive(Pid)],
  207. %% Application masters have the same problem, but they hold the old group
  208. %% leader in their state and hold on to it. Re-point the processes whose
  209. %% leaders are application masters. This can mess up a few things around
  210. %% shutdown time, but is nicer than the current lock-up.
  211. OldMasters = [Pid
  212. || Pid <- erlang:processes(),
  213. Pid < NewUser, % only change old masters
  214. {_,Dict} <- [erlang:process_info(Pid, dictionary)],
  215. {application_master,init,4} == proplists:get_value('$initial_call', Dict)],
  216. _ = [catch erlang:group_leader(NewUser, Pid)
  217. || Pid <- erlang:processes(),
  218. lists:member(proplists:get_value(group_leader, erlang:process_info(Pid)),
  219. OldMasters)],
  220. try
  221. case erlang:function_exported(logger, module_info, 0) of
  222. false ->
  223. %% Old style logger had a lock-up issue and other problems related
  224. %% to group leader handling.
  225. %% enable error_logger's tty output
  226. error_logger:swap_handler(tty),
  227. %% disable the simple error_logger (which may have been added
  228. %% multiple times). removes at most the error_logger added by
  229. %% init and the error_logger added by the tty handler
  230. remove_error_handler(3),
  231. %% reset the tty handler once more for remote shells
  232. error_logger:swap_handler(tty);
  233. true ->
  234. %% This is no longer a problem with the logger interface
  235. ok
  236. end
  237. catch
  238. ?WITH_STACKTRACE(E,R,S) % may fail with custom loggers
  239. ?DEBUG("Logger changes failed for ~p:~p (~p)", [E,R,S]),
  240. hope_for_best
  241. end.
  242. setup_paths(State) ->
  243. %% Add deps to path
  244. code:add_pathsa(rebar_state:code_paths(State, all_deps)),
  245. %% add project app test paths
  246. ok = add_test_paths(State).
  247. maybe_run_script(State) ->
  248. case first_value([fun find_script_option/1,
  249. fun find_script_rebar/1], State) of
  250. no_value ->
  251. ?DEBUG("No script_file specified.", []),
  252. ok;
  253. "none" ->
  254. ?DEBUG("Shell script execution skipped (--script none).", []),
  255. ok;
  256. RelFile ->
  257. File = filename:absname(RelFile),
  258. try run_script_file(File)
  259. catch
  260. ?WITH_STACKTRACE(C,E,S)
  261. ?ABORT("Couldn't run shell escript ~p - ~p:~p~nStack: ~p",
  262. [File, C, E, S])
  263. end
  264. end.
  265. -spec find_script_option(rebar_state:t()) -> no_value | list().
  266. find_script_option(State) ->
  267. {Opts, _} = rebar_state:command_parsed_args(State),
  268. debug_get_value(script_file, Opts, no_value,
  269. "Found script file from command line option.").
  270. -spec find_script_rebar(rebar_state:t()) -> no_value | list().
  271. find_script_rebar(State) ->
  272. Config = rebar_state:get(State, shell, []),
  273. %% Either a string, or undefined
  274. debug_get_value(script_file, Config, no_value,
  275. "Found script file from rebar config file.").
  276. run_script_file(File) ->
  277. ?DEBUG("Extracting escript from ~p", [File]),
  278. {ok, Script} = escript:extract(File, [compile_source]),
  279. Beam = proplists:get_value(source, Script),
  280. Mod = proplists:get_value(module, beam_lib:info(Beam)),
  281. ?DEBUG("Compiled escript as ~p", [Mod]),
  282. FakeFile = "/fake_path/" ++ atom_to_list(Mod),
  283. {module, Mod} = code:load_binary(Mod, FakeFile, Beam),
  284. ?DEBUG("Evaling ~p:main([]).", [Mod]),
  285. Result = Mod:main([]),
  286. ?DEBUG("Result: ~p", [Result]),
  287. Result.
  288. maybe_boot_apps(State) ->
  289. _ = maybe_set_env_vars(State),
  290. case find_apps_to_boot(State) of
  291. undefined ->
  292. %% try to read in sys.config file
  293. ok = reread_config([], State);
  294. Apps ->
  295. %% load apps, then check config, then boot them.
  296. load_apps(Apps),
  297. ok = reread_config(Apps, State),
  298. boot_apps(Apps)
  299. end.
  300. simulate_proc_lib() ->
  301. FakeParent = spawn_link(fun() -> timer:sleep(infinity) end),
  302. put('$ancestors', [FakeParent]),
  303. put('$initial_call', {rebar_agent, init, 1}).
  304. setup_name(State) ->
  305. {Long, Short, Opts} = rebar_dist_utils:find_options(State),
  306. rebar_dist_utils:either(Long, Short, Opts).
  307. find_apps_to_boot(State) ->
  308. %% Try the shell_apps option
  309. case first_value([fun find_apps_option/1,
  310. fun find_apps_rebar/1,
  311. fun find_apps_relx/1], State) of
  312. no_value ->
  313. undefined;
  314. Apps ->
  315. Apps
  316. end.
  317. -spec find_apps_option(rebar_state:t()) -> no_value | [atom()].
  318. find_apps_option(State) ->
  319. {Opts, _} = rebar_state:command_parsed_args(State),
  320. case debug_get_value(apps, Opts, no_value,
  321. "Found shell apps from command line option.") of
  322. no_value ->
  323. case debug_get_value(start_clean, Opts, false,
  324. "Found start-clean argument to disable apps") of
  325. false -> no_value;
  326. true -> []
  327. end;
  328. AppsStr ->
  329. [ list_to_atom(AppStr)
  330. || AppStr <- rebar_string:lexemes(AppsStr, " ,:") ]
  331. end.
  332. -spec find_apps_rebar(rebar_state:t()) -> no_value | list().
  333. find_apps_rebar(State) ->
  334. ShellOpts = rebar_state:get(State, shell, []),
  335. debug_get_value(apps, ShellOpts, no_value,
  336. "Found shell opts from command line option.").
  337. -spec find_apps_relx(rebar_state:t()) -> no_value | list().
  338. find_apps_relx(State) ->
  339. {Opts, _} = rebar_state:command_parsed_args(State),
  340. RelxOpts = rebar_state:get(State, relx, []),
  341. {Defname, Defvsn} = debug_get_value(default_release, RelxOpts,
  342. {undefined, undefined},
  343. "Found default release from config"),
  344. Relname = debug_get_value(relname, Opts, Defname,
  345. "Found relname from command line option"),
  346. Relvsn = debug_get_value(relvsn, Opts, Defvsn,
  347. "Found relvsn from command line option"),
  348. Releases = [Rel || Rel <- rebar_state:get(State, relx, []),
  349. is_tuple(Rel), element(1, Rel) =:= release,
  350. tuple_size(Rel) =:= 3 orelse tuple_size(Rel) =:= 4,
  351. {Name, Vsn} <- [element(2, Rel)],
  352. Relname == undefined orelse Name == Relname,
  353. Relvsn == undefined orelse Vsn == Relvsn],
  354. case Releases of
  355. [] ->
  356. no_value;
  357. [{_, _, Apps}|_] ->
  358. ?DEBUG("Found shell apps from relx.", []),
  359. Apps;
  360. [{_, _, Apps, _}|_] ->
  361. ?DEBUG("Found shell apps from relx.", []),
  362. Apps
  363. end.
  364. load_apps(Apps) ->
  365. [case application:load(App) of
  366. ok ->
  367. {ok, Ks} = application:get_all_key(App),
  368. load_apps(proplists:get_value(applications, Ks));
  369. _ ->
  370. error % will be caught when starting the app
  371. end || App <- normalize_load_apps(Apps),
  372. not lists:keymember(App, 1, application:loaded_applications())],
  373. ok.
  374. reread_config(AppsToStart, State) ->
  375. case find_config(State) of
  376. no_config ->
  377. ok;
  378. ConfigList ->
  379. %% This allows people who use applications that are also
  380. %% depended on by rebar3 or its plugins to change their
  381. %% configuration at runtime based on the configuration files.
  382. %%
  383. %% To do this, we stop apps that are already started before
  384. %% reloading their configuration.
  385. %%
  386. %% We make an exception for apps that:
  387. %% - are not already running
  388. %% - would not be restarted (and hence would break some
  389. %% compatibility with rebar3)
  390. %% - are not in the config files and would see no config
  391. %% changes
  392. %% - are not in a blacklist, where changing their config
  393. %% would be risky to the shell or the rebar3 agent
  394. %% functionality (i.e. changing inets may break proxy
  395. %% settings, stopping `kernel' would break everything)
  396. Running = [App || {App, _, _} <- application:which_applications()],
  397. BlackList = [inets, stdlib, kernel, rebar],
  398. _ = [application:stop(App)
  399. || Config <- ConfigList,
  400. {App, _} <- Config,
  401. lists:member(App, Running),
  402. lists:member(App, AppsToStart),
  403. not lists:member(App, BlackList)],
  404. _ = rebar_utils:reread_config(ConfigList, [update_logger]),
  405. ok
  406. end.
  407. boot_apps(Apps) ->
  408. Normalized = normalize_boot_apps(Apps),
  409. Res = [application:ensure_all_started(App) || App <- Normalized],
  410. _ = [?INFO("Booted ~p", [App])
  411. || {ok, Booted} <- Res,
  412. App <- Booted],
  413. _ = [?ERROR("Failed to boot ~p for reason ~p", [App, Reason])
  414. || {error, {App, Reason}} <- Res],
  415. ok.
  416. normalize_load_apps([]) -> [];
  417. normalize_load_apps([{_App, none} | T]) -> normalize_load_apps(T);
  418. normalize_load_apps([{App, _} | T]) -> [App | normalize_load_apps(T)];
  419. normalize_load_apps([{App, _Vsn, load} | T]) -> [App | normalize_load_apps(T)];
  420. normalize_load_apps([{_App, _Vsn, none} | T]) -> normalize_load_apps(T);
  421. normalize_load_apps([{App, _Vsn, Operator} | T]) when is_atom(Operator) ->
  422. [App | normalize_load_apps(T)];
  423. normalize_load_apps([App | T]) when is_atom(App) -> [App | normalize_load_apps(T)].
  424. normalize_boot_apps([]) -> [];
  425. normalize_boot_apps([{_App, load} | T]) -> normalize_boot_apps(T);
  426. normalize_boot_apps([{_App, _Vsn, load} | T]) -> normalize_boot_apps(T);
  427. normalize_boot_apps([{_App, none} | T]) -> normalize_boot_apps(T);
  428. normalize_boot_apps([{_App, _Vsn, none} | T]) -> normalize_boot_apps(T);
  429. normalize_boot_apps([{App, _Vsn, Operator} | T]) when is_atom(Operator) ->
  430. [App | normalize_boot_apps(T)];
  431. normalize_boot_apps([{App, _Vsn} | T]) -> [App | normalize_boot_apps(T)];
  432. normalize_boot_apps([App | T]) when is_atom(App) -> [App | normalize_boot_apps(T)].
  433. remove_error_handler(0) ->
  434. ?WARN("Unable to remove simple error_logger handler", []);
  435. remove_error_handler(N) ->
  436. case gen_event:delete_handler(error_logger, error_logger, []) of
  437. {error, module_not_found} -> ok;
  438. {error_logger, _} -> remove_error_handler(N-1)
  439. end.
  440. %% Timeout is a period to wait before giving up
  441. wait_until_user_started(0) ->
  442. ?ABORT("Timeout exceeded waiting for `user` to register itself", []),
  443. erlang:error(timeout);
  444. wait_until_user_started(Timeout) ->
  445. case whereis(user) of
  446. %% if user is not yet registered wait a tenth of a second and try again
  447. undefined -> timer:sleep(100), wait_until_user_started(Timeout - 100);
  448. _ -> ok
  449. end.
  450. add_test_paths(State) ->
  451. _ = [begin
  452. AppDir = rebar_app_info:out_dir(App),
  453. %% ignore errors resulting from non-existent directories
  454. _ = code:add_path(filename:join([AppDir, "test"]))
  455. end || App <- rebar_state:project_apps(State)],
  456. _ = code:add_path(filename:join([rebar_dir:base_dir(State), "test"])),
  457. ok.
  458. % First try the --config flag, then try the relx sys_config
  459. -spec find_config(rebar_state:t()) -> [[tuple()]] | no_config.
  460. find_config(State) ->
  461. case first_value([fun find_config_option/1,
  462. fun find_config_rebar/1,
  463. fun find_config_relx/1], State) of
  464. no_value ->
  465. no_config;
  466. Filename when is_list(Filename) ->
  467. case is_src_config(Filename) of
  468. false ->
  469. rebar_file_utils:consult_config(State, Filename);
  470. true ->
  471. consult_env_config(State, Filename)
  472. end
  473. end.
  474. -spec first_value([Fun], State) -> no_value | Value when
  475. Value :: any(),
  476. State :: rebar_state:t(),
  477. Fun :: fun ((State) -> no_value | Value).
  478. first_value([], _) -> no_value;
  479. first_value([Fun | Rest], State) ->
  480. case Fun(State) of
  481. no_value ->
  482. first_value(Rest, State);
  483. Value ->
  484. Value
  485. end.
  486. debug_get_value(Key, List, Default, Description) ->
  487. case proplists:get_value(Key, List, Default) of
  488. Default -> Default;
  489. Value ->
  490. ?DEBUG(Description, []),
  491. Value
  492. end.
  493. -spec find_config_option(rebar_state:t()) -> Filename::list() | no_value.
  494. find_config_option(State) ->
  495. {Opts, _} = rebar_state:command_parsed_args(State),
  496. debug_get_value(config, Opts, no_value,
  497. "Found config from command line option.").
  498. -spec find_config_rebar(rebar_state:t()) -> [tuple()] | no_value.
  499. find_config_rebar(State) ->
  500. debug_get_value(config, rebar_state:get(State, shell, []), no_value,
  501. "Found config from rebar config file.").
  502. -spec find_config_relx(rebar_state:t()) -> [tuple()] | no_value.
  503. find_config_relx(State) ->
  504. %% The order in relx is to load the src version first;
  505. %% we do the same.
  506. RelxCfg = rebar_state:get(State, relx, []),
  507. Src = debug_get_value(sys_config_src, RelxCfg, no_value,
  508. "Found config.src from relx."),
  509. case Src of
  510. no_value ->
  511. debug_get_value(sys_config, RelxCfg, no_value,
  512. "Found config from relx.");
  513. _ ->
  514. Src
  515. end.
  516. -spec is_src_config(file:filename()) -> boolean().
  517. is_src_config(Filename) ->
  518. filename:extension(Filename) =:= ".src".
  519. -spec consult_env_config(rebar_state:t(), file:filename()) -> [[tuple()]].
  520. consult_env_config(State, Filename) ->
  521. RawString = case file:read_file(Filename) of
  522. {error, _} -> "[].";
  523. {ok, Bin} -> unicode:characters_to_list(Bin)
  524. end,
  525. ReplacedStr = replace_env_vars(RawString),
  526. case rebar_string:consult(unicode:characters_to_list(ReplacedStr)) of
  527. {error, Reason} ->
  528. throw(?PRV_ERROR({bad_term_file, Filename, Reason}));
  529. [Terms] ->
  530. rebar_file_utils:consult_config_terms(State, Terms)
  531. end.
  532. maybe_set_env_vars(State) ->
  533. EnvFile =debug_get_value(env_file, rebar_state:get(State, shell, []), undefined,
  534. "Found env_file from config."),
  535. {Opts, _} = rebar_state:command_parsed_args(State),
  536. EnvFile1 = debug_get_value(env_file, Opts, EnvFile,
  537. "Found env_file from command line option."),
  538. case maybe_read_file(EnvFile1) of
  539. ignore ->
  540. ok;
  541. {error, _} ->
  542. ?WARN("Failed to read file with environment variables: ~p", [EnvFile1]);
  543. {ok, Bin} ->
  544. Lines = string:split(unicode:characters_to_list(Bin), "\n", all),
  545. [handle_env_var_line(Line) || Line <- Lines]
  546. end.
  547. handle_env_var_line(Line) ->
  548. Trimmed = rebar_string:trim(Line, both, [$\s]),
  549. %% ignore lines starting with # and
  550. %% fail if there are spaces around =
  551. case re:run(Trimmed, "^(?<key>[^#][^\s=]*)=(?<value>[^\s]\.*)",
  552. [{capture, [key, value], list}, unicode]) of
  553. {match, [Key, Value]} ->
  554. os:putenv(Key, Value);
  555. _ ->
  556. case Trimmed of
  557. [$# | _] -> ignore;
  558. [] -> ignore;
  559. Other ->
  560. ?WARN("Unable to parse environment variable from this line: ~ts", [Other])
  561. end
  562. end.
  563. maybe_read_file(undefined) ->
  564. ignore;
  565. maybe_read_file(EnvFile) ->
  566. file:read_file(EnvFile).
  567. %% @doc quick and simple variable substitution writeup.
  568. %% Supports `${varname}' but not `$varname' nor nested
  569. %% values such as `${my_${varname}}'.
  570. %% The variable are also defined as only supporting
  571. %% the form `[a-zA-Z_]+[a-zA-Z0-9_]*' as per the POSIX
  572. %% standard.
  573. -spec replace_env_vars(string()) -> unicode:charlist().
  574. replace_env_vars("") -> "";
  575. replace_env_vars("${" ++ Str) ->
  576. case until_var_end(Str) of
  577. {ok, VarName, Rest} ->
  578. replace_varname(VarName) ++ replace_env_vars(Rest);
  579. error ->
  580. "${" ++ replace_env_vars(Str)
  581. end;
  582. replace_env_vars([Char|Str]) ->
  583. [Char | replace_env_vars(Str)].
  584. until_var_end(Str) ->
  585. case re:run(Str, "([a-zA-Z_]+[a-zA-Z0-9_]*)}", [{capture, [1], list}]) of
  586. nomatch ->
  587. error;
  588. {match, [Name]} ->
  589. {ok, Name, drop_varname(Name, Str)}
  590. end.
  591. replace_varname(Var) ->
  592. %% os:getenv(Var, "") is only available in OTP-18.0
  593. case os:getenv(Var) of
  594. false -> "";
  595. Val -> Val
  596. end.
  597. drop_varname("", "}" ++ Str) -> Str;
  598. drop_varname([_|Var], [_|Str]) -> drop_varname(Var, Str).