lager/README.org

* Overview
  Lager (as in the beer) is a logging framework for Erlang. Its purpose is
  to provide a more traditional way to perform logging in an erlang application
  that plays nicely with traditional UNIX logging tools like logrotate and
  syslog. You can view the Lager EDocs [[http://basho.github.com/lager/][here]].

  [[http://travis-ci.org/basho/lager][Travis-CI]] :: [[https://secure.travis-ci.org/basho/lager.png]]

* Features
    - Finer grained log levels (debug, info, notice, warning, error, critical,
      alert, emergency)
    - Logger calls are transformed using a parse transform to allow capturing
      Module/Function/Line/Pid information
    - When no handler is consuming a log level (eg. debug) no event is even sent
      to the log handler
    - Supports multiple backends, including console and file. More are planned.

* Usage
  To use lager in your application, you need to define it as a rebar dep or have
  some other way of including it in erlang's path. You can then add the
  following option to the erlang compiler flags

#+BEGIN_EXAMPLE
  {parse_transform, lager_transform}
#+END_EXAMPLE

  Alternately, you can add it to the module you wish to compile with logging
  enabled:

#+BEGIN_EXAMPLE
  -compile([{parse_transform, lager_transform}]).
#+END_EXAMPLE

  Before logging any messages, you'll need to start the lager application. The 
  lager module's start function takes care of loading and starting any dependencies
  lager requires.
  
#+BEGIN_EXAMPLE
  lager:start().
#+END_EXAMPLE

  Once you have built your code with lager and started the lager application, 
  you can then generate log messages by doing the following:

#+BEGIN_EXAMPLE
  lager:error("Some message")
#+END_EXAMPLE

  Or:

#+BEGIN_EXAMPLE
  lager:warning("Some message with a term: ~p", [Term])
#+END_EXAMPLE

  The general form is lager:Severity() where Severity is one of the log levels
  mentioned above.

* Configuration
  To configure lager's backends, you use an application variable (probably in
  your app.config):

#+BEGIN_EXAMPLE
  {lager, [
    {handlers, [
      {lager_console_backend, info},
      {lager_file_backend, [
        {"error.log", error, 10485760, "$D0", 5},
        {"console.log", info, 10485760, "$D0", 5}
      ]}
    ]}
  ]}.
#+END_EXAMPLE

  The available configuration options for each backend are listed in their
  module's documentation.

* Custom Formatting
  All loggers have a default formatting that can be overriden.  A formatter is any module that
  exports format(#lager_log_message{},Config#any()).  It is specified as part of the configuration
  for the backend:

#+BEGIN_EXAMPLE
  {lager, [
    {handlers, [
      {lager_console_backend, [info, {lager_default_formatter, [time," [",severity,"] ", message, "\n"]}},
      {lager_file_backend, [
        [{"error.log", error, 10485760, "$D0", 5,{lager_default_formatter,[date, " ", time," [",severity,"] ",pid, " ", message, "\n"]}],
        {"console.log", info, 10485760, "$D0", 5}
      ]}
    ]}
  ]}.
#+END_EXAMPLE  
  
  Included is lager_default_formatter.  This provides a generic, default formatting for log messages using a "semi-iolist" 
  as configuration.  Any iolist allowed elements in the configuration are printed verbatim.  Atoms in the configuration 
  are treated as metadata properties and extracted from the log message.  
  The metadata properties date,time, message, and severity will always exist.  
  The properties pid, file, line, module, and function will always exist if the parser transform is used.

#+BEGIN_EXAMPLE
    ["Foo"] -> "Foo", regardless of message content.
    [message] -> The content of the logged message, alone.
    [{pid,"Unknown Pid"}] -> "<?.?.?>" if pid is in the metadata, "Unknown Pid" if not.
    [{pid, ["My pid is ", pid], "Unknown Pid"}] -> if pid is in the metadata print "My pid is <?.?.?>", otherwise print "Unknown Pid"
#+END_EXAMPLE  

  Optionally, a tuple of {atom(),semi-iolist()}   
  can be used.  The atom will look up the property, but if not found it will use the semi-iolist() instead.  These fallbacks 
  can be nested or refer to other properties.

#+BEGIN_EXAMPLE
    [{pid,"Unknown Pid"}] -> "<?.?.?>" if pid is in the metadata, "Unknown Pid" if not.
    [{server,[$(,{pid,"Unknown Server"},$)]}}] -> user provided server metadata, otherwise "(<?.?.?>)", otherwise "(Unknown Server)"
#+END_EXAMPLE  

* Error logger integration
  Lager is also supplied with a error_logger handler module that translates
  traditional erlang error messages into a friendlier format and sends them into
  lager itself to be treated like a regular lager log call. To disable this, set
  the lager application variable `error_logger_redirect' to `false'.

  The error_logger handler will also log more complete error messages (protected
  with use of trunc_io) to a "crash log" which can be referred to for further
  information. The location of the crash log can be specified by the crash_log
  application variable. If undefined it is not written at all.

  Messages in the crash log are subject to a maximum message size which can be
  specified via the crash_log_msg_size application variable.

* Runtime loglevel changes
  You can change the log level of any lager backend at runtime by doing the
  following:

#+BEGIN_EXAMPLE
  lager:set_loglevel(lager_console_backend, debug).
#+END_EXAMPLE

  Or, for the backend with multiple handles (files, mainly):

#+BEGIN_EXAMPLE
  lager:set_loglevel(lager_file_backend, "console.log", debug).
#+END_EXAMPLE

  Lager keeps track of the minium log level being used by any backend and
  supresses generation of messages lower than that level. This means that debug
  log messages, when no backend is consuming debug messages, are effectively
  free. A simple benchmark of doing 1 million debug log messages while the
  minimum threshold was above that takes less than half a second.

* Internal log rotation
  Lager can rotate its own logs or have it done via an external process. To
  use internal rotation, use the last 3 values in the file backend's
  configuration tuple. For example

#+BEGIN_EXAMPLE
  {"error.log", error, 10485760, "$D0", 5}
#+END_EXAMPLE

  This tells lager to log error and above messages to "error.log" and to
  rotate the file at midnight or when it reaches 10mb, whichever comes first
  and to keep 5 rotated logs, in addition to the current one. Setting the
  count to 0 does not disable rotation, it instead rotates the file and keeps
  no previous versions around. To disable rotation set the size to 0 and the
  date to "".

  The "$D0" syntax is taken from the syntax newsyslog uses in newsyslog.conf.
  The relevant extract follows:

#+BEGIN_EXAMPLE
  Day, week and month time format: The lead-in character
  for day, week and month specification is a `$'-sign.
  The particular format of day, week and month
  specification is: [Dhh], [Ww[Dhh]] and [Mdd[Dhh]],
  respectively.  Optional time fields default to
  midnight.  The ranges for day and hour specifications
  are:

    hh      hours, range 0 ... 23
    w       day of week, range 0 ... 6, 0 = Sunday
    dd      day of month, range 1 ... 31, or the
            letter L or l to specify the last day of
            the month.

  Some examples:
    $D0     rotate every night at midnight
    $D23    rotate every day at 23:00 hr
    $W0D23  rotate every week on Sunday at 23:00 hr
    $W5D16  rotate every week on Friday at 16:00 hr
    $M1D0   rotate on the first day of every month at
            midnight (i.e., the start of the day)
    $M5D6   rotate on every 5th day of the month at
            6:00 hr
#+END_EXAMPLE

  To configure the crash log rotation, the following application variables are
  used:
  - crash_log_size
  - crash_log_date
  - crash_log_count

  See the .app.src file for further details.

* Syslog Support
  Lager syslog output is provided as a separate application;
  [[https://github.com/basho/lager_syslog][lager_syslog]]. It is packaged as a
  separate application so Lager itself doesn't have an indirect dependancy on a
  port driver. Please see the lager_syslog README for configuration information.

* AMQP Support
  Jon Brisbin has written a lager backend to send lager messages into AMQP, so
  you can aggregate logs from a cluster into a central point. You can find it
  under the [[https://github.com/jbrisbin/lager_amqp_backend][lager_amqp_backend]]
  project on github.

* Loggly Support
  The team at [[https://www.kivra.com][KIVRA]] has written a lager backend to send
  lager messages into [[http://www.loggly.com][Loggly]]. You can find it
  under the [[https://github.com/kivra/lager_loggly][lager_loggly]]
  project on github.

* Tracing
  Lager supports basic support for redirecting log messages based on log message
  attributes. Lager automatically captures the pid, module, function and line at the
  log message callsite. However, you can add any additional attributes you wish:

#+BEGIN_EXAMPLE
  lager:warning([{request, RequestID},{vhost, Vhost}], "Permission denied to ~s", [User])
#+END_EXAMPLE

  Then, in addition to the default trace attributes, you'll be able to trace
  based on request or vhost:

#+BEGIN_EXAMPLE
  lager:trace_file("logs/example.com.error", [{vhost, "example.com"}], error)
#+END_EXAMPLE

  You can also omit the final argument, and the loglevel will default to
  'debug'.

  Tracing to the console is similar:

#+BEGIN_EXAMPLE
  lager:trace_console([{request, 117}])
#+END_EXAMPLE

  In the above example, the loglevel is omitted, but it can be specified as the
  second argument if desired.

  You can also specify multiple expressions in a filter, or use the '*' atom as
  a wildcard to match any message that has that attribute, regardless of its
  value.

  Tracing to an existing logfile is also supported, if you wanted to log
  warnings from a particular module to the default error.log:

#+BEGIN_EXAMPLE
  lager:trace_file("log/error.log", [{module, mymodule}], warning)
#+END_EXAMPLE

  To view the active log backends and traces, you can use the lager:status()
  function. To clear all active traces, you can use lager:clear_all_traces().

  To delete a specific trace, store a handle for the trace when you create it,
  that you later pass to lager:stop_trace/1:

#+BEGIN_EXAMPLE
  {ok, Trace} = lager:trace_file("log/error.log", [{module, mymodule}]),
  ...
  lager:stop_trace(Trace)
#+END_EXAMPLE

  Tracing to a pid is somewhat of a special case, since a pid is not a
  data-type that serializes well. To trace by pid, use the pid as a string:

#+BEGIN_EXAMPLE
  lager:trace_console([{pid, "<0.410.0>"}])
#+END_EXAMPLE

* Setting the truncation limit at compile-time
  Lager defaults to truncating messages at 4096 bytes, you can alter this by
  using the {lager_truncation_size, X} option. In rebar, you can add it to
  erl_opts:

#+BEGIN_EXAMPLE
  {erl_opts, [{parse_transform, lager_transform}, {lager_truncation_size, 1024}]}.
#+END_EXAMPLE

  You can also pass it to erlc, if you prefer:

#+BEGIN_EXAMPLE
  erlc -pa lager/ebin +'{parse_transform, lager_transform}' +'{lager_truncation_size, 1024}' file.erl
#+END_EXAMPLE