Move the Erlang services off lager onto OTP logger - #4246
Open
tas50 wants to merge 1 commit into
Open
Conversation
Replaces every lager call in oc_erchef, oc_bifrost and bookshelf with
the OTP logger macros from kernel/include/logger.hrl, and drops the
lager dependency, its parse transform and its application entries.
86 call sites across 31 files, 5 rebar.config files and 6 .app.src.
lager is unmaintained and reaches inside OTP rather than merely using
it: lager_stdlib.erl is a verbatim copy of unexported OTP stdlib
functions (Copyright Ericsson AB 1996-2009) and lager_format.erl a fork
of io_lib_format (1996-2011-2012); lager_app starts the deprecated
error_logger, installs OTP's internal error_logger module as a logger
handler by hand rather than through error_logger:add_report_handler,
then calls logger:remove_handler(default) to delete OTP's own default
handler -- its own comment calls that "a band-aid" for OTP 21; and
error_logger_lager_h pattern-matches OTP's internal report tuples.
Upstream's last commit is e6b3178 (2023-11-02).
To be precise about scope: lager compiles fine on OTP 27. This is not a
compile-time blocker. The case against it is unmaintained code reaching
into OTP internals -- runtime fragility and accumulating risk.
The macros rather than logger:Level/2 calls: ?LOG_ERROR and friends
capture module, function and line, which is the job lager's parse
transform did. Plain function calls lose that silently.
Per service:
- oc_erchef: 74 sites across 25 files, all plain level calls.
lager:start/0 goes from a test helper; logger is part of kernel and
always running.
- bookshelf: 10 sites. internal.hrl already defined ?LOG_INFO,
?LOG_ERROR and ?LOG_DEBUG at exactly the arities logger.hrl
provides, so including it there migrates every existing call site
untouched and retires the deprecated error_logger:info_msg and
error_msg it used for two of them.
- oc_bifrost: 2 sites.
Several of these are only findable by building rather than reading:
- goldrush in oc_chef_wm.app.src is lager's own dependency. Left
behind it breaks rebar3 dialyzer ("Could not find application:
goldrush") and rebar3 release ("Application needed for release not
found"), after every unit test has already passed.
- apps/data_collector and apps/chef_telemetry each declare lager and
the parse transform in their own rebar.config, independently of the
top-level one, on a rolling master branch.
- Six oc_chef_wm itest suites carry the parse transform, setup_helper
sets lager's error_logger_redirect and starts lager and goldrush,
and the bookshelf SUITE calls lager_common_test_backend:bounce/1.
None are built by the default profile.
- The relx release app list in oc_bifrost/rebar.config.
- Every -include_lib was checked to sit outside -ifdef blocks; one
placed inside -ifdef(TEST) vanishes from normal builds.
Verified in erlang:26 containers from a clean _build: oc_bifrost,
bookshelf and oc_erchef all compile with warnings_as_errors on,
oc_erchef assembles its release, and its dialyzer run is clean over 160
files.
This does not remove lager from the build yet. chef_secrets,
folsom_graphite and opscoderl_httpc still declare it and pull it in
transitively, so it stays in _build and in the assembled release until
those three land (chef/chef_secrets#88, chef/folsom_graphite#16,
chef/opscoderl_httpc#21). The lockfiles are deliberately untouched here:
re-pinning now would point them at fork branches.
The sys.config templates carrying the {lager, [...]} blocks live in
chef-server-omnibus-config, not this repo, so converting those to logger
handlers is a separate change there.
Signed-off-by: Tim Smith <tim@mondoo.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces every
lagercall inoc_erchef,oc_bifrostandbookshelfwith the OTP logger macros fromkernel/include/logger.hrl, and drops the lager dependency, its parse transform and its application entries.86 call sites across 31 files, plus 5
rebar.configs and 6.app.srcfiles.Why
lageris unmaintained and reaches inside OTP in four places rather than merely using it:lager_stdlib.erlis a verbatim copy of unexported OTP stdlib functions — "Functions from Erlang OTP distribution that are really useful but aren't exported", Copyright Ericsson AB 1996-2009 — andlager_format.erlis a fork ofio_lib_format(1996-2011-2012).lager_appstarts the deprecatederror_logger, installs OTP's internalerror_loggermodule as a logger handler by hand rather than througherror_logger:add_report_handler, then callslogger:remove_handler(default)to delete OTP's own default handler. Its own comment calls this "a band-aid" for OTP 21.error_logger_lager_hpattern-matches OTP's internal{error_report,_}/{info_report,_}shapes.lager_transformrewrites the AST at compile time — hence "lager has to come first" in everyrebar.config.Upstream's last commit is
e6b3178(2023-11-02), before OTP 27 existed, so none of that is being re-synced.To be precise about scope: lager compiles fine on OTP 27 — I checked. This is not a compile-time blocker. The case is unmaintained code reaching into OTP internals, which is runtime fragility and accumulating risk, not a build break.
?LOG_*macros rather thanlogger:Level/2calls: the macros capture module, function and line, which is the joblager_transformdid. Plain function calls silently lose that metadata.Per service
lager:md, sinks or traces).lager:start/0goes from a test helper; logger is part of kernel and always running.internal.hrlalready defined?LOG_INFO/?LOG_ERROR/?LOG_DEBUGat exactly the aritieslogger.hrlprovides, so including it there migrates every existing?LOG_*call site untouched and retires the deprecatederror_logger:info_msg/error_msgit used for two of them.Things only a build finds
These are not in the obvious places and are why this touches 31 files rather than 30:
goldrushinoc_chef_wm.app.src— lager's own dependency. Leaving it behind breaksrebar3 dialyzer("Could not find application: goldrush") andrebar3 release("Application needed for release not found"), after every unit test has already passed. Removed.rebar.configs —apps/data_collectorandapps/chef_telemetryeach declare lager and the parse transform independently of the top-level config, on a rollingmasterbranch.oc_chef_wmitest suites carrying-compile([{parse_transform, lager_transform}]),setup_helpersetting lager'serror_logger_redirectand starting lager/goldrush, and the bookshelf SUITE callinglager_common_test_backend:bounce/1. None are built by the default profile.oc_bifrost/rebar.config.-include_libwas checked to be outside-ifdefblocks — an include placed inside-ifdef(TEST)silently vanishes from normal builds.Verified
erlang:26containers, clean_build:oc_bifrost,bookshelfandoc_erchefall compile withwarnings_as_errorson.oc_erchefassembles:Release successfully assembled: _build/default/rel/oc_erchef.oc_erchefdialyzer is clean — 160 files analysed, exit 0. This matters specifically:Resolving project filesis the step that fails on a stalegoldrushentry, so a clean run here is what confirms the.app.srccleanup is complete.This does not remove lager from the build yet
Three libraries still declare lager and are pulled in transitively, so it remains in
_buildand in the assembled release until they land:chef_secretsapplicationsentry, which is what starts lagerfolsom_graphiteopscoderl_httpcMerge those three first. The lockfiles are deliberately left alone here — re-pinning them now would mean pointing at fork branches.
The
sys.configtemplates that carry the{lager, [...]}blocks live inchef-server-omnibus-config, not this repo, so converting those to logger handlers (and moving lager's$D0midnight rotation to logrotate, sincelogger_std_hhas no time trigger) is a separate change there.CI note
buildkite and the Grype scan fail on this PR. They fail identically on every open PR in this repo, including #4239, #4230 and #4209 — Grype reports
1 CRITICAL, 3 HIGHfrom the existing dependency tree, gated bygrype-fail-on-high: true. Neither is related to this change.