Skip to content
Merged

Log #100

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/cn/streaming_log.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,10 @@ CHECK(x > y); // Check failed: x > y.

如果重命名失败(比如日志所在目录不可写),brpc会打印一条错误,然后等文件又长了一个`log_rotate_size_mb`再重试,而不是每写一条日志都去关闭、重开、重命名一次文件。期间日志继续追加到当前文件,不会丢。如果某个历史文件因为被其它进程占用等原因移不走,切分会整体放弃这一次操作,而不会用当前文件把它覆盖掉,因此不会悄悄丢掉一代日志。如果发现文件比失败时更小,说明它已经被外部替换或截断,切分立刻恢复正常。

切分过程中不会提前关闭日志句柄:POSIX允许rename一个正在打开的文件,所以即使rename失败,或者路径因为权限收紧(比如运行中文件变成0400、目录变成0500)或进程降权而无法再次打开,已有的句柄仍然可以继续写,日志不会丢。新建的文件会沿用被替换文件的权限,一个0600的私有日志不会因为切分变成0644而对其他用户可读。

历史文件的清理按目录里实际存在的文件进行,而不是假设序号连续:某次移位中途失败留下的空洞不会让更老的日志永久残留,`log_rotate_max_backups`设得再大也不会让每次切分去尝试那么多次rename。

> 不要在同一个日志文件上同时使用logrotate等外部切分工具。外部工具把文件改名之后,brpc下一次写入会重新创建原路径的文件,两边的切分规则会互相干扰,保留下来的历史文件数也不再受`log_rotate_max_backups`控制。二选一。

两个flag都可以在运行时通过`/flags`修改。
Expand Down
4 changes: 4 additions & 0 deletions docs/en/streaming_log.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,10 @@ Rotation is disabled by default, so upgrading changes neither the file names nor

If the rename fails, for example because the directory holding the log file is not writable, brpc prints an error and retries only once the file has grown by another `log_rotate_size_mb`, instead of closing, reopening and renaming the file for every single log. Logs keep being appended to the current file in the meantime, none of them are lost. A backup that cannot be renamed out of the way, for example because another process holds it open, stops the rotation altogether rather than being overwritten with the current file, so no generation is silently dropped. If the file turns out to be smaller than it was when rotation failed, it has been replaced or truncated from the outside and rotation resumes immediately.

The handle is never closed ahead of the rename: POSIX allows renaming a file that is still open, so even when the rename fails, or when the path can no longer be opened because permissions were tightened (the file turned 0400 or the directory 0500 under a running process) or privileges were dropped, the handle that is already open keeps accepting logs and none of them are lost. The fresh file is created with the permissions of the file it replaces, so a private 0600 log file does not turn into a world readable 0644 one on the first rotation.

Old backups are cleaned up by listing what is really in the directory instead of assuming that the indexes are contiguous: a hole left behind by a partly failed shift cannot keep the older generations around forever, and a large `log_rotate_max_backups` does not make every rotation attempt that many renames.

> Do not run an external rotator such as logrotate on the same log file. Once an external tool renames the file, the next write recreates it under the original name, the two sets of rotation rules interfere with each other and the number of files kept around is no longer bounded by `log_rotate_max_backups`. Use one or the other.

Both flags are modifiable at run time via `/flags`.
Expand Down
Loading
Loading