fix: port upstream log fixes, JSON log attributes and trace context (B7 of #369)

[mayankpande88]

Summary

B7 of #369: the log fixes from upstream coroot-node-agent, on top of nudgebee/logparser#34, which merges coroot/logparser v1.4.2 into our logparser fork. Merge nudgebee/logparser#34 first, with a merge commit. go.mod here pins its head commit.

Upstream	Change
coroot/coroot-node-agent@e1cd714	JSON logs are exported over OTLP with their message as the body, their level as the severity and the other fields as attributes. --disable-json-log-parsing turns this off.
coroot/coroot-node-agent@18cfe3c	A trace_id/span_id field in a JSON log sets the OTLP record's trace context.
coroot/coroot-node-agent@cd65ce5	A container log file that doesn't exist yet is retried instead of failing.
coroot/coroot-node-agent@ab9b049	--log-pattern-extraction-limit: a per-container cap on pattern extraction under log storms. Default 0 (off) here; upstream uses 100/s.
coroot/coroot-node-agent@e4be150	When the fd of a just-opened /var/log file is already gone (freopen), the process's open log files are found from /proc/<pid>/fd.

All five are by Nikolay Sivko.

What changes by default:

• JSON logs: the OTLP log body for a JSON line is now its message, with the other fields as attributes and the trace context set. OTLP log export is off by default (--logs-endpoint unset), so this only affects installs that send logs.
• Pattern metrics: container_log_messages_total keeps the same pattern hashes.

Engineering detail

Why the extraction limit defaults to 0:

• Measured: in a storm of about 59,000 error lines in 30s, 100/s saved 139 ms of agent CPU (about 4.6 millicores).
• Cost: 55,070 of those messages were counted under the "event was sampled" pattern, so the storm's own pattern showed about 4,000.
• Conclusion: pattern extraction is cheap here, and pattern spikes are what matters, so the flag stays for installs that need the cap.

Over the limit: the logparser sync emits over-limit messages instead of dropping them (see nudgebee/logparser#34).

Conflicts with this fork:

• Logparser import: this fork imports github.com/nudgebee/logparser (upstream imports github.com/coroot/logparser), so upstream's go.mod bumps are replaced by the sync commit.
• NewParser keeps this fork's SensitiveConfig.
• logs.Init, flag style and the fd-closing paths in TailReader are kept.
• The freopen fallback sits behind --enable-dynamic-log-tailing, which still gates tailing of files a process opens.
• windows/ isn't part of this fork.
• Test: common/log_parser_test.go is updated for the new NewParser signature.

From review (e345645):

• The limiter's burst is now at least 1, so a limit below 0.1/s no longer stops extraction.
• The logparser pin moved to merge coroot/logparser v1.4.2: JSON parser, pattern rate limit, level fixes logparser#34's head, which decodes a JSON line once and accepts CRLF line endings. Pattern hashes are unchanged; logparser tests cover this.

CI: gofmt, goimports, vet, golangci-lint, go test (excluding /containers) and the build all pass in a Linux container with Go 1.26.5.

Local e2e: I built agent binaries from this branch and from main and ran them as systemd services on a local Debian 12 VM (kernel 6.1), exporting logs over OTLP to a local OpenTelemetry collector.

• JSON log line ({"level":"error","msg":"payment failed","order_id":...,"trace_id":...,"span_id":...}): this branch sent body payment failed, severity error, attribute order_id, and the trace and span IDs. Main sent the raw JSON line, with no trace context.
• Pattern hash: the same in both builds.
• Error-line storm through /var/log tailing (about 59,000 lines in 30s, same binary): with no limit, the agent used 1109 ms of CPU and counted 59,200 messages under one pattern. With a limit of 100/s, it used 970 ms and counted 59,000 messages, 55,070 of them under the sampled pattern.
• Re-run on the current logparser pin (single JSON decode, CRLF): JSON lines arrived with their order_id attributes. JSON lines ending in \r\n arrived with body crlf line and their n attribute.

[gemini-code-assist]

Code Review

This pull request introduces several enhancements to log parsing and tailing. Key changes include extracting trace and span IDs from log attributes to associate them with OpenTelemetry log records, implementing a rate limiter for log pattern extraction, and allowing the tail reader to start even if the target log file does not exist yet by polling until it is created. Additionally, it adds a flag to disable JSON log parsing and introduces a mechanism to find log files held by a process when a file descriptor is redirected. Feedback on these changes highlights two issues: first, the rate limiter burst size can evaluate to zero and reject all events if the extraction limit is set below 0.1; second, findLogFiles may return duplicate log paths if a process has multiple file descriptors open for the same file, which can be resolved by deduplicating the results.

[blue4209211]

LGTM. One note on masking first: there's no regression. The fork's SensitiveConfig only detects and counts sensitive data; it never redacted the OTLP body, which was the raw line on main. So the new attributes carry nothing that wasn't already exported, and detection still scans the full line. A few things for a later pass:

1. A JSON field can duplicate or spoof pattern.hash (logs/otel.go:116, logRecordAttrs). The agent's pattern.hash is appended first, then every JSON field, so {"pattern.hash":"x"} yields two pattern.hash keys. OTLP expects unique keys, and a backend that keeps the last one loses the agent's hash. Skip, or prefix, user keys that collide with keys the agent sets.
2. The freopen fallback runs on the event loop with no throttle (containers/container.go ~713, findLogFiles). This is behind --enable-dynamic-log-tailing, so it's off by default. When enabled, processes that open and close a /var/log file on every write (PHP error_log, shell >>) trigger a full /proc/<pid>/fd readdir plus readlinks on every write, synchronously in the registry loop. A per-pid cooldown would bound it. The path filtering is tight, so it can't tail arbitrary files, sockets or pipes.
3. Missing log files are retried forever, silently (logs/tail_reader.go). There's no leak, but a wrong log path now produces an idle poller and no warning. Log once (Info) when a reader starts in the waiting-for-file state.
4. Minor:
   • Records over the rate limit are exported with the "event was sampled" hash, so they lose their real pattern.
   • The limiter is created per NewParser, so per log source, not per container as the flag text says.
   • A 16-digit decimal dd.span_id would be read as hex.
   • A span id with no trace id is still set.
   • JSON with no message key is exported twice: the full line as the body and every field as attributes.