Logintel is an open-source SIEM and endpoint monitoring platform that collects Linux kernel events in real time using eBPF, fanotify, and D-Bus, stores them efficiently in Elasticsearch.
Logintel Central Server is a SIEM System that is responsible for collecting logs from Logintel Agent, storing the logs efficiently in Elasticsearch with 3 hours of TTL, connecting with Kibana.
Logintel Agent collects events and that's not enough. The logs should be processed and stored efficiently. If anything goes wrong, instead of roaming file to file to get logs, required logs should be at one place.
- Ingest logs from different agents
- Store logs in Elasticsearch for 3 hours
- Log visualization in Kibana
- Stream logs to NATS Jetstream
Log ingestion service is responsible for collecting logs from different services via gRPC. After collecting logs, it saves the logs in Elasticsearch in different data streams and streams the logs in NATS Jetstream Pub/Sub for fan-out distribution.
Kibana dashboard is connected with Elasticseach and used for log visualization, filtration and exporting as CSV.
flowchart LR
A[Logintel Agent] -->|gRPC| B[Log Ingestion Service]
B --> C[Elasticsearch]
B --> D[NATS JetStream]
C --> H[Kibana]
The entire project, agent and server, are built with Golang. Golang is very lightweight and well-connected to Linux kernel system and has well-maintained packages of eBPF that makes it the best choice for the agent. Also, Golang has very good performance in backend that most of the big companies are now using it in their backend and building cloud native tools with it. After considering all these points, it was crystal clear that Golang is the best choice for the system.
eBPF is a technology that lets developers run custom code within the kernel without writing any kernel module. It also provide enough security to handle bugs in the kernel, so the kernel will not crash. In this project, eBPF is used to capture kernel events that creates a transparency which is the best way to understand what actually happened and how it was happened.
Fanotify is a kernel notification subsystem that notifies user-space applications when any file events occur in the kernel. It keeps an eye on the entire filesystem. It is one of the best choice to track file operarions in a system.
It produces massive amount of logs. A good amount of logs have no connection with security events.
A file path-based blacklist filtration is implemented in the agent. A log with blacklisted path will never reach gRPC. The list is configurable with a YAML file.
Dbus Inter-Process Communication is a system in which user-applications get notified about status of systemd services. Well, this is not connected with any security purpose but we integrated it to get notified whenever anything goes wrong with any systemd service including Logintel Agent.
Elasticsearch is used to store logs and suspicious logs. With low-latency filtration, in-built rollover system, auto-deletion with ILM make the database perfect fit for the system.
Kibana provides stored log visualization with table, graphs and filtration without any extra-coding. One of the best choice for log visualization with in-built integration Elasticsearch REST APIs.
What else should we choose for RDBMS. RBAC structure, command history, agent management all are handled by this database. Chose for scalability and robustness.
NATS JetStream is the persistence and streaming layer built into NATS, a high-performance messaging system used in distributed applications and microservices. While Core NATS is an in-memory pub/sub system that delivers messages only to active subscribers, JetStream adds durable storage, message replay, acknowledgments, and delivery guarantees.
Integration of Apache Kafka is heavy, latency of RabbitMQ pub/sub is painful. NATS Jetstream provides speed, persistent storage, guarantee and that are the reason it is here.
RabbitMQ handles the background jobs here. For now, it is used to send email on every alert.
React JS is for admin panel.
Rule engine and C2 system.
- ✅ Log ingestion from different agents
- ✅ Storing logs in Elasticsearch
- ✅ Visualizing logs in Kibana
The system is still under development and getting better day by day. The first release of the system will be within October, 2026.
git clone https://github.com/incodi404/logintel-server.git
cd logintel-server
docker compose upDipankar Chowdhury — Backend Engineer (Security Focused) | GitHub · LinkedIn