kflowd
v0.9
Si desea unirse a nuestro canal comunitario de Slack, envíe un correo electrónico a [email protected] para recibir una invitación. También puede contactarnos directamente a [email protected] para cualquier consulta.
kflowd contiene un programa eBPF que se ejecuta en el contexto del kernel y su aplicación de control que se ejecuta en el espacio de usuario.
El programa eBPF rastrea las funciones del kernel para monitorear procesos basados en el sistema de archivos y eventos de red. Los eventos se agregan en registros y se envían a un búfer circular donde la aplicación de control del espacio de usuario los sondea. Todos los registros se enriquecen con información del proceso y luego se convierten en un mensaje en formato de salida JSON.
Los mensajes finales se imprimen en la consola estándar y se pueden enviar mediante el protocolo UDP a hosts específicos para su ingesta en una canalización de datos de seguridad.
kflowd se ejecuta en los kernels de Linux 5.10+ y está construido con la cadena de herramientas de desarrollo eBPF libbpf+CO-RE (Compile-Once-Run-Everywhere) usando BTF (BPF Type Format) para permitir la portabilidad evitando dependencias en las diferencias en los encabezados del kernel entre las versiones del kernel. en el despliegue.
kflowd genera mensajes JSON generados para cada registro del sistema de archivos agregado y eventos de red TCP, UDP y, opcionalmente, mensajes de aplicación DNS, HTTP y SYSLOG en los formatos que se muestran en los siguientes ejemplos:
{
"InfoSequenceNumber": 1,
"InfoTimestamp": "Thu, Apr 04 2024 15:34:35.643031330 UTC",
"InfoMonitor": "filesystem",
"InfoHostName": "dev.kflow.co",
"InfoHostIP": "38.110.1.24",
"InfoSystem": "Linux",
"InfoKernel": "6.1.0-10-amd64",
"InfoVersion": "kflowd-v0.9.1",
"InfoUptime": 21.262713426,
"ProcParent": "sshd",
"Proc": "sftp-server",
"ProcVersion": "1:9.2p1-2+deb12u1",
"ProcUser": "dirk",
"ProcGroup": "dirk",
"ProcPPID": 183546,
"ProcPID": 183547,
"ProcTID": 183547,
"ProcUID": 1002,
"ProcGID": 1002,
"ProcAge": 2.408293862,
"FilePath": "/home/dirk/",
"File": "malware",
"FileVersion": "0.9",
"FileMode": "regular",
"FileEventCount": 4,
"FileEvents": {
"OPEN": 1,
"MODIFY": 2,
"CLOSE_WRITE": 1
},
"FileEventsDuration": 0.811829334,
"FileInode": 19567988,
"FileInodeLinkCount": 1,
"FileDevice": "902h:/dev/md2:/:ext4",
"FilePermissions": "0755/-rwxr-xr-x",
"FileUser": "dirk",
"FileGroup": "dirk",
"FileUID": 1002,
"FileGID": 1002,
"FileSize": 41,
"FileSizeChange": 41,
"FileAccessTime": "Thu, Apr 04 2024 15:12:01.435718956 UTC",
"FileStatusChangeTime": "Thu, Apr 04 2024 15:34:35.154106191 UTC",
"FileModificationTime": "Thu, Apr 04 2024 15:34:35.154106191 UTC",
"FileModificationTimeChange": 0.327993681,
"FileMD5": "96760f46bd29ba986279f22bed9839f5",
"FileSHA256": "72c58c2d02ae3a87f521594373433b7d05477c4994fc0ab4376827cadb29ba7e"
}
{
"InfoSequenceNumber": 2,
"InfoTimestamp": "Thu, Apr 04 2024 15:39:11.463732866 UTC",
"InfoMonitor": "socket",
"InfoHostName": "dev.kflow.co",
"InfoHostIP": "38.110.1.24",
"InfoSystem": "Linux",
"InfoKernel": "6.1.0-10-amd64",
"InfoVersion": "kflowd-v0.9.1",
"InfoUptime": 23.972984597,
"ProcParent": "bash",
"Proc": "curl",
"ProcVersion": "7.45.2-3",
"ProcUser": "dirk",
"ProcGroup": "dirk",
"ProcPPID": 199853,
"ProcPID": 199856,
"ProcTID": 199857,
"ProcUID": 1002,
"ProcGID": 1002,
"ProcAge": 0.044454620,
"SockProtocol": "UDP",
"SockRole": "CLIENT",
"SockState": "UDP_ESTABLISHED",
"SockFamily": "AF_INET",
"SockLocalIP": "38.110.1.24",
"SockLocalPort": 56664,
"SockRemoteIP": "8.8.4.4",
"SockRemotePort": 53,
"SockTxInterface": "4:enp5s0:0c:c4:7a:88:84:c2",
"SockTxPackets": 2,
"SockTxDuration": 0.000048490,
"SockTxBytes": 52,
"SockTxInterface": "4:enp5s0:0c:c4:7a:88:84:c2",
"SockRxPackets": 2,
"SockRxPacketsQueued": 0,
"SockRxPacketsDrop": 0,
"SockRxPacketsFrag": 0,
"SockRxDuration": 22.475134750,
"SockRxBytes": 155,
"SockRxTTL": 125,
"SockAge": 0.043689149,
"App": "DNS",
"AppTxDns": [{
"_Timestamp": 0.000001177,
"TransactionId": 56864,
"OpCode": "QUERY",
"Flags": ["RD"],
"ResourceRecords": [
["A", "kflow.co"]
]
},
{
"_Timestamp": 0.000048627,
"TransactionId": 52515,
"OpCode": "QUERY",
"Flags": ["RD"],
"ResourceRecords": [
["AAAA", "kflow.co"]
]
}],
"AppRxDns": [{
"_Timestamp": 0.037965555,
"TransactionId": 52515,
"ResponseCode": "NOERROR",
"Flags": ["QR", "RD", "RA"],
"AnswerCount": 0,
"ResourceRecords": []
},
{
"_Timestamp": 0.043688644,
"TransactionId": 56864,
"ResponseCode": "NOERROR",
"Flags": ["QR", "RD", "RA"],
"AnswerCount": 2,
"ResourceRecords": [
["A", "kflow.co", 600, "IN", "15.197.142.173"],
["A", "kflow.co", 600, "IN", "3.33.152.147"]
]
}]
}
{
"InfoSequenceNumber": 3,
"InfoTimestamp": "Thu, Apr 04 2024 15:39:11.928989997 UTC",
"InfoMonitor": "socket",
"InfoHostName": "dev.kflow.co",
"InfoHostIP": "38.110.1.24",
"InfoSystem": "Linux",
"InfoKernel": "6.1.0-10-amd64",
"InfoVersion": "kflowd-v0.9.1",
"InfoUptime": 24.873001288,
"ProcParent": "bash",
"Proc": "curl",
"ProcVersion": "7.45.2-3",
"ProcUser": "dirk",
"ProcGroup": "dirk",
"ProcPPID": 199853,
"ProcPID": 216998,
"ProcTID": 216998,
"ProcUID": 1002,
"ProcGID": 1002,
"ProcAge": 0.114196829,
"SockProtocol": "TCP",
"SockRole": "CLIENT",
"SockState": "TCP_CLOSE",
"SockFamily": "AF_INET",
"SockLocalIP": "38.110.1.24",
"SockLocalPort": 43302,
"SockRemoteIP": "15.197.142.173",
"SockRemotePort": 80,
"SockTxInterface": "4:enp5s0:0c:c4:7a:88:84:c2",
"SockTxDataPackets": 1,
"SockTxPackets": 6,
"SockTxPacketsRetrans": 0,
"SockTxPacketsDups": 0,
"SockTxFlags": {
"SYN": 1,
"ACK": 3,
"PSH-ACK": 1,
"FIN-ACK": 1
},
"SockTxDuration": 0.057274799,
"SockTxBytes": 72,
"SockTxBytesAcked": 74,
"SockTxBytesRetrans": 0,
"SockTxRTO": 51,
"SockTxInterface": "4:enp5s0:0c:c4:7a:88:84:c2",
"SockRxDataPackets": 1,
"SockRxPackets": 4,
"SockRxPacketsQueued": 0,
"SockRxPacketsDrop": 0,
"SockRxPacketsReorder": 0,
"SockRxPacketsFrag": 0,
"SockRxFlags": {
"SYN-ACK": 1,
"ACK": 1,
"PSH-ACK": 1,
"FIN-ACK": 1
},
"SockRxDuration": 0.057197399,
"SockRxBytes": 342,
"SockRxTTL": 185,
"SockRTT": 0.000450625,
"SockAge": 0.057344028,
"App": "HTTP",
"AppTxHttp": [{
"_Timestamp": 0.000876589,
"_Method": "GET",
"_Url": "/",
"_Version": "HTTP/1.1",
"Host": "kflow.co",
"User-Agent": "curl/7.88.1",
"Accept": "*/*"
}],
"AppRxHttp": [{
"_Timestamp": 0.056237469,
"_Version": "HTTP/1.1",
"_Status": 301,
"_Reason": "Moved Permanently",
"Date": "Thu, 04 Apr 2024 15:49:12 GMT",
"Content-Type": "text/html; charset=utf-8",
"Content-Length": "59",
"Connection": "keep-alive",
"Location": "https://kflowd.github.io",
"Server": "ip-10-123-123-119.ec2.internal",
"X-Request-Id": "5c331cbe-fbe1-40ea-ba2b-989691e687a0",
"_Body": "<a href="https://kflowd.github.io">Moved Permanently</a>."
}]
}
{
"InfoSequenceNumber": 4,
"InfoTimestamp": "Mon, Sep 16 2024 14:30:19.409100980 UTC",
"InfoMonitor": "socket",
"InfoHostName": "dev.kflow.co",
"InfoHostIP": "38.110.1.24",
"InfoSystem": "Linux",
"InfoKernel": "6.1.0-10-amd64",
"InfoVersion": "kflowd-v0.9.11",
"ProcParent": "cron",
"Proc": "cron",
"ProcVersion": "3.0pl1-137",
"ProcUser": "root",
"ProcGroup": "root",
"ProcPPID": 990,
"ProcPID": 2122368,
"ProcTID": 2122368,
"ProcUID": 0,
"ProcGID": 0,
"ProcAge": 18.100170007,
"SockRole": "CLIENT",
"SockAddress": "/run/systemd/journal/dev-log",
"SockFamily": "AF_UNIX",
"SockTxBytes": 192,
"SockAge": 18.095146325,
"App": "SYSLOG",
"AppTxSyslog": [{
"Facility": "Security/Authorization",
"Severity": "Notice (5)",
"Priority": 85,
"Version": 0,
"Timestamp": "Sep 16 14:30:01",
"Appname": "CRON",
"ProcId": "2122368",
"Message": "pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)"
},
{
"Facility": "Security/Authorization",
"Severity": "Informational (6)",
"Priority": 86,
"Version": 0,
"Timestamp": "Sep 16 14:30:01",
"Appname": "CRON",
"ProcId": "2122368",
"Message": "pam_unix(cron:session): session closed for user root"
}]
}
Kernel 5.10+ compilado con:
Kernel 5.10+ compilado con el compilador Just-In-Time eBPF (JIT):
Configuración de control del sistema JIT habilitada:
El siguiente enlace proporciona una descripción general de las distribuciones de Linux con eBPF CO-RE y BTF habilitados de forma predeterminada:
Distribuciones de Linux con eBPF CO-RE y BTF
Para una salida UDP de alto rendimiento, se recomiendan las siguientes configuraciones de red del kernel:
Usage:
kflowd [-m file,socket] [-t IDLE,ACTIVE] [-e EVENTS] [-o json|json-min|table] [-v] [-c]
[-p dns|http|syslog=PROTO/PORT,...] [-u IP:PORT] [-q] [-d] [-V] [-T TOKEN] [-P PATH]
[-D PROCESS], [-l] [--legend], [-h] [--help], [--version]
-m file,socket Monitor only specified kernel subsystem (filesystem or sockets)
(default: all, option omitted!)
-t IDLE,ACTIVE Timeout in seconds for idle or active network sockets until export
(default: idle '15' seconds, active '1800' seconds)
-e EVENTS Max number of filesystem events per aggregated record until export
(default: disabled, '1': no aggregation)
-o json Json output with formatting (default)
json-min Json output with minimal formatting
table Tabular output with limited keys and no UDP output
-v Version of executable files identified by installed package
(supported only for rpm- and deb-based package management)
-c Checksum hashes of MD5 and SHA256 calculated for executables
-p dns=PROTO/PORT,... Port(s) examined for decoding of DNS application protocol
(default: 'dns=udp/53,tcp/53', disabled: 'dns=off')
-p http=PROTO/PORT,... Port(s) examined for decoding of HTTP application protocol
(default: 'http=tcp/80', disabled: 'http=off')
-p syslog=PROTO/PORT,... Port(s) examined for decoding of SYSLOG application protocol
(default: 'syslog=udp/514,tcp/514,unix', disabled: 'syslog=off')
-u IP:PORT,... UDP server(s) IPv4 or IPv6 address to send json output to.
Output also printed to stdout console unless quiet option -q or
daemon mode -d specified
-q Quiet mode to suppress output to stdout console
-d Daemonize program to run in background
-V Verbose output
Print eBPF load and co-re messages on start of eBPF program
to stderr console
-T TOKEN Token specified on host to be included in json output
-P PATH Path to search for kflowd plugin modules (default: '../lib/')
-l, --legend Show legend
-h, --help Show help
--version Show version
-D PROCESS Debug
Print ebpf kernel log messages of process or expiration queue to
kernel trace pipe (any process: '*', with quotes!, queue: 'q')
Use command:
'sudo cat /sys/kernel/debug/tracing/trace_pipe'
Examples:
sudo ./kflowd # terminal mode
sudo ./kflowd -m file,socket -v -c -u 1.2.3.4:2056,127.0.0.1:2057 -d # daemon mode
sudo ./kflowd -m socket -v -c -u 1.2.3.4:2056 -V -D '*' # debug mode
sudo ./kflowd --legend # show legend
sudo ./kflowd --version # show version
uname -a
cat /boot/config-* | grep CONFIG_DEBUG_INFO_BTF
...
kernel.bpf_stats_enabled=1
kernel.unprivileged_bpf_disabled=0
Basado en Debian (Ubuntu, Mint)
sudo apt install libz-dev
sudo apt install libelf-dev
sudo apt install libcap-dev
sudo apt install libbfd-dev
sudo apt install libc6-dev-i386
sudo apt install build-essential
sudo apt install pkg-config
sudo apt install clang-16*
sudo apt install llvm-16*
sudo ln -s /usr/bin/clang-16 /usr/bin/clang
sudo ln -s /usr/bin/llvm-strip-16 /usr/bin/llvm-strip
echo 'deb [trusted=yes] https://repo.goreleaser.com/apt/
/' | sudo tee /etc/apt/sources.list.d/goreleaser.list
sudo apt update
sudo apt install nfpm
Basado en Redhat (Amazon Linux, Fedora)
sudo yum install zlib-devel
sudo yum install elfutils-libelf-devel
sudo yum install libcap-devel
sudo yum install binutils-devel
sudo yum install glibc-devel.i386 or .i686
sudo yum groupinstall 'Development Tools'
sudo yum install pkgconfig
sudo yum install clang*
sudo yum install llvm*
echo '[goreleaser]
name=GoReleaser
baseurl=https://repo.goreleaser.com/yum/
enabled=1
gpgcheck=0' | sudo tee /etc/yum.repos.d/goreleaser.repo
sudo yum install nfpm
git clone https://github.com/tarsal-oss/kflowd.git
cd kflowd
git submodule update --init --recursive
cd src
make
make rpm deb
Los paquetes se pueden instalar en plataformas basadas en Linux x86_64 y arm64:
sudo apt install ./kflowd_x.x.x_amd64.deb
sudo apt install ./kflowd_x.x.x_arm64.deb
sudo yum install ./kflowd-x.x.x.x86_64.rpm
sudo yum install ./kflowd-x.x.x.aarch64.rpm
Tenga en cuenta que los artefactos de compilación se pueden descargar en Acciones de GitHub en la sección Artefactos del flujo de trabajo kflowd-ci ejecutado con binarios y paquetes compatibles para las plataformas x86_64 y arm64 (glibc 2.31+):
Paquetes binarios, RPM y DEB prediseñados (comprimidos)
Este trabajo tiene la licencia GNU General Public License v2.0.
SPDX-License-Identifier: GPL-2.0
