kflowd
v0.9
Se você gostaria de ingressar no canal Slack da nossa comunidade, envie um e-mail para [email protected] para receber um convite. Você também pode nos contatar diretamente em [email protected] para qualquer dúvida.
kflowd contém um programa eBPF rodando no contexto do kernel e seu aplicativo de controle rodando no espaço do usuário.
O programa eBPF rastreia funções do kernel para monitorar processos baseados no sistema de arquivos e eventos de rede. Os eventos são agregados em registros e enviados para um ringbuffer onde são pesquisados pelo aplicativo de controle do espaço do usuário. Todos os registros são enriquecidos com informações do processo e depois convertidos em uma mensagem no formato de saída JSON.
As mensagens finais são impressas no console stdout e podem ser enviadas via protocolo UDP para hosts especificados para ingestão em um pipeline de dados de segurança.
kflowd é executado em kernels Linux 5.10+ e é construído com a cadeia de ferramentas de desenvolvimento eBPF libbpf+CO-RE (Compile-Once-Run-Everywhere) usando BTF (BPF Type Format) para permitir portabilidade, evitando dependências em diferenças nos cabeçalhos do kernel entre versões do kernel na implantação.
kflowd gera mensagens JSON geradas para cada registro do sistema de arquivos agregado e eventos de rede TCP, UDP e, opcionalmente, mensagens de aplicativos DNS, HTTP e SYSLOG nos formatos mostrados nos exemplos a seguir:
{
"InfoSequenceNumber": 1,
"InfoTimestamp": "Thu, Apr 04 2024 15:34:35.643031330 UTC",
"InfoMonitor": "filesystem",
"InfoHostName": "dev.kflow.co",
"InfoHostIP": "38.110.1.24",
"InfoSystem": "Linux",
"InfoKernel": "6.1.0-10-amd64",
"InfoVersion": "kflowd-v0.9.1",
"InfoUptime": 21.262713426,
"ProcParent": "sshd",
"Proc": "sftp-server",
"ProcVersion": "1:9.2p1-2+deb12u1",
"ProcUser": "dirk",
"ProcGroup": "dirk",
"ProcPPID": 183546,
"ProcPID": 183547,
"ProcTID": 183547,
"ProcUID": 1002,
"ProcGID": 1002,
"ProcAge": 2.408293862,
"FilePath": "/home/dirk/",
"File": "malware",
"FileVersion": "0.9",
"FileMode": "regular",
"FileEventCount": 4,
"FileEvents": {
"OPEN": 1,
"MODIFY": 2,
"CLOSE_WRITE": 1
},
"FileEventsDuration": 0.811829334,
"FileInode": 19567988,
"FileInodeLinkCount": 1,
"FileDevice": "902h:/dev/md2:/:ext4",
"FilePermissions": "0755/-rwxr-xr-x",
"FileUser": "dirk",
"FileGroup": "dirk",
"FileUID": 1002,
"FileGID": 1002,
"FileSize": 41,
"FileSizeChange": 41,
"FileAccessTime": "Thu, Apr 04 2024 15:12:01.435718956 UTC",
"FileStatusChangeTime": "Thu, Apr 04 2024 15:34:35.154106191 UTC",
"FileModificationTime": "Thu, Apr 04 2024 15:34:35.154106191 UTC",
"FileModificationTimeChange": 0.327993681,
"FileMD5": "96760f46bd29ba986279f22bed9839f5",
"FileSHA256": "72c58c2d02ae3a87f521594373433b7d05477c4994fc0ab4376827cadb29ba7e"
}
{
"InfoSequenceNumber": 2,
"InfoTimestamp": "Thu, Apr 04 2024 15:39:11.463732866 UTC",
"InfoMonitor": "socket",
"InfoHostName": "dev.kflow.co",
"InfoHostIP": "38.110.1.24",
"InfoSystem": "Linux",
"InfoKernel": "6.1.0-10-amd64",
"InfoVersion": "kflowd-v0.9.1",
"InfoUptime": 23.972984597,
"ProcParent": "bash",
"Proc": "curl",
"ProcVersion": "7.45.2-3",
"ProcUser": "dirk",
"ProcGroup": "dirk",
"ProcPPID": 199853,
"ProcPID": 199856,
"ProcTID": 199857,
"ProcUID": 1002,
"ProcGID": 1002,
"ProcAge": 0.044454620,
"SockProtocol": "UDP",
"SockRole": "CLIENT",
"SockState": "UDP_ESTABLISHED",
"SockFamily": "AF_INET",
"SockLocalIP": "38.110.1.24",
"SockLocalPort": 56664,
"SockRemoteIP": "8.8.4.4",
"SockRemotePort": 53,
"SockTxInterface": "4:enp5s0:0c:c4:7a:88:84:c2",
"SockTxPackets": 2,
"SockTxDuration": 0.000048490,
"SockTxBytes": 52,
"SockTxInterface": "4:enp5s0:0c:c4:7a:88:84:c2",
"SockRxPackets": 2,
"SockRxPacketsQueued": 0,
"SockRxPacketsDrop": 0,
"SockRxPacketsFrag": 0,
"SockRxDuration": 22.475134750,
"SockRxBytes": 155,
"SockRxTTL": 125,
"SockAge": 0.043689149,
"App": "DNS",
"AppTxDns": [{
"_Timestamp": 0.000001177,
"TransactionId": 56864,
"OpCode": "QUERY",
"Flags": ["RD"],
"ResourceRecords": [
["A", "kflow.co"]
]
},
{
"_Timestamp": 0.000048627,
"TransactionId": 52515,
"OpCode": "QUERY",
"Flags": ["RD"],
"ResourceRecords": [
["AAAA", "kflow.co"]
]
}],
"AppRxDns": [{
"_Timestamp": 0.037965555,
"TransactionId": 52515,
"ResponseCode": "NOERROR",
"Flags": ["QR", "RD", "RA"],
"AnswerCount": 0,
"ResourceRecords": []
},
{
"_Timestamp": 0.043688644,
"TransactionId": 56864,
"ResponseCode": "NOERROR",
"Flags": ["QR", "RD", "RA"],
"AnswerCount": 2,
"ResourceRecords": [
["A", "kflow.co", 600, "IN", "15.197.142.173"],
["A", "kflow.co", 600, "IN", "3.33.152.147"]
]
}]
}
{
"InfoSequenceNumber": 3,
"InfoTimestamp": "Thu, Apr 04 2024 15:39:11.928989997 UTC",
"InfoMonitor": "socket",
"InfoHostName": "dev.kflow.co",
"InfoHostIP": "38.110.1.24",
"InfoSystem": "Linux",
"InfoKernel": "6.1.0-10-amd64",
"InfoVersion": "kflowd-v0.9.1",
"InfoUptime": 24.873001288,
"ProcParent": "bash",
"Proc": "curl",
"ProcVersion": "7.45.2-3",
"ProcUser": "dirk",
"ProcGroup": "dirk",
"ProcPPID": 199853,
"ProcPID": 216998,
"ProcTID": 216998,
"ProcUID": 1002,
"ProcGID": 1002,
"ProcAge": 0.114196829,
"SockProtocol": "TCP",
"SockRole": "CLIENT",
"SockState": "TCP_CLOSE",
"SockFamily": "AF_INET",
"SockLocalIP": "38.110.1.24",
"SockLocalPort": 43302,
"SockRemoteIP": "15.197.142.173",
"SockRemotePort": 80,
"SockTxInterface": "4:enp5s0:0c:c4:7a:88:84:c2",
"SockTxDataPackets": 1,
"SockTxPackets": 6,
"SockTxPacketsRetrans": 0,
"SockTxPacketsDups": 0,
"SockTxFlags": {
"SYN": 1,
"ACK": 3,
"PSH-ACK": 1,
"FIN-ACK": 1
},
"SockTxDuration": 0.057274799,
"SockTxBytes": 72,
"SockTxBytesAcked": 74,
"SockTxBytesRetrans": 0,
"SockTxRTO": 51,
"SockTxInterface": "4:enp5s0:0c:c4:7a:88:84:c2",
"SockRxDataPackets": 1,
"SockRxPackets": 4,
"SockRxPacketsQueued": 0,
"SockRxPacketsDrop": 0,
"SockRxPacketsReorder": 0,
"SockRxPacketsFrag": 0,
"SockRxFlags": {
"SYN-ACK": 1,
"ACK": 1,
"PSH-ACK": 1,
"FIN-ACK": 1
},
"SockRxDuration": 0.057197399,
"SockRxBytes": 342,
"SockRxTTL": 185,
"SockRTT": 0.000450625,
"SockAge": 0.057344028,
"App": "HTTP",
"AppTxHttp": [{
"_Timestamp": 0.000876589,
"_Method": "GET",
"_Url": "/",
"_Version": "HTTP/1.1",
"Host": "kflow.co",
"User-Agent": "curl/7.88.1",
"Accept": "*/*"
}],
"AppRxHttp": [{
"_Timestamp": 0.056237469,
"_Version": "HTTP/1.1",
"_Status": 301,
"_Reason": "Moved Permanently",
"Date": "Thu, 04 Apr 2024 15:49:12 GMT",
"Content-Type": "text/html; charset=utf-8",
"Content-Length": "59",
"Connection": "keep-alive",
"Location": "https://kflowd.github.io",
"Server": "ip-10-123-123-119.ec2.internal",
"X-Request-Id": "5c331cbe-fbe1-40ea-ba2b-989691e687a0",
"_Body": "<a href="https://kflowd.github.io">Moved Permanently</a>."
}]
}
{
"InfoSequenceNumber": 4,
"InfoTimestamp": "Mon, Sep 16 2024 14:30:19.409100980 UTC",
"InfoMonitor": "socket",
"InfoHostName": "dev.kflow.co",
"InfoHostIP": "38.110.1.24",
"InfoSystem": "Linux",
"InfoKernel": "6.1.0-10-amd64",
"InfoVersion": "kflowd-v0.9.11",
"ProcParent": "cron",
"Proc": "cron",
"ProcVersion": "3.0pl1-137",
"ProcUser": "root",
"ProcGroup": "root",
"ProcPPID": 990,
"ProcPID": 2122368,
"ProcTID": 2122368,
"ProcUID": 0,
"ProcGID": 0,
"ProcAge": 18.100170007,
"SockRole": "CLIENT",
"SockAddress": "/run/systemd/journal/dev-log",
"SockFamily": "AF_UNIX",
"SockTxBytes": 192,
"SockAge": 18.095146325,
"App": "SYSLOG",
"AppTxSyslog": [{
"Facility": "Security/Authorization",
"Severity": "Notice (5)",
"Priority": 85,
"Version": 0,
"Timestamp": "Sep 16 14:30:01",
"Appname": "CRON",
"ProcId": "2122368",
"Message": "pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)"
},
{
"Facility": "Security/Authorization",
"Severity": "Informational (6)",
"Priority": 86,
"Version": 0,
"Timestamp": "Sep 16 14:30:01",
"Appname": "CRON",
"ProcId": "2122368",
"Message": "pam_unix(cron:session): session closed for user root"
}]
}
Kernel 5.10+ compilado com:
Kernel 5.10+ compilado com compilador Just-In-Time eBPF (JIT):
Configurações de controle do sistema JIT habilitadas:
O link a seguir fornece uma visão geral das distribuições Linux com eBPF CO-RE e BTF habilitados por padrão:
Distribuições Linux com eBPF CO-RE e BTF
Para saída UDP de alto desempenho, são recomendadas as seguintes configurações de rede do kernel:
Usage:
kflowd [-m file,socket] [-t IDLE,ACTIVE] [-e EVENTS] [-o json|json-min|table] [-v] [-c]
[-p dns|http|syslog=PROTO/PORT,...] [-u IP:PORT] [-q] [-d] [-V] [-T TOKEN] [-P PATH]
[-D PROCESS], [-l] [--legend], [-h] [--help], [--version]
-m file,socket Monitor only specified kernel subsystem (filesystem or sockets)
(default: all, option omitted!)
-t IDLE,ACTIVE Timeout in seconds for idle or active network sockets until export
(default: idle '15' seconds, active '1800' seconds)
-e EVENTS Max number of filesystem events per aggregated record until export
(default: disabled, '1': no aggregation)
-o json Json output with formatting (default)
json-min Json output with minimal formatting
table Tabular output with limited keys and no UDP output
-v Version of executable files identified by installed package
(supported only for rpm- and deb-based package management)
-c Checksum hashes of MD5 and SHA256 calculated for executables
-p dns=PROTO/PORT,... Port(s) examined for decoding of DNS application protocol
(default: 'dns=udp/53,tcp/53', disabled: 'dns=off')
-p http=PROTO/PORT,... Port(s) examined for decoding of HTTP application protocol
(default: 'http=tcp/80', disabled: 'http=off')
-p syslog=PROTO/PORT,... Port(s) examined for decoding of SYSLOG application protocol
(default: 'syslog=udp/514,tcp/514,unix', disabled: 'syslog=off')
-u IP:PORT,... UDP server(s) IPv4 or IPv6 address to send json output to.
Output also printed to stdout console unless quiet option -q or
daemon mode -d specified
-q Quiet mode to suppress output to stdout console
-d Daemonize program to run in background
-V Verbose output
Print eBPF load and co-re messages on start of eBPF program
to stderr console
-T TOKEN Token specified on host to be included in json output
-P PATH Path to search for kflowd plugin modules (default: '../lib/')
-l, --legend Show legend
-h, --help Show help
--version Show version
-D PROCESS Debug
Print ebpf kernel log messages of process or expiration queue to
kernel trace pipe (any process: '*', with quotes!, queue: 'q')
Use command:
'sudo cat /sys/kernel/debug/tracing/trace_pipe'
Examples:
sudo ./kflowd # terminal mode
sudo ./kflowd -m file,socket -v -c -u 1.2.3.4:2056,127.0.0.1:2057 -d # daemon mode
sudo ./kflowd -m socket -v -c -u 1.2.3.4:2056 -V -D '*' # debug mode
sudo ./kflowd --legend # show legend
sudo ./kflowd --version # show version
uname -a
cat /boot/config-* | grep CONFIG_DEBUG_INFO_BTF
...
kernel.bpf_stats_enabled=1
kernel.unprivileged_bpf_disabled=0
Baseado em Debian (Ubuntu, Mint)
sudo apt install libz-dev
sudo apt install libelf-dev
sudo apt install libcap-dev
sudo apt install libbfd-dev
sudo apt install libc6-dev-i386
sudo apt install build-essential
sudo apt install pkg-config
sudo apt install clang-16*
sudo apt install llvm-16*
sudo ln -s /usr/bin/clang-16 /usr/bin/clang
sudo ln -s /usr/bin/llvm-strip-16 /usr/bin/llvm-strip
echo 'deb [trusted=yes] https://repo.goreleaser.com/apt/
/' | sudo tee /etc/apt/sources.list.d/goreleaser.list
sudo apt update
sudo apt install nfpm
Baseado em Redhat (Amazon Linux, Fedora)
sudo yum install zlib-devel
sudo yum install elfutils-libelf-devel
sudo yum install libcap-devel
sudo yum install binutils-devel
sudo yum install glibc-devel.i386 or .i686
sudo yum groupinstall 'Development Tools'
sudo yum install pkgconfig
sudo yum install clang*
sudo yum install llvm*
echo '[goreleaser]
name=GoReleaser
baseurl=https://repo.goreleaser.com/yum/
enabled=1
gpgcheck=0' | sudo tee /etc/yum.repos.d/goreleaser.repo
sudo yum install nfpm
git clone https://github.com/tarsal-oss/kflowd.git
cd kflowd
git submodule update --init --recursive
cd src
make
make rpm deb
Os pacotes podem ser instalados em plataformas baseadas em Linux x86_64 e arm64:
sudo apt install ./kflowd_x.x.x_amd64.deb
sudo apt install ./kflowd_x.x.x_arm64.deb
sudo yum install ./kflowd-x.x.x.x86_64.rpm
sudo yum install ./kflowd-x.x.x.aarch64.rpm
Observe que os artefatos de construção podem ser baixados em GitHub Actions na seção Artifacts do fluxo de trabalho kflowd-ci executado com binários e pacotes compatíveis para plataformas x86_64 e arm64 (glibc 2.31+):
Binários pré-construídos, pacotes RPM e DEB (zipados)
Este trabalho está licenciado sob GNU General Public License v2.0.
SPDX-License-Identifier: GPL-2.0
