kafka proxy
v0.3.12
يعتمد وكيل Kafka على فكرة وكيل Cloud SQL. يسمح للخدمة بالاتصال بسماسرة كافكا دون الحاجة إلى التعامل مع شهادات SASL/Plain وشهادات SSL.
إنه يعمل عن طريق فتح مآخذ TCP على الماكينة المحلية واتصالات وكيل الوسطاء الكافكا المرتبطين عند استخدام المقابس. يتم استبدال المضيف والميناء في البيانات الوصفية و FindCoordinator التي تم استلامها من الوسطاء بنظيرات محلية. بالنسبة للوسطاء المكتشفين (لم يتم تكوينه كخوادم Boostrap) ، يتم بدء المستمعين المحليين على منافذ عشوائية. يمكن تعطيل ميزة المستمعين المحليين الديناميكيين ويمكن توفير قائمة إضافية من تعيينات الخادم الخارجية.
يمكن للوكالة إنهاء حركة مرور TLS ومصادقة المستخدمين باستخدام SASL/Plain. طريقة التحقق من بيانات الاعتماد قابلة للتكوين وتستخدم نظام Golang Plugin عبر RPC.
يمكن للوكلاء أيضًا مصادقة بعضهم البعض باستخدام طريقة قابلة للتوصيل والتي تكون شفافة لخوادم Kafka والعملاء الأخرى. حاليًا ، يتم تنفيذ رمز Google ID لحسابات الخدمة IE طلبات عميل الوكيل ويرسل حساب الخدمة JWT و Proxy Server ويتحقق من صحته ضد Google JWKS.
يمكن تقييد مكالمات API Kafka لمنع بعض العمليات على سبيل المثال حذف الموضوع أو تقديم طلبات.
يرى:
بروكسي كافكا مع Amazon MSK
دليل لبروتوكول كافكا
دليل بروتوكول كافكا
يوفر الجدول التالي نظرة عامة على إصدارات Kafka المدعومة (محددة وجميع إصدارات Kafka السابقة). نظرًا لأن كل إصدار Kafka يضيف رسائل/إصدارات جديدة ذات صلة بوكيل Kafka ، يمكن أن تعمل إصدارات Kafka الأحدث أيضًا.
| نسخة بروكسي كافكا | نسخة كافكا |
|---|---|
| من 0.11.0 | |
| 0.2.9 | إلى 2.8.0 |
| 0.3.1 | إلى 3.4.0 |
| 0.3.11 | إلى 3.7.0 |
| 0.3.12 | إلى 3.9.0 |
قم بتنزيل أحدث إصدار
Linux
curl -Ls https://github.com/grepplabs/kafka-proxy/releases/download/v0.3.12/kafka-proxy-v0.3.12-linux-amd64.tar.gz | tar xz
ماكوس
curl -Ls https://github.com/grepplabs/kafka-proxy/releases/download/v0.3.12/kafka-proxy-v0.3.12-darwin-amd64.tar.gz | tar xz
انقل الثنائي إلى طريقك.
sudo mv ./kafka-proxy /usr/local/bin/kafka-proxy
make clean build
صور Docker متوفرة على Docker Hub.
يمكنك إطلاق حاوية kafka-proxy لمحاكمةها
docker run --rm -p 30001-30003:30001-30003 grepplabs/kafka-proxy:0.3.12 server --bootstrap-server-mapping "localhost:19092,0.0.0.0:30001" --bootstrap-server-mapping "localhost:29092,0.0.0.0:30002" --bootstrap-server-mapping "localhost:39092,0.0.0.0:30003" --dial-address-mapping "localhost:19092,172.17.0.1:19092" --dial-address-mapping "localhost:29092,172.17.0.1:29092" --dial-address-mapping "localhost:39092,172.17.0.1:39092" --debug-enable
يمكن الآن الوصول إلى Kafka-Proxy على المضيف المحلي localhost:19092 localhost:30001 ، localhost:30002 localhost:39092 localhost:29092 localhost:30003 ، والتواصل مع وسطاء الكافكا الذين يعملون في Docker (Network Bridge Gateway 172.17.0.1 ). .
يتم وضع علامة على صور Docker مع ملحقات مكونة مسبق موجودة في /opt/kafka-proxy/bin/ مع <release>-all .
يمكنك إطلاق حاوية kafka-proxy مع المكون الإضافي لـ Auth-ldap لتجربته
docker run --rm -p 30001-30003:30001-30003 grepplabs/kafka-proxy:0.3.12-all server --bootstrap-server-mapping "localhost:19092,0.0.0.0:30001" --bootstrap-server-mapping "localhost:29092,0.0.0.0:30002" --bootstrap-server-mapping "localhost:39092,0.0.0.0:30003" --dial-address-mapping "localhost:19092,172.17.0.1:19092" --dial-address-mapping "localhost:29092,172.17.0.1:29092" --dial-address-mapping "localhost:39092,172.17.0.1:39092" --debug-enable --auth-local-enable --auth-local-command=/opt/kafka-proxy/bin/auth-ldap --auth-local-param=--url=ldap://172.17.0.1:389 --auth-local-param=--start-tls=false --auth-local-param=--bind-dn=cn=admin,dc=example,dc=org --auth-local-param=--bind-passwd=admin --auth-local-param=--user-search-base=ou=people,dc=example,dc=org --auth-local-param=--user-filter="(&(objectClass=person)(uid=%u)(memberOf=cn=kafka-users,ou=realm-roles,dc=example,dc=org))"
Run the kafka-proxy server
Usage:
kafka-proxy server [flags]
Flags:
--auth-gateway-client-command string Path to authentication plugin binary
--auth-gateway-client-enable Enable gateway client authentication
--auth-gateway-client-log-level string Log level of the auth plugin (default "trace")
--auth-gateway-client-magic uint Magic bytes sent in the handshake
--auth-gateway-client-method string Authentication method
--auth-gateway-client-param stringArray Authentication plugin parameter
--auth-gateway-client-timeout duration Authentication timeout (default 10s)
--auth-gateway-server-command string Path to authentication plugin binary
--auth-gateway-server-enable Enable proxy server authentication
--auth-gateway-server-log-level string Log level of the auth plugin (default "trace")
--auth-gateway-server-magic uint Magic bytes sent in the handshake
--auth-gateway-server-method string Authentication method
--auth-gateway-server-param stringArray Authentication plugin parameter
--auth-gateway-server-timeout duration Authentication timeout (default 10s)
--auth-local-command string Path to authentication plugin binary
--auth-local-enable Enable local SASL/PLAIN authentication performed by listener - SASL handshake will not be passed to kafka brokers
--auth-local-log-level string Log level of the auth plugin (default "trace")
--auth-local-mechanism string SASL mechanism used for local authentication: PLAIN or OAUTHBEARER (default "PLAIN")
--auth-local-param stringArray Authentication plugin parameter
--auth-local-timeout duration Authentication timeout (default 10s)
--bootstrap-server-mapping stringArray Mapping of Kafka bootstrap server address to local address (host:port,host:port(,advhost:advport))
--debug-enable Enable Debug endpoint
--debug-listen-address string Debug listen address (default "0.0.0.0:6060")
--default-listener-ip string Default listener IP (default "0.0.0.0")
--dial-address-mapping stringArray Mapping of target broker address to new one (host:port,host:port). The mapping is performed during connection establishment
--dynamic-advertised-listener string Advertised address for dynamic listeners. If empty, default-listener-ip is used
--dynamic-listeners-disable Disable dynamic listeners.
--dynamic-sequential-min-port int If set to non-zero, makes the dynamic listener use a sequential port starting with this value rather than a random port every time.
--external-server-mapping stringArray Mapping of Kafka server address to external address (host:port,host:port). A listener for the external address is not started
--forbidden-api-keys ints Forbidden Kafka request types. The restriction should prevent some Kafka operations e.g. 20 - DeleteTopics
--forward-proxy string URL of the forward proxy. Supported schemas are socks5 and http
--gssapi-auth-type string GSSAPI auth type: KEYTAB or USER (default "KEYTAB")
--gssapi-disable-pa-fx-fast Used to configure the client to not use PA_FX_FAST.
--gssapi-keytab string krb5.keytab file location
--gssapi-krb5 string krb5.conf file path, default: /etc/krb5.conf (default "/etc/krb5.conf")
--gssapi-password string Password for auth type USER
--gssapi-realm string Realm
--gssapi-servicename string ServiceName (default "kafka")
--gssapi-spn-host-mapping stringToString Mapping of Kafka servers address to SPN hosts (default [])
--gssapi-username string Username (default "kafka")
-h, --help help for server
--http-disable Disable HTTP endpoints
--http-health-path string Path on which to health endpoint (default "/health")
--http-listen-address string Address that kafka-proxy is listening on (default "0.0.0.0:9080")
--http-metrics-path string Path on which to expose metrics (default "/metrics")
--kafka-client-id string An optional identifier to track the source of requests (default "kafka-proxy")
--kafka-connection-read-buffer-size int Size of the operating system's receive buffer associated with the connection. If zero, system default is used
--kafka-connection-write-buffer-size int Sets the size of the operating system's transmit buffer associated with the connection. If zero, system default is used
--kafka-dial-timeout duration How long to wait for the initial connection (default 15s)
--kafka-keep-alive duration Keep alive period for an active network connection. If zero, keep-alives are disabled (default 1m0s)
--kafka-max-open-requests int Maximal number of open requests pro tcp connection before sending on it blocks (default 256)
--kafka-read-timeout duration How long to wait for a response (default 30s)
--kafka-write-timeout duration How long to wait for a transmit (default 30s)
--log-format string Log format text or json (default "text")
--log-level string Log level debug, info, warning, error, fatal or panic (default "info")
--log-level-fieldname string Log level fieldname for json format (default "@level")
--log-msg-fieldname string Message fieldname for json format (default "@message")
--log-time-fieldname string Time fieldname for json format (default "@timestamp")
--producer-acks-0-disabled Assume fire-and-forget is never sent by the producer. Enabling this parameter will increase performance
--proxy-listener-ca-chain-cert-file string PEM encoded CA's certificate file. If provided, client certificate is required and verified
--proxy-listener-cert-file string PEM encoded file with server certificate
--proxy-listener-cipher-suites strings List of supported cipher suites
--proxy-listener-curve-preferences strings List of curve preferences
--proxy-listener-keep-alive duration Keep alive period for an active network connection. If zero, keep-alives are disabled (default 1m0s)
--proxy-listener-key-file string PEM encoded file with private key for the server certificate
--proxy-listener-key-password string Password to decrypt rsa private key
--proxy-listener-read-buffer-size int Size of the operating system's receive buffer associated with the connection. If zero, system default is used
--proxy-listener-tls-enable Whether or not to use TLS listener
--proxy-listener-tls-required-client-subject strings Required client certificate subject common name; example; s:/CN=[value]/C=[state]/C=[DE,PL] or r:/CN=[^val.{2}$]/C=[state]/C=[DE,PL]; check manual for more details
--proxy-listener-write-buffer-size int Sets the size of the operating system's transmit buffer associated with the connection. If zero, system default is used
--proxy-request-buffer-size int Request buffer size pro tcp connection (default 4096)
--proxy-response-buffer-size int Response buffer size pro tcp connection (default 4096)
--sasl-aws-profile string AWS profile
--sasl-aws-region string Region for AWS IAM Auth
--sasl-enable Connect using SASL
--sasl-jaas-config-file string Location of JAAS config file with SASL username and password
--sasl-method string SASL method to use (PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, GSSAPI, AWS_MSK_IAM (default "PLAIN")
--sasl-password string SASL user password
--sasl-plugin-command string Path to authentication plugin binary
--sasl-plugin-enable Use plugin for SASL authentication
--sasl-plugin-log-level string Log level of the auth plugin (default "trace")
--sasl-plugin-mechanism string SASL mechanism used for proxy authentication: PLAIN or OAUTHBEARER (default "OAUTHBEARER")
--sasl-plugin-param stringArray Authentication plugin parameter
--sasl-plugin-timeout duration Authentication timeout (default 10s)
--sasl-username string SASL user name
--tls-ca-chain-cert-file string PEM encoded CA's certificate file
--tls-client-cert-file string PEM encoded file with client certificate
--tls-client-key-file string PEM encoded file with private key for the client certificate
--tls-client-key-password string Password to decrypt rsa private key
--tls-enable Whether or not to use TLS when connecting to the broker
--tls-insecure-skip-verify It controls whether a client verifies the server's certificate chain and host name
--tls-same-client-cert-enable Use only when mutual TLS is enabled on proxy and broker. It controls whether a proxy validates if proxy client certificate exactly matches brokers client cert (tls-client-cert-file)kafka-proxy server --bootstrap-server-mapping "192.168.99.100:32400,0.0.0.0:32399" kafka-proxy server --bootstrap-server-mapping "192.168.99.100:32400,127.0.0.1:32400" --bootstrap-server-mapping "192.168.99.100:32401,127.0.0.1:32401" --bootstrap-server-mapping "192.168.99.100:32402,127.0.0.1:32402" --dynamic-listeners-disable kafka-proxy server --bootstrap-server-mapping "kafka-0.example.com:9092,0.0.0.0:32401,kafka-0.grepplabs.com:9092" --bootstrap-server-mapping "kafka-1.example.com:9092,0.0.0.0:32402,kafka-1.grepplabs.com:9092" --bootstrap-server-mapping "kafka-2.example.com:9092,0.0.0.0:32403,kafka-2.grepplabs.com:9092" --dynamic-listeners-disable kafka-proxy server --bootstrap-server-mapping "192.168.99.100:32400,127.0.0.1:32400" --external-server-mapping "192.168.99.100:32401,127.0.0.1:32402" --external-server-mapping "192.168.99.100:32402,127.0.0.1:32403" --forbidden-api-keys 20 export BOOTSTRAP_SERVER_MAPPING="192.168.99.100:32401,0.0.0.0:32402 192.168.99.100:32402,0.0.0.0:32403" && kafka-proxy server
kafka-proxy server --bootstrap-server-mapping "localhost:19092,0.0.0.0:30001,localhost:30001" --bootstrap-server-mapping "localhost:29092,0.0.0.0:30002,localhost:30002" --bootstrap-server-mapping "localhost:39092,0.0.0.0:30003,localhost:30003" --proxy-listener-cert-file "tls/ca-cert.pem" --proxy-listener-key-file "tls/ca-key.pem" --proxy-listener-tls-enable --proxy-listener-cipher-suites TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256
بدأت مصادقة SASL بواسطة الوكيل. يتم تعطيل مصادقة SASL على العملاء وتمكينها على سماسرة Kafka.
kafka-proxy server --bootstrap-server-mapping "kafka-0.grepplabs.com:9093,0.0.0.0:32399" --tls-enable --tls-insecure-skip-verify --sasl-enable --sasl-username myuser --sasl-password mysecret kafka-proxy server --bootstrap-server-mapping "kafka-0.example.com:9092,0.0.0.0:30001" --bootstrap-server-mapping "kafka-1.example.com:9092,0.0.0.0:30002" --bootstrap-server-mapping "kafka-1.example.com:9093,0.0.0.0:30003" --sasl-enable --sasl-username "alice" --sasl-password "alice-secret" --sasl-method "SCRAM-SHA-512" --log-level debug make clean build plugin.unsecured-jwt-provider && build/kafka-proxy server --sasl-enable --sasl-plugin-enable --sasl-plugin-mechanism "OAUTHBEARER" --sasl-plugin-command build/unsecured-jwt-provider --sasl-plugin-param "--claim-sub=alice" --bootstrap-server-mapping "192.168.99.100:32400,127.0.0.1:32400"
مصادقة GSSAPI / Kerberos
kafka-proxy server --bootstrap-server-mapping "kafka-0.grepplabs.com:9092,127.0.0.1:32500" --bootstrap-server-mapping "kafka-1.grepplabs.com:9092,127.0.0.1:32501" --bootstrap-server-mapping "kafka-2.grepplabs.com:9092,127.0.0.1:32502" --sasl-enable --sasl-method "GSSAPI" --gssapi-servicename kafka --gssapi-username kafkaclient1 --gssapi-realm EXAMPLE.COM --gssapi-krb5 /etc/krb5.conf --gssapi-keytab /etc/security/keytabs/kafka.keytab
AWS MSK IAM
kafka-proxy server --bootstrap-server-mapping "b-1-public.kafkaproxycluster.uls9ao.c4.kafka.eu-central-1.amazonaws.com:9198,0.0.0.0:30001" --bootstrap-server-mapping "b-2-public.kafkaproxycluster.uls9ao.c4.kafka.eu-central-1.amazonaws.com:9198,0.0.0.0:30002" --bootstrap-server-mapping "b-3-public.kafkaproxycluster.uls9ao.c4.kafka.eu-central-1.amazonaws.com:9198,0.0.0.0:30003" --tls-enable --tls-insecure-skip-verify --sasl-enable --sasl-method "AWS_MSK_IAM" --sasl-aws-region "eu-central-1" --log-level debug
يتم تنفيذ مصادقة SASL بواسطة الوكيل. يتم تمكين مصادقة SASL على العملاء والتعطيل على سماسرة Kafka.
make clean build plugin.auth-user && build/kafka-proxy server --proxy-listener-key-file "server-key.pem" --proxy-listener-cert-file "server-cert.pem" --proxy-listener-ca-chain-cert-file "ca.pem" --proxy-listener-tls-enable --auth-local-enable --auth-local-command build/auth-user --auth-local-param "--username=my-test-user" --auth-local-param "--password=my-test-password" make clean build plugin.auth-ldap && build/kafka-proxy server --auth-local-enable --auth-local-command build/auth-ldap --auth-local-param "--url=ldaps://ldap.example.com:636" --auth-local-param "--user-dn=cn=users,dc=exemple,dc=com" --auth-local-param "--user-attr=uid" --bootstrap-server-mapping "192.168.99.100:32400,127.0.0.1:32400" make clean build plugin.unsecured-jwt-info && build/kafka-proxy server --auth-local-enable --auth-local-command build/unsecured-jwt-info --auth-local-mechanism "OAUTHBEARER" --auth-local-param "--claim-sub=alice" --auth-local-param "--claim-sub=bob" --bootstrap-server-mapping "192.168.99.100:32400,127.0.0.1:32400"
التحقق من صحة شهادة العميل المستخدمة من قبل عميل الوكيل هي بالضبط نفس شهادة العميل في المصادقة التي بدأها الوكيل
kafka-proxy server --bootstrap-server-mapping "kafka-0.grepplabs.com:9093,0.0.0.0:32399" --tls-enable --tls-client-cert-file client.crt --tls-client-key-file client.pem --tls-client-key-password changeit --proxy-listener-tls-enable --proxy-listener-key-file server.pem --proxy-listener-cert-file server.crt --proxy-listener-key-password changeit --proxy-listener-ca-chain-cert-file ca.crt --tls-same-client-cert-enable
المصادقة بين عميل Kafka Proxy و Kafka Proxy Server مع Google-ID (حساب الخدمة JWT)
kafka-proxy server --bootstrap-server-mapping "kafka-0.grepplabs.com:9092,127.0.0.1:32500" --bootstrap-server-mapping "kafka-1.grepplabs.com:9092,127.0.0.1:32501" --bootstrap-server-mapping "kafka-2.grepplabs.com:9092,127.0.0.1:32502" --dynamic-listeners-disable --http-disable --proxy-listener-tls-enable --proxy-listener-cert-file=/var/run/secret/server.cert.pem --proxy-listener-key-file=/var/run/secret/server.key.pem --auth-gateway-server-enable --auth-gateway-server-method google-id --auth-gateway-server-magic 3285573610483682037 --auth-gateway-server-command google-id-info --auth-gateway-server-param "--timeout=10" --auth-gateway-server-param "--audience=tcp://kafka-gateway.grepplabs.com" --auth-gateway-server-param "--email-regex=^[email protected]$" kafka-proxy server --bootstrap-server-mapping "127.0.0.1:32500,127.0.0.1:32400" --bootstrap-server-mapping "127.0.0.1:32501,127.0.0.1:32401" --bootstrap-server-mapping "127.0.0.1:32502,127.0.0.1:32402" --dynamic-listeners-disable --http-disable --tls-enable --tls-ca-chain-cert-file /var/run/secret/client/ca-chain.cert.pem --auth-gateway-client-enable --auth-gateway-client-method google-id --auth-gateway-client-magic 3285573610483682037 --auth-gateway-client-command google-id-provider --auth-gateway-client-param "--credentials-file=/var/run/secret/client/service-account.json" --auth-gateway-client-param "--target-audience=tcp://kafka-gateway.grepplabs.com" --auth-gateway-client-param "--timeout=10"
تواصل من خلال خادم اختبار Socks5 Proxy
kafka-proxy tools socks5-proxy --addr localhost:1080 kafka-proxy server --bootstrap-server-mapping "kafka-0.grepplabs.com:9092,127.0.0.1:32500" --bootstrap-server-mapping "kafka-1.grepplabs.com:9092,127.0.0.1:32501" --bootstrap-server-mapping "kafka-2.grepplabs.com:9092,127.0.0.1:32502" --forward-proxy socks5://localhost:1080
kafka-proxy tools socks5-proxy --addr localhost:1080 --username my-proxy-user --password my-proxy-password kafka-proxy server --bootstrap-server-mapping "kafka-0.grepplabs.com:9092,127.0.0.1:32500" --bootstrap-server-mapping "kafka-1.grepplabs.com:9092,127.0.0.1:32501" --bootstrap-server-mapping "kafka-2.grepplabs.com:9092,127.0.0.1:32502" --forward-proxy socks5://my-proxy-user:my-proxy-password@localhost:1080
الاتصال من خلال اختبار خادم وكيل HTTP باستخدام طريقة الاتصال
kafka-proxy tools http-proxy --addr localhost:3128 kafka-proxy server --bootstrap-server-mapping "kafka-0.grepplabs.com:9092,127.0.0.1:32500" --bootstrap-server-mapping "kafka-1.grepplabs.com:9092,127.0.0.1:32501" --bootstrap-server-mapping "kafka-2.grepplabs.com:9092,127.0.0.1:32502" --forward-proxy http://localhost:3128
kafka-proxy tools http-proxy --addr localhost:3128 --username my-proxy-user --password my-proxy-password kafka-proxy server --bootstrap-server-mapping "kafka-0.grepplabs.com:9092,127.0.0.1:32500" --bootstrap-server-mapping "kafka-1.grepplabs.com:9092,127.0.0.1:32501" --bootstrap-server-mapping "kafka-2.grepplabs.com:9092,127.0.0.1:32502" --forward-proxy http://my-proxy-user:my-proxy-password@localhost:3128
في بعض الأحيان ، قد يكون من الضروري ليس فقط التحقق من صحة أن شهادة العميل صالحة ولكن أيضًا يتم إصدار شهادة العميل DN لحالة استخدام ملموسة. يمكن تحقيق ذلك باستخدام مجموعة الوسيطات التالية:
--proxy-listener-tls-client-cert-validate-subject bool Whether to validate client certificate subject (default false) --proxy-listener-tls-required-client-subject-common-name string Required client certificate subject common name --proxy-listener-tls-required-client-subject-country stringArray Required client certificate subject country --proxy-listener-tls-required-client-subject-province stringArray Required client certificate subject province --proxy-listener-tls-required-client-subject-locality stringArray Required client certificate subject locality --proxy-listener-tls-required-client-subject-organization stringArray Required client certificate subject organization --proxy-listener-tls-required-client-subject-organizational-unit stringArray Required client certificate subject organizational unit
عن طريق الإعداد --proxy-listener-tls-client-cert-validate-subject true ، سوف يدرس وكيل أن وكيل الحقول DN من أجل القيم المتوقعة المعة مع- --proxy-listener-tls-required-client-* الوسيطات. تكون المباريات دائمًا دقيقة وتستخدم معًا ، لجميع القيم غير الفارغة. على سبيل المثال ، للسماح بشهادة صالحة للبلد country=DE and organization=grepplabs ، قم بتكوين وكيل kafka بالطريقة التالية:
kafka-proxy server --proxy-listener-tls-client-cert-validate-subject true --proxy-listener-tls-required-client-subject-country DE --proxy-listener-tls-required-client-subject-organization grepplabs
--- apiversion: التطبيقات/v1kind: DeploymentMetadata: الاسم: myappspec: النسخ المتماثلة: 1
المحدد: MatchLabels: App: MyApp
قالب: بيانات التعريف: التسميات: التطبيق: التعليقات التوضيحية MyApp: Prometheus.io/scrape: 'True'spec: حاويات:
-الاسم: Kafka-proxy الصورة: Grepplabs/Kafka-proxy: آخر args:
-'server'-'--log-format = json'- '-bootstrap-server-mapping = kafka-0: 9093،127.0.0.1: 32400'-'-bootstrap-server = kafka-1: 9093،127.0.0.1: 32401'- '-Bootstrap-server-mapping = Kafka-2: 9093،127.0.0.1: 32402'-'-tls-enable'- '-tls-ca-chain-cert- file =/var/run/secret/kafka-ca-chain-certificate/ca-chain.cert.pem'- '-tls-cert-file =/var/run/secret/kafka-certificate/ client.cert.pem'- '--TLS-Client-key-file =/var/run/secret/kafka-client-key/client.key.pem'-'-tls-client-key-password = $ (TLS_CLIENT_KEY_PASSWORD) '-'-sasl-enable'- '-sasl-jaas-config-file =/var/run/secret/kafka-client-jaas/jaas.config' env:
-الاسم: tls_client_key_passwordvaluefrom: secretKeyRef: الاسم: tls-client-key-basswkey: voluredsounts:
-الاسم: "Sasl-Jaas-Config-File" MountPath: "/var/run/secret/kafka-client-jaas"-الاسم: "TLS-CA-Chain-Certificate" MountPath: "/var/secret/secret/ kafka-ca-chain-certificate "-الاسم:" tls-cert-file "mountpath:"/var/run/secret/kafka-client-certificate "-الاسم:" tls-client-key-file "mountpath: "/var/run/secret/kafka-client-key" المنافذ:
- الاسم: metricscontainerport: 9080 livengeprobe: httpget: path: /Health Port: 5- الاسم: myapp صورة: myapp: أحدث المنافذ:
- حاوية: 8080NAME: مقاييس ENV:
- الاسم: bootstrap_serversvalue: "127.0.0.1:32400،127.0.0.1:32401،127.0.0.1:32402" volumes:
-الاسم: SASL-Jaas-Config-FilesCret: SecretName: SASL-Jaas-Config-File-الاسم: TLS-CA-CA-CA-CERTIFICATESECRET: SecretName: TLS-CA-CA-CERTIFICATE-الاسم: TLS-CERT-FILESECRET : SecretName: TLS-Cert-Certfile-الاسم: TLS-CLIENT-KEY-FILESECRET: SecretName: TLS-CLIENT-KEY --- Apiversion: Apps/V1Kind: StateFulsetMetadata: الاسم: Kafka-Proxyspec: Selector: MatchLabels: App: Kafka-Proxy
النسخ المتماثلة: 1
Servicename: كافكا بروكسي
قالب: بيانات التعريف: التسميات: التطبيق: kafka-proxyspec: الحاويات:
-الاسم: Kafka-proxy الصورة: Grepplabs/Kafka-proxy: آخر args:
-'server'-'--log-format = json'- '-bootstrap-server-mapping = kafka-0: 9093،127.0.0.1: 32400'-'-bootstrap-server = kafka-1: 9093،127.0.0.1: 32401'- '-Bootstrap-server-mapping = Kafka-2: 9093،127.0.0.1: 32402'-'-tls-enable'- '-tls-ca-chain-cert- file =/var/run/secret/kafka-ca-chain-certificate/ca-chain.cert.pem'- '-tls-cert-file =/var/run/secret/kafka-certificate/ client.cert.pem'- '--TLS-Client-key-file =/var/run/secret/kafka-client-key/client.key.pem'-'-tls-client-key-password = $ (TLS_CLIENT_KEY_PASSWORD) '-'-sasl-enable'- '-sasl-jaas-config-file =/var/run/secret/kafka-client-jaas/jaas.config'-'-proxy-request-buffer -size = 32768'- '-proxy-reponse-buffer-size = 32768'-'-proxy-listener-read-size-size = 32768'- '-proxy-listener-write-buffer-size = 131072 '-'-kafka-connection-read-buffer-size = 131072'- '-kafka-connection-write-buffer-size = 32768' env:
-الاسم: tls_client_key_passwordvaluefrom: secretKeyRef: الاسم: tls-client-key-basswkey: voluredsounts:
-الاسم: "Sasl-Jaas-Config-File" MountPath: "/var/run/secret/kafka-client-jaas"-الاسم: "TLS-CA-Chain-Certificate" MountPath: "/var/secret/secret/ kafka-ca-chain-certificate "-الاسم:" tls-cert-file "mountpath:"/var/run/secret/kafka-client-certificate "-الاسم:" tls-client-key-file "mountpath: "/var/run/secret/kafka-client-key" المنافذ:
- الاسم: metricscontainerport: 9080 - الاسم: kafka -0containerport: 32400 - kafka -1containerport: 32401 - الاسم: kafka -2containerpor الطريق : /ميناء الصحة: 9080InitialDelaySeconds: 5periodseconds: 10TimeOtseconds: 5 SugcecessThold: 2failurethreshold: 5 الموارد: الطلبات: الذاكرة: 128 ميجا بوكية وحدة المعالجة المركزية: 1000m Retartpolicy: دائمًا أحجام:
-الاسم: SASL-Jaas-Config-FilesCret: SecretName: SASL-Jaas-Config-File-الاسم: TLS-CA-CA-CA-CERTIFICATESECRET: SecretName: TLS-CA-CA-CERTIFICATE-الاسم: TLS-CERT-FILESECRET : SecretName: TLS-Cert-Certfile-الاسم: TLS-CLIENT-KEY-FILESECRET: SecretName: TLS-CLIENT-KEYKubectl Port-Forward Kafka-Proxy-0 32400: 32400 32401: 32401 32402: 32402
استخدم المضيف المحلي: 32400 ، المضيف المحلي: 32401 و LocalHost: 32402 كخوادم bootstrap
kafka.properties
broker.id=0 advertised.listeners=PLAINTEXT://kafka-0.kafka-headless.kafka:9092 ...
Kubectl Port-Forward -N Kafka Kafka-0 9092: 9092
Kafka-Proxy Server-Bootsstrap-Server-Mapping "127.0.0.1:9092،0.0.0
استخدم المضيف المحلي: 19092 كخوادم bootstrap
Strimzi 0.13.0 CRD
Apiversion: kafka.strimzi.io/v1beta1kind: kafkametadata: الاسم: مجموعة الاختبار
مساحة الاسم: kafkaspec: kafka: الإصدار: 2.3.0replicas: 3listeners: plain: {} tls: {} config: Offsets.topic.replication.factor: 3 Transaction.State.log.Replication.factor: 3 concles.state.log. min.isr: 2 num.partitions: 60 default.replication.factor: 3Storage: Type: JBod Volumes:
- معرف: 0 النوع: الدعامة المستمرة: 20GI DELETECALY: TRUE
Zookeeper: النسخ المتماثلة: 3storage: النوع: الحجم المستمر: 5Gi Deleteclad: True
EntityOperator: TopicOperator: {} userOperator: {}Kubectl Port-Forward -N Kafka Test-Cluster-Kafka-0 9092: 9092 Kubectl Port-Forward -N Kafka Test-Cluster-Kafka-1 9093: 9092 Kubectl Port-Forward -N Kafka Test-Cluster-Kafka-2 9094: 9092 خادم Kafka-proxy-تصحيح مستوى السطوح -Bootstrap-Server-Mapping "127.0.0.1:9092،0.0.0.0:19092" -Bootstrap-Server-Mapping "127.0.0.1:9093،0.0.0.0:19093" -Bootstrap-Server-Mapping "127.0.0.1:9094،0.0.0.0:19094" --Dial-Address Mapping "Test-Cluster-Kafka-0.Test-Cluster-Kafka-Hers.kafka.svc.cluster.local: 9092،0.0.0.0: 9092" --Dial-Address Mapping "Test-Cluster-Kafka-1.Test-Cluster-Kafka-Hers.kafka.svc.cluster.local: 9092،0.0.0.0: 9093" --Dial-Address Mapping "Test-Cluster-Kafka-2.Test-Cluster-Kafka-Hers.kafka.svc.cluster.local: 9092،0.0.0.0: 9094"
استخدم المضيف المحلي: 19092 كخوادم bootstrap
Cloud SQL Proxy
ساراما
