编程知识 cdmana.com

kubernetes环境利用sidecar采集业务日志(filebeat+kafka+es)

kubernetes环境利用sidecar模式采集业务日志

概述

业务服务上k8s后需要平衡的将logback产生的业务日志采集并且持久,并且通过自研后台方便查询到日志信息。

处理流程

整体日志处理流程如下

关键技术点

  • 业务container如何与filebeat container共享 volume
  • 将特定container env加入kafka messagetag
  • filebeat多行读取异常信息

整体配置内容

这边只展示kubernetes相关配置信息,logstash配置信息没有展示

sidecar filebeat cm

logging.level: info
logging.to_files: true
logging.files:
     path: /logging
     name: filebeat
     keepfiles: 7
     permissions: 0644

filebeat.config.inputs:
   enabled: true
   path: /filebeat/filebeat.yml
   reload.enabled: true
   reload.period: 1m

queue.mem:
   events: 60
   flush.min_events: 10
   flush.timeout: 2s
logging.metrics.enabled: false
monitoring.enabled: false
http.enabled: true
http.port: 5067
http.host: "0.0.0.0"
#=========================== Filebeat prospectors =============================
filebeat.inputs:
- type: log
  enabled: true
  paths:
     - /gc-log/logs/gc.log
  tail_files: true
  close_removed: true
  scan_frequency: 2m
  fields:
    logResource: "gc-log"
- type: log
  enabled: true
  paths:
    - /gc-log/logs/${APP_NAME}/*.log
  fields:
    profile: ${PROFILE}
    logResource: "server-log"
  multiline.pattern: '^\[.+\]'
  multiline.negate: true
  multiline.match: "after"
  tail_files: true
  close_removed: true
  scan_frequency: 2m
#================================ kafka =====================================
# kafka: kafka-logcenter-node0
output.kafka:
  hosts: ["x.x.x.x:9092","x.x.x.x:9092"]
  topic: "%{[fields.logResource]}"
  partition.round_robin:
  reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 200000
max_procs: 1

sidecar filebeat deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: default
  name: xxx-test
  labels:
    app: xxx-test
spec:
  replicas: 1
  revisionHistoryLimit: 0
  selector:
    matchLabels:
      app: xxx-test
  strategy:
    type: RollingUpdate
  template:
    metadata:
      name: xxx-test
      labels:
        app: xxx-test
    spec:
      containers:
        - name: xxx-test
          image: registry-vpc.cn-hangzhou.aliyuncs.com/tomcat8:v1
          imagePullPolicy: Always
          livenessProbe:
            initialDelaySeconds: 600
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 10
            httpGet:
              path: /
              port: 8080
          readinessProbe:
            initialDelaySeconds: 10
            periodSeconds: 20
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 10
            httpGet:
              path: /
              port: 8080
          ports:
            - name: "app-port"
              containerPort: 8080
            - name: "ssh-port"
              containerPort: 22
            - name: "jvm-port"
              containerPort: 8090
          resources:
            limits:
              memory: "500Mi"
              cpu: 0
            requests:
              memory: "500Mi"
              cpu: 0
          env:
            - name: JVM_XMX
              value: "300m"
            - name: JVM_XMS
              value: "200m"
            - name: APP_ID
              value: "$APP_ENG_NAME"
          volumeMounts:
          - name: gc-log
            mountPath: /data/
        - name: filebeat-gc
          image: prima/filebeat:6.2
          command: ["filebeat"]
          args: ["-c","/filebeat/filebeat.yml","-e", "-strict.perms=false"]
          resources:
            limits:
              memory: "80Mi"
            requests:
              memory: "60Mi"
          volumeMounts:
          - name: config
            mountPath: /filebeat
          - name: filebeat-data
            mountPath: /data
          - name: gc-log
            readOnly: true
            mountPath: /gc-log
      volumes:
      - name: config
        configMap:
          name: filebeat-gc-conf
      - name: filebeat-data
        emptyDir: {}
      - name: gc-log
        emptyDir: {}

版权声明
本文为[重力波波动]所创,转载请带上原文链接,感谢
https://my.oschina.net/u/4725115/blog/4839518

Scroll to Top