跳到主要内容
版本：3.7.0

故障排查工具

查看部署各实例状态​

部署完成后，环境中将包含以下组件：

  • 1个 deployment/smartagent-operator（全栈部署模式）
  • 1个 deployment/bonree-webhook （仅监控应用模式）
  • 1个 daemonset/bonree-smartagent-csi-driver （可选，仅监控应用模式）
  • 1个 daemonset/{smartkube名称}-smartagent（仅全栈部署模式）
[root@lll-arm-openeuler ~]# kubectl get all -n bonree-agent
NAME                                      READY   STATUS    RESTARTS   AGE
pod/bonree-smartagent-csi-driver-5gzdh    3/3     Running   3          5h11m
pod/bonree-webhook-8454b78bf6-tfhwz       1/1     Running   1          5h11m
pod/smartagent-operator-dd4994b47-kg4kv   1/1     Running   1          5h11m
pod/smartkube-smartagent-hvfs5            1/1     Running   25         4h29m

NAME                     TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
service/bonree-webhook   ClusterIP   10.96.168.8   <none>        443/TCP   5h11m

NAME                                          DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/bonree-smartagent-csi-driver   1         1         1       1            1           <none>          5h11m
daemonset.apps/smartkube-smartagent           1         1         1       1            1           <none>          4h42m

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/bonree-webhook        1/1     1            1           5h11m
deployment.apps/smartagent-operator   1/1     1            1           5h11m

NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/bonree-webhook-8454b78bf6       1         1         1       5h11m
replicaset.apps/smartagent-operator-dd4994b47   1         1         1       5h11m

查看SmartKube实例列表​

使用以下命令查看所有SmartKube实例：

[root@lll-arm-openeuler ~]# kubectl get sk -A
NAMESPACE      NAME        APIURL                                          STATUS      AGE
bonree-agent   smartkube   https://one.bonree.com/rest/apm/agentDown       Deploying   15h

查看SmartKube实例状态​

通过describe命令详细查看SmartKube实例状态：

[root@lll-arm-openeuler ~]# kubectl describe sk smartkube -n bonree-agent
Name:         smartkube
Namespace:    bonree-agent
Labels:       <none>
Annotations:  <none>
API Version:  bonree.com/v1alpha1
Kind:         SmartKube
Metadata:
  Creation Timestamp:  2023-12-04T07:11:49Z
  Generation:          1
  Resource Version:    3573037
  UID:                 52416042-1178-47e3-bb7d-06fbbe79aa2f
Spec:
  API URL:  https://one.bonree.com/rest/apm/agentDown
  Classic Full Stack:
    Enabled:  true
    Tolerations:
      Effect:    NoExecute
      Key:       node-role.kubernetes.io/master
      Operator:  Exists
  Controller Info:
    Account GUID:        xxxxxxxxxxxxxxxxx
    Controller URL:      https://oneupload.bonree.com/APM
    Log Controller URL:  https://onelog.joinbr.com/controller/logs
Status:
  Communication Host For Client:
    Host:      one.bonree.com
    Port:      443
    Protocol:  https
  Conditions:
    Last Transition Time:  2023-12-04T07:13:09Z
    Message:               Ready
    Reason:                TokenReady
    Status:                True
    Type:                  PaaSToken  # PaaSToken状态
  Connection Info:
    Account GUID:  xxxxxxxxxxxxxxxxx
    Communication Hosts:
      Host:                           oneupload.bonree.com
      Port:                           443
      Protocol:                       https
  Kube System UUID:                   7a921db9-631d-4032-a828-07f9f10bf3c3
  Last Paa S Token Probe Timestamp:   2023-12-04T11:35:22Z
  Latest Agent Version Unix Default:  7.4.2     # 检测到的SmartAgent最新版本
  Latest Agent Version Unix Paas:     7.4.2     # 检测到的SmartAgent Paas 最新版本
  Phase:                              Running   # 部署状态
  Smart Agent:
    Instances: # SmartAgent 全栈模式部署的Pod实例信息
      Kind - Control - Plane: # 节点名
        Ip Address:                172.18.0.2  # 节点ip
        Pod Name:                  smartkube-smartagent-hvfs5 # pod名称
    Last Hosts Request Timestamp:  2023-12-04T11:35:22Z
    Last Update Probe Timestamp:   2023-12-04T11:30:22Z
    Version:                       7.4.2   # 安装的版本
  Tokens:                          smartkube # 使用的 secret 名称
  Updated Timestamp:               2023-12-04T11:35:22Z
Events:                            <none>  # 部署异常事件, webhook 注入serverless探针事件等
  Type    Reason  Age                   From            Message
  ----    ------  ----                  ----            -------
  Normal  Inject  9m53s (x18 over 93m)  bonree-webhook  Injecting the necessary info into pod nginx-deployment-56c88b7477- in namespace default

各组件日志查看方法​

# operator 日志
kubectl -n bonree-agent logs -f deployment/smartagent-operator
# webhook 日志
kubectl -n bonree-agent logs -f deployment/bonree-webhook
# csi-driver 日志
kubectl -n bonree-agent logs -f daemonset/bonree-smartagent-csi-driver bonree-driver
kubectl -n bonree-agent logs -f daemonset/bonree-smartagent-csi-driver bonree-registrar
kubectl -n bonree-agent logs -f daemonset/bonree-smartagent-csi-driver bonree-liveness-probe
# smartagent pod日志
kubectl -n bonree-agent logs -f {smartkube实例名}-smartagent-xxxxx
# 仅监控应用模式下的 install-smartagent sidecar容器日志
kubectl logs 目标Pod -c install-smartagent

CSI Driver 缓存探针所在路径及目录结构​

tree /var/lib/kubelet/plugins/csi.smartagent.bonree.com -L 4
.
|-- csi.sock
`-- data
    |-- 6505b917-b1b4-11ed-a56c-a2f32a921294 // accountGUID
    |   |-- bin // 存储各版本探针目录，以版本号命名
    |   |   `-- 8.5.0
    |   |-- gc // 各版本挂载
    |   |   `-- 8.5.0
    |   |-- run // 各个应用pod，运行时挂载目录，对容器内探针文件的修改不影响下次启动，如需全局生效，需要修改bin目录下的文件
    |   |   |-- csi-44ff4cb776e917743698aa198f060f2ab7db617f9c614cdc66f621a07b61ee4e
    |   |   |-- csi-a7703e2d35dab0a20b15eac8427022dcb86a5c82ae5fcf22d163fa63b3952641
    |   |   |-- csi-e3ec6b87ef258fae36704029abab2b926a6f7f1e7ba737d5cddbbb466b37bb93
    |   |   |-- csi-e8ead32dce314443e287c59c491cc4dc4cf4947c652c59276eb2bc8aaeadd300
    |   |   `-- csi-edd52ef3ef48aecc52014b8d074feccc8ab2376a72fb502d557c9b4e6ad2e2bd
    |   `-- version (当前版本)
    |-- gc
    |   |-- csi-a7703e2d35dab0a20b15eac8427022dcb86a5c82ae5fcf22d163fa63b3952641
    |   |-- csi-e3ec6b87ef258fae36704029abab2b926a6f7f1e7ba737d5cddbbb466b37bb93
    |   |-- csi-e8ead32dce314443e287c59c491cc4dc4cf4947c652c59276eb2bc8aaeadd300
    |   `-- csi-edd52ef3ef48aecc52014b8d074feccc8ab2376a72fb502d557c9b4e6ad2e2bd
    `-- tenant-smartkube // 当前生效的accountGUID

13 directories, 7 files

查看部署、注入事件​

[root@lll-arm-openeuler ~]# kubectl get event -n bonree-agent
Events:                            <none>  # 部署异常事件, webhook 注入serverless探针事件等
  Type    Reason  Age                   From            Message
  ----    ------  ----                  ----            -------
  Normal  Inject  9m53s (x18 over 93m)  bonree-webhook  Injecting the necessary info into pod nginx-deployment-56c88b7477- in namespace default