Skip to main content
Version: 3.7.0

Custom Mounts and Non-Privileged Container Deployment

Overview​

In Kubernetes, a DaemonSet is a special resource object that ensures each node in the cluster runs one and only one Pod instance.
By deploying SmartAgent with a DaemonSet, deployment is automatically completed across all cluster nodes without manual installation on each node.

This document provides three mounting and permission strategies to adapt to different business security requirements:

  • The mounting scope is gradually reduced from maximum to minimum, while security is gradually improved.
  • All three modes support non-privileged containers (controlled via privileged and capabilities settings).

Deployment Preparation​

Refer to Kubernetes Deployment (DaemonSet) to obtain all necessary files for deployment.

Mounting Mode Selection​

  • Host root directory mounted as read-only, reducing risk of host filesystem damage
  • Only installation and runtime directories are mounted as read/write
  • Runtime prunes unnecessary system directories (e.g., boot, initrd, root, vmlinuz)
  • Use cases: General production environments, balance of functionality and security
warning
  • Installation and runtime directories must be mounted separately with all directory levels having permissions ≥ 755.
  • When switching runtime or installation directories, mount the original directory in read/write mode.
  • For non-root users, /etc must be mounted as read/write; otherwise, fallback to root execution.
kubernetes.yaml Key Snippet
spec:
  template:
    spec:
      hostPID: true      # Used to identify the process list on the host by entering the host's PID namespace
      hostNetwork: true  # Used to collect host network information by entering the host's network namespace; required for eBPF-based network analysis
      volumes:
        # Installation directory. If customization is required, it must be used together with SMARTAGENT_INSTALL_PATH
        - name: host-install-path
          hostPath:
            path: /opt
            type: DirectoryOrCreate
        # [Optional] Custom runtime directory. Must be used together with SMARTAGENT_RUNTIME_PATH
        # Do not use the same directory as the installation directory
        #- name: custom-runtime-path
        #  hostPath:
        #    path: /data/lib/bonree
        #    type: DirectoryOrCreate

        # [Mandatory, cannot be removed] Even when using a custom runtime directory,
        # the default runtime directory mount must not be removed
        - name: host-runtime-path
          hostPath:
            path: /var/lib/bonree
            type: DirectoryOrCreate
        - name: host-crio-containers
          hostPath:
            path: /usr/share/containers
        - name: host-etc
          hostPath:
            path: /etc
        - name: host-root
          hostPath:
            path: /
        - name: container-tmp-dir
          emptyDir: {}
        - name: container-mnt
          emptyDir: {}
      containers:
        - name: bonree-smartagent
          securityContext:
            # [1.1] Added support for read-only containers
            readOnlyRootFilesystem: true
            privileged: true
          env:
            # [Optional] Installation directory path (default: /opt). Can be customized based on deployment needs.
            # Note: This path must be consistent with the host path and mounted as read-write;
            # all directory levels must have permissions no less than 755.
            - name: SMARTAGENT_INSTALL_PATH
              value: "/opt"
            # [Optional] [1.1] Custom runtime data directory (default: /var/lib). Supports specifying a custom path.
            # Note: This path must be consistent with the host path and mounted as read-write;
            # all directory levels must have permissions no less than 755.
            #- name: SMARTAGENT_RUNTIME_PATH
            #  value: "/data/lib"
          volumeMounts:
            # Installation directory. If customization is required, it must be used together with SMARTAGENT_INSTALL_PATH
            - name: host-install-path
              mountPath: /mnt/root/opt
              readOnly: false
            # [Optional] Custom runtime directory. Must be used together with SMARTAGENT_RUNTIME_PATH
            # Do not use the same directory as the installation directory
            #- name: custom-runtime-path
            #  mountPath: /mnt/root/data/lib/bonree
            #  readOnly: false
            - name: host-crio-containers
              mountPath: /mnt/root/usr/share/containers
              readOnly: false
            # [Optional] In read-only root filesystem scenario, /etc must be mounted as read-write
            # This is required to create a non-root user to run the SmartAgent service process.
            # If /etc is not mounted read-write, you must specify an existing host user,
            # otherwise the agent will fall back to running as root.
            - name: host-etc
              mountPath: /mnt/root/etc
              readOnly: false
            # [Required] Even when using a custom runtime path, the default runtime mount cannot be removed
            - name: host-runtime-path
              mountPath: /mnt/root/var/lib/bonree
              readOnly: false
            - name: host-root
              mountPath: /mnt/root
              readOnly: true
            - name: container-tmp-dir
              mountPath: /tmp/apm
              readOnly: false
            - name: container-mnt
              mountPath: /mnt
              readOnly: false

Non-Privileged Container Deployment​

Regardless of whether the container runs in privileged mode, SmartAgent automatically reduces the privilege scope of its service processes during startup and runs under the specified user with the minimal required set of Linux Capabilities.

In the deployment YAML, you can further restrict initial privileges via securityContext.capabilities, achieving stricter control from the container startup phase.

From SmartAgent image version 1.1 and SmartAgent version 9.7.0, required Capabilities are modularized and the agent automatically enables functionalities based on available permissions.

kubernetes.yaml Snippet
          env:
          # [Optional] Whether to disable secondary remounting at container runtime (default: false)
          # When the container does not have the SYS_ADMIN capability, further reduction of host mounts is not supported.
          # In such cases, this parameter must be set to true.
          #- name: SMARTAGENT_NO_REMOUNT_ROOT
          #  value: "false"
          securityContext:
            privileged: false
            # Non-root execution mode:
            # Must set allowPrivilegeEscalation=true to retain the minimal set of privileges required by the application.
            # If set to false, only root mode is supported, and you must explicitly set the startup argument:
            #   --non-root-mode false
            # to run the smartagent service as root.
            allowPrivilegeEscalation: true
            # [1.1] Added support for read-only containers
            readOnlyRootFilesystem: true
            capabilities:
              drop: [ "ALL" ] # Drop all default capabilities first
              add:
                - "CHOWN"
                - "DAC_OVERRIDE"
                - "DAC_READ_SEARCH"
                - "FOWNER"
                - "FSETID"
                - "SYS_RESOURCE"
                - "SYS_CHROOT"
                - "SYS_ADMIN"      # Removable since 9.7.0. If not used, you must also set SMARTAGENT_NO_REMOUNT_ROOT: true
                - "SYSLOG"         # Removable since 9.7.0
                - "NET_ADMIN"      # Removable since 9.7.0
                - "NET_RAW"        # Removable since 9.7.0
                - "NET_BROADCAST"  # Removable since 9.7.0
                - "KILL"           # Removable since 9.7.0 (does not affect any functionality)
                - "SETFCAP"        # Removable since 9.7.0
                - "SETGID"         # Removable since 9.7.0
                - "SETUID"         # Removable since 9.7.0
                - "SYS_PTRACE"     # Removable since 9.7.0

Impact of Using Non-Privileged Containers​

By default, the image includes a complete set of capabilities, but some features may be restricted in non-privileged mode:

  • Cannot access kernel logs /dev/kmsg, so certain host failure events cannot be reported, including:
    • PodOOMKilling (Pod killed due to OOM)
    • OOMKilling (system-wide OOM kills processes)
    • TaskHung (process stuck for a long time)
    • ReadonlyFilesystem (filesystem remounted read-only)

Capability Descriptions​

CAP_SETUID, CAP_SETGID
  • Purpose: Switch process user to non-root via setresuid().
  • Usage frequency: One-time
  • Required: No
  • Impact if missing: SmartAgent runs as root.
CAP_SETFCAP
  • Purpose: Assign minimal required privileges during initialization via cap_set_flag() from libcap.so.
  • Usage frequency: One-time
  • Required: No
  • Impact if missing: Privileges cannot be narrowed, fallback to root execution. Some features unavailable:
    • APM agent auto-injection
CAP_SYS_PTRACE
  • Purpose: Extract process info from /proc for monitoring, logging, profiling.
  • Usage frequency: Continuous
  • Required: No
  • Impact if missing: Features unavailable:
    • Container attributes & metrics collection
    • APM agent auto-injection
    • Network performance analysis
    • eBPF profiling
    • Host failure event collection
    • Application log auto-discovery
CAP_DAC_OVERRIDE, CAP_FOWNER, CAP_DAC_READ_SEARCH
  • Purpose: Bypass file permissions for accessing logs, system files, container metadata.
  • Usage frequency: Continuous
  • Required: Yes
  • Impact if missing: Features unavailable:
    • Application auto-discovery
    • Container info & metrics collection
    • APM agent auto-injection
    • Network performance analysis
    • eBPF profiling
    • Host failure event collection
CAP_CHOWN, CAP_FSETID
  • Purpose: Set agent file permissions.
  • Usage frequency: One-time
  • Required: Yes
  • Impact if missing: File permission errors during user switching, causing agent failures.
CAP_SYS_CHROOT
  • Purpose: chroot into trimmed virtual root during startup.
  • Usage frequency: Continuous
  • Required: Yes
  • Impact if missing: SmartAgent container cannot start.
CAP_SYS_RESOURCE
  • Purpose: Control resource usage, remove BPF restrictions (via setrlimit()).
  • Usage frequency: One-time
  • Required: Yes
  • Impact if missing: Features unavailable:
    • Network performance analysis
    • eBPF profiling
    • Host failure event collection
CAP_NET_ADMIN, CAP_NET_RAW, CAP_NET_BROADCAST
  • Purpose: Required for eBPF network agents.
  • Usage frequency: Continuous
  • Required: No
  • Impact if missing: Features unavailable:
    • Network performance analysis
    • eBPF profiling
    • Host failure event collection
CAP_SYSLOG
  • Purpose: Access /dev/kmsg for kernel call stacks.
  • Usage frequency: Continuous
  • Required: No
  • Impact if missing: Unable to collect kernel call stacks.
CAP_SYS_ADMIN

  • Purpose:

    • Used with setns() to switch to the target container’s network namespace for collecting NIC information.
    • Used with mount() to mount the agent into the target container.
    • Required by eBPF agents to call bpf-related functions.

  • Frequency of Use: Continuous

  • Required: No

  • Impact Without Privilege:

    • APM agent auto-injection, network performance analysis, eBPF profiling, and host failure event collection are not supported.
      • For APM agent auto-injection, you can use the Monitor-Only Application Mode in SmartAgent Operator as an alternative.
    • In Minimal Host Mount Mode, secondary remounting is not supported (only read-only mounts configured in YAML are allowed). To start the container successfully, you must add the environment variable: 
      SMARTAGENT_NO_REMOUNT_ROOT: true
    • Kubernetes attribute enhancement is not supported (because mount cannot be executed, the Kubernetes service account token cannot be re-mounted). As a result, related features are limited, such as:
      • LogsAgent filtering logs based on annotations and labels
      • Pod IP and Service Name attribute extraction
    • ConfigMap-based configuration file modifications are not supported