NVIDIA DCGM User Guide

1. What DCGM Is

DCGM (Data Center GPU Manager) is a toolset NVIDIA provides for managing and monitoring GPUs in the data center. It offers the following capabilities:

GPU behavior monitoring
GPU configuration management
GPU policy supervision
GPU health and diagnostics
GPU accounting and process statistics
NVSwitch configuration and monitoring

2. Installing DCGM

2.1 Installing libnvidia-nscq

GPUs are generally connected over NVLink. You can run nvidia-smi topo -m and check whether the output contains the word NVSwitch to decide whether this needs to be installed.

If NvSwitch is present, related information may fail to be queried

The error looks like this:

1
2
3
cat /var/log/nv-hostengine.log

09:52:13.204 ERROR [2539914:2539930] [[NvSwitch]] Not attached to NvSwitches. Aborting [/workspaces/dcgm-rel_dcgm_3_3-postmerge/modules/nvswitch/DcgmNvSwitchManager.cpp:975] [DcgmNs::DcgmNvSwitchManager::ReadNvSwitchStatusAllSwitches]

Check the NVIDIA driver version

1
2
nvidia-smi | grep "Driver Version"
| NVIDIA-SMI 570.158.01             Driver Version: 570.158.01     CUDA Version: 12.8     |

Install the matching version of libnvidia-nscq

1
apt install libnvidia-nscq-570

2.2 Installing DCGM

Add the repository

Ubuntu 20.04

1
2
3
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.0-1_all.deb
dpkg -i cuda-keyring_1.0-1_all.deb
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"

Ubuntu 22.04

1
2
3
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
dpkg -i cuda-keyring_1.0-1_all.deb
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"

Install DCGM

1
apt-get install -y datacenter-gpu-manager

Start the DCGM service

1
systemctl start nvidia-dcgm

You can also enable it to start on boot

1
systemctl enable nvidia-dcgm

3. Command-Line Help

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
dcgmi --help

Usage: dcgmi
   dcgmi subsystem
   dcgmi -v

Flags:
  -v    vv          获取 DCGMI 版本信息
        subsystem   要访问的目标子系统
 Subsystems Available:
        topo        GPU 拓扑信息（dcgmi topo -h 查看更多）
        stats       进程统计信息（dcgmi stats -h 查看更多）
        diag        系统验证/诊断（dcgmi diag -h 查看更多）
        policy      策略管理（dcgmi policy -h 查看更多）
        health      健康监控（dcgmi health -h 查看更多）
        config      配置管理（dcgmi config -h 查看更多）
        group       GPU 组管理（dcgmi group -h 查看更多）
        fieldgroup  字段组管理（dcgmi fieldgroup -h 查看更多）
        discovery   发现系统中的 GPU（dcgmi discovery -h 查看更多）
        introspect  收集 DCGM 本身的信息（dcgmi introspect -h 查看更多）
        nvlink      显示 NvLink 链路状态和错误计数（dcgmi nvlink -h 查看更多）
        dmon        GPU 统计监控（dcgmi dmon -h 查看更多）
        modules     控制并列出 DCGM 模块
        profile     控制并列出 DCGM 性能分析指标
        set         配置 hostengine 设置
  --    ignore_rest 忽略该标志后面的所有已标记参数
      --version     显示版本信息并退出
  -h  --help        显示使用说明并退出

4. Device Management

4.1 discovery: Viewing GPUs

1
dcgmi discovery -l

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
8 GPUs found.
+--------+----------------------------------------------------------------------+
| GPU ID | Device Information                                                   |
+--------+----------------------------------------------------------------------+
| 0      | Name: NVIDIA H20                                                     |
|        | PCI Bus ID: 00000000:0F:00.0                                         |
|        | Device UUID: GPU-e3425ddb-41fc-f3ca-4d3f-dd8cbcb6896b                |
+--------+----------------------------------------------------------------------+
...
0 NvSwitches found.
+-----------+
| Switch ID |
+-----------+
+-----------+
0 CPUs found.

4.2 topo: Viewing GPU Topology

1
dcgmi topo --gpuid 0

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
+-------------------+------------------------------------------------------------------------------+
| Topology Information                                                                             |
| GPU ID: 0                                                                                        |
+===================+==============================================================================+
| CPU Core Affinity | 0 - 55, 112 - 167                                                            |
| To GPU 1          | Connected via a CPU-level link                                               |
|                   | Connected via eighteen NVLINKs (Links: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) |
| To GPU 2          | Connected via a CPU-level link                                               |
|                   | Connected via eighteen NVLINKs (Links: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) |
| To GPU 3          | Connected via a CPU-level link                                               |
|                   | Connected via eighteen NVLINKs (Links: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) |
| To GPU 4          | Connected via a CPU-level link                                               |
|                   | Connected via eighteen NVLINKs (Links: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) |
| To GPU 5          | Connected via a CPU-level link                                               |
|                   | Connected via eighteen NVLINKs (Links: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) |
| To GPU 6          | Connected via a CPU-level link                                               |
|                   | Connected via eighteen NVLINKs (Links: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) |
| To GPU 7          | Connected via a CPU-level link                                               |
|                   | Connected via eighteen NVLINKs (Links: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) |
+-------------------+------------------------------------------------------------------------------+

You can see that GPU 0 is connected to the other GPUs over NVLink.

4.3 nvlink: Viewing Link Status and Error Counts

1
dcgmi nvlink -s

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
+----------------------+
|  NvLink Link Status  |
+----------------------+
GPUs:
    gpuId 0:
        U U U U U U U U U U U U U U U U U U
    gpuId 1:
        U U U U U U U U U U U U U U U U U U
    gpuId 2:
        U U U U U U U U U U U U U U U U U U
    gpuId 3:
        U U U U U U U U U U U U U U U U U U
    gpuId 4:
        U U U U U U U U U U U U U U U U U U
    gpuId 5:
        U U U U U U U U U U U U U U U U U U
    gpuId 6:
        U U U U U U U U U U U U U U U U U U
    gpuId 7:
        U U U U U U U U U U U U U U U U U U
NvSwitches:
    No NvSwitches found.

Key: Up=U, Down=D, Disabled=X, Not Supported=_

Each card has 18 NVLink links, and every link is Up.

5. Organization and Structure Management

5.1 group: Managing Groups

Groups let you group GPUs together, which makes them easier to monitor and manage.

Create a group

1
dcgmi group -c production

If you restart the DCGM service, the group is lost.

List groups

1
dcgmi group -l

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
+-------------------+----------------------------------------------------------+
| GROUPS                                                                       |
| 3 groups found.                                                              |
+===================+==========================================================+
| Groups            |                                                          |
| -> 0              |                                                          |
|    -> Group ID    | 0                                                        |
|    -> Group Name  | DCGM_ALL_SUPPORTED_GPUS                                  |
|    -> Entities    | GPU 0, GPU 1, GPU 2, GPU 3, GPU 4, GPU 5, GPU 6, GPU 7   |
| -> 1              |                                                          |
|    -> Group ID    | 1                                                        |
|    -> Group Name  | DCGM_ALL_SUPPORTED_NVSWITCHES                            |
|    -> Entities    | None                                                     |
| -> 2              |                                                          |
|    -> Group ID    | 2                                                        |
|    -> Group Name  | production                                               |
|    -> Entities    | None                                                     |
+-------------------+----------------------------------------------------------+

Add GPUs to a group

Here GPU 0 and GPU 1 are added to the production group.

1
dcgmi group -g 2 -a 0,1

View group information

1
dcgmi group -l

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
+-------------------+----------------------------------------------------------+
| GROUPS                                                                       |
| 3 groups found.                                                              |
+===================+==========================================================+
| Groups            |                                                          |
| -> 0              |                                                          |
|    -> Group ID    | 0                                                        |
|    -> Group Name  | DCGM_ALL_SUPPORTED_GPUS                                  |
|    -> Entities    | GPU 0, GPU 1, GPU 2, GPU 3, GPU 4, GPU 5, GPU 6, GPU 7   |
| -> 1              |                                                          |
|    -> Group ID    | 1                                                        |
|    -> Group Name  | DCGM_ALL_SUPPORTED_NVSWITCHES                            |
|    -> Entities    | None                                                     |
| -> 2              |                                                          |
|    -> Group ID    | 2                                                        |
|    -> Group Name  | production                                               |
|    -> Entities    | GPU 0, GPU 1                                             |
+-------------------+----------------------------------------------------------+

5.2 fieldgroup: Managing Field Groups

fieldgroup creates and maintains field groups. A field group is a collection of metrics used for monitoring and statistics; each field is one metric.

List field groups

1
dcgmi fieldgroup -l

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
3 field groups found.
+-------------------+----------------------------------------------------------+
| FIELD GROUPS                                                                 |
+===================+==========================================================+
| ID                | 1                                                        |
| Name              | DCGM_INTERNAL_30SEC                                      |
| Field IDs         | 300                                                      |
+-------------------+----------------------------------------------------------+
+-------------------+----------------------------------------------------------+
| FIELD GROUPS                                                                 |
+===================+==========================================================+
| ID                | 2                                                        |
| Name              | DCGM_INTERNAL_HOURLY                                     |
| Field IDs         | 501, 509, 510, 511, 512, 513                             |
+-------------------+----------------------------------------------------------+
+-------------------+----------------------------------------------------------+
| FIELD GROUPS                                                                 |
+===================+==========================================================+
| ID                | 3                                                        |
| Name              | DCGM_INTERNAL_JOB                                        |
| Field IDs         | 205, 155, 156, 200, 201, 202, 203, 204, 311, 100, 101,   |
|                   | 230, 221, 220, 240, 241, 242, 210, 211, 390, 391,        |
|                   | 392, 84, 241, 240, 409, 419, 429, 439                    |
+-------------------+----------------------------------------------------------+

Create a field group

1
dcgmi fieldgroup -c testgroup -f 1002,1003

Listing the groups again, you can see the newly created field group.

1
2
3
4
5
6
7
+-------------------+----------------------------------------------------------+
| FIELD GROUPS                                                                 |
+===================+==========================================================+
| ID                | 18                                                       |
| Name              | testgroup                                                |
| Field IDs         | 1002, 1003                                               |
+-------------------+----------------------------------------------------------+

6. Monitoring and Statistics

When monitoring and collecting statistics, you can specify a GPU card or a group, and the metrics can be selected either by ID or by field group.

6.1 profile: Performance Metrics

profile provides collection and querying of performance metrics, and must be used together with the dmon and stats commands to view the data.

View performance-related metrics

1
dcgmi profile -l

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
+----------------+----------+------------------------------------------------------+
| Group.Subgroup | Field ID | Field Tag                                            |
+----------------+----------+------------------------------------------------------+
| A.0            | 1001     | gr_engine_active                                     |
| A.0            | 1002     | sm_active                                            |
| A.0            | 1003     | sm_occupancy                                         |
| A.0            | 1004     | tensor_active                                        |
| A.0            | 1005     | dram_active                                          |
| A.0            | 1006     | fp64_active                                          |
| A.0            | 1007     | fp32_active                                          |
| A.0            | 1008     | fp16_active                                          |
| A.0            | 1009     | pcie_tx_bytes                                        |
| A.0            | 1010     | pcie_rx_bytes                                        |
| A.0            | 1011     | nvlink_tx_bytes                                      |
| A.0            | 1012     | nvlink_rx_bytes                                      |
| A.0            | 1013     | tensor_imma_active                                   |
| A.0            | 1014     | tensor_hmma_active                                   |
...

View the metrics available for a group or a specific GPU

1
dcgmi profile -g 2 -l

Pause and resume collection

This is mainly for nvprof, nsight compute, and nsight systems developers.

6.2 dmon: Real-Time Statistics Monitoring

View the available metric IDs

1
dcgmi fieldgroup -l

Specify a card group and metric IDs

1
dcgmi dmon -g 2 -e 100,101

1
2
3
4
#Entity   SMCLK        MMCLK
ID
GPU 1     1980         2619
GPU 0     1980         2619

Specify a card group and a metric group

1
dcgmi dmon -g 2 -f 2

1
2
3
4
#Entity   SPINF                       VTID                        VTPINF                      VTPNM                       VTPCLS                      VTPLC
ID
GPU 1     N/A                         N/A                         N/A                         N/A                         N/A                         N/A
GPU 0     N/A                         N/A                         N/A                         N/A                         N/A                         N/A

Specify a card directly and view a metric

1
dcgmi dmon -i 0 -e 1004

1
2
3
4
5
#Entity   TENSO
ID
GPU 0     N/A
GPU 0     0.000
GPU 0     0.000

6.3 stats: Process Statistics

Start collection

1
dcgmi stats -g 2 -e

View statistics for a process

1
dcgmi stats -g 2 -p 2038635 -v

View statistics for all processes

1
dcgmi stats -g 2 -j -v

Stop collection

1
dcgmi stats -g 2 -d

7. Health and Diagnostics

7.1 health: Health Monitoring

View the health monitoring items

1
dcgmi health -g 2 -f

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Health monitor systems report
+-----------------+--------------------------------------------------------------------+
| PCIe            | Off                                                                |
| NVLINK          | Off                                                                |
| Memory          | Off                                                                |
| SM              | Off                                                                |
| InfoROM         | Off                                                                |
| Thermal         | Off                                                                |
| Power           | Off                                                                |
| Driver          | Off                                                                |
| NvSwitch NF     | Off                                                                |
| NvSwitch F      | Off                                                                |
+-----------------+--------------------------------------------------------------------+

Enable all monitoring

1
dcgmi health -g 2 -s a

With dcgmi health -g 2 --clear

Check the DCGM service status

1
dcgmi health -g 2 -c

1
2
3
4
5
+---------------------------+----------------------------------------------------------+
| Health Monitor Report                                                                |
+===========================+==========================================================+
| Overall Health            | Healthy                                                  |
+---------------------------+----------------------------------------------------------+

When it is empty, the health status is good.

7.2 diag: Diagnosing Faults

It mainly provides the following diagnostic capabilities:

Environmental integrity of the NVML library, CUDA library, etc.
User access permissions to GPU devices
Driver or process conflicts
Status of hardware components such as GPU memory and InfoROM
Configuration such as persistence mode and environment variables

1
dcgmi diag -r 1

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
Successfully ran diagnostic for group.
+---------------------------+------------------------------------------------+
| Diagnostic                | Result                                         |
+===========================+================================================+
|-----  Metadata  ----------+------------------------------------------------|
| DCGM Version              | 3.3.9                                          |
| Driver Version Detected   | 570.158.01                                     |
| GPU Device IDs Detected   | 20f3,20f3,20f3,20f3,20f3,20f3,20f3,20f3        |
|-----  Deployment  --------+------------------------------------------------|
| Denylist                  | Pass                                           |
| NVML Library              | Pass                                           |
| CUDA Main Library         | Pass                                           |
| Permissions and OS Blocks | Pass                                           |
| Persistence Mode          | Pass                                           |
| Environment Variables     | Pass                                           |
| Page Retirement/Row Remap | Pass                                           |
| Graphics Processes        | Pass                                           |
| Inforom                   | Pass                                           |
+---------------------------+------------------------------------------------+

The -r here has four levels, where 1-4 correspond to different test coverage:

Plugin	Test Name	r1 (short) sec	r2 (medium) < 2 min	r3 (long) < 30 min	r4 (extra long) 1-2 hours
Software	software	yes	yes	yes	yes
PCIe + NVLink	pcie		yes	yes	yes
GPU Memory	memory		yes	yes	yes
Memory Bandwidth	memory_bandwidth		yes	yes	yes
Diagnostic	diagnostic			yes	yes
Targeted Stress	targeted_stress			yes	yes
Targeted Power	targeted_power			yes	yes
NVB Bandwidth	nvbandwidth			yes	yes
Memory Stress Test	memtest				yes
Input EDPp	pulse				yes

During diagnostics, if the cards are under load, some Warning alerts may be printed.

8. System Management

8.1 config: Configuration Management

View the configuration

1
dcgmi config -g 2 --get -v

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
+------------------------------+------------------------------+------------------------------+
| GPU ID: 0                                                                                  |
| NVIDIA H20                                                                                 |
+==============================+==============================+==============================+
| Field                        | Target                       | Current                      |
+------------------------------+------------------------------+------------------------------+
| Compute Mode                 | Not Specified                | Unrestricted                 |
| ECC Mode                     | Not Specified                | Enabled                      |
| Sync Boost                   | Not Specified                | Not Supported                |
| Memory Application Clock     | Not Specified                | 2619                         |
| SM Application Clock         | Not Specified                | 1980                         |
| Power Limit                  | Not Specified                | 500                          |
+------------------------------+------------------------------+------------------------------+
+------------------------------+------------------------------+------------------------------+
| GPU ID: 1                                                                                  |
| NVIDIA H20                                                                                 |
+==============================+==============================+==============================+
| Field                        | Target                       | Current                      |
+------------------------------+------------------------------+------------------------------+
| Compute Mode                 | Not Specified                | Unrestricted                 |
| ECC Mode                     | Not Specified                | Enabled                      |
| Sync Boost                   | Not Specified                | Not Supported                |
| Memory Application Clock     | Not Specified                | 2619                         |
| SM Application Clock         | Not Specified                | 1980                         |
| Power Limit                  | Not Specified                | 500                          |
+------------------------------+------------------------------+------------------------------+

Set the configuration

Limit the power to 499 W

1
dcgmi config -g 2 --set -P 499

Apply the configuration

1
dcgmi config -g 2 --enforce

View the configuration

1
dcgmi config -g 2 --get -v | grep "Power Limit"

1
2
| Power Limit                  | 499                          | 499                          |
| Power Limit                  | 499                          | 499                          |

8.2 policy: Policy Management

Policies define the actions triggered when a particular event occurs, which allows automatic handling of anomalies.

View the policy list

1
dcgmi policy --get

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
Policy information
+-----------------------------+------------------------------------------------+
| Policy Information                                                           |
| DCGM_ALL_SUPPORTED_GPUS                                                      |
+=============================+================================================+
| Violation conditions        | None                                           |
| Isolation mode              | Automatic                                      |
| Action on violation         | None                                           |
| Validation after action     | None                                           |
| Validation failure action   | None                                           |
+-----------------------------+------------------------------------------------+

Field	Meaning
Violation conditions	None → no violation trigger conditions are set (for example, temperature, power, and ECC errors are all unset)
Isolation mode	Automatic → the isolation mode is automatic; when a GPU has a problem, DCGM decides on its own whether to isolate it (the specific action still depends on whether a policy is set)
Action on violation	None → no action is taken on violation (for example, the GPU is not reset and the machine is not shut down)
Validation after action	None → no system validation is performed after the violation action (for example, a temperature or performance test)
Validation failure action	None → no action is taken when validation fails either

With the default configuration, DCGM does not perform any action on the GPUs.

Add a policy

The available action values are 0 - take no action; 1 - reset the GPU;

Validation runs after the action: 0 - no validation; 1 - check the basic health of the GPU core and memory; 2 - comprehensive memory and core check; 3 - comprehensive hardware check.

Actions and validations come in pairs. For example, 0,0 means take no action and no validation; 1,2 means reset the GPU and run a comprehensive memory and core check.

1
dcgmi policy -g 2 --set 1,2 -T 100

When the GPU temperature exceeds 100 degrees, DCGM executes the 1,2 policy.

The optional trigger conditions are -e ECC double-bit errors, -x xid errors, -p PCIe replay errors, -n NVLink errors, -T temperature, and -P power.

View the policy

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
dcgmi policy --get -v

Policy information
+-----------------------------+------------------------------------------------+
| Policy Information                                                           |
| GPU ID: 0                                                                    |
+=============================+================================================+
| Violation conditions        | Max temperature threshold - 100                |
| Isolation mode              | Manual                                         |
| Action on violation         | Reset GPU                                      |
| Validation after action     | System Validation (Medium)                     |
| Validation failure action   | None                                           |
+-----------------------------+------------------------------------------------+
...

At this point you can see that the policy for GPU 0 in group 2 has been set to: when the temperature exceeds 100 degrees, reset the GPU and run a medium validation.

8.3 modules: Viewing Loaded Modules

1
dcgmi modules -l

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
+-----------+--------------------+--------------------------------------------------+
| List Modules                                                                      |
| Status: Success                                                                   |
+===========+====================+==================================================+
| Module ID | Name               | State                                            |
+-----------+--------------------+--------------------------------------------------+
| 0         | Core               | Loaded                                           |
| 1         | NvSwitch           | Loaded                                           |
| 2         | VGPU               | Not loaded                                       |
| 3         | Introspection      | Not loaded                                       |
| 4         | Health             | Not loaded                                       |
| 5         | Policy             | Not loaded                                       |
| 6         | Config             | Not loaded                                       |
| 7         | Diag               | Not loaded                                       |
| 8         | Profiling          | Loaded                                           |
| 9         | SysMon             | Not loaded                                       |
+-----------+--------------------+--------------------------------------------------+

Module Name	Purpose
Core	GPU discovery and enumeration, basic API interface, foundation for metric data collection, dependency base for other modules
NvSwitch	Monitor NVSwitch status and performance, manage multi-GPU high-speed interconnect, DGX/HGX system support
VGPU	Monitor vGPU instance resources, manage virtual GPU configuration, collect performance data for virtualized environments
Introspection	DCGM internal state inspection, debugging and troubleshooting, module interaction state monitoring
Health	GPU temperature, power, and fan monitoring, hardware anomaly detection, health report alerts, predictive maintenance
Policy	GPU usage policy setting and enforcement, power and performance limit management, resource quota and access control
Config	GPU configuration parameter management, configuration change application and validation, configuration backup and restore, bulk configuration
Diag	GPU hardware test execution, memory and compute unit testing, diagnostic report generation, fault localization
Profiling	GPU performance metric collection, application performance analysis, PCIe bandwidth monitoring, NVIDIA event metrics
SysMon	CPU, memory, and network monitoring, system-GPU correlation monitoring, full-stack performance analysis, system metric collection

Some modules can be dynamically loaded and unloaded on demand. For example, running the dcgmi diag command automatically loads the Diag module.

8.4 introspect: Viewing DCGM Process Resource Usage

1
dcgmi introspect -s -H

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
+----------------------------------------------------------------------------+
| Introspection Information                                                  |
+============================================================================+

+----------------------------------------------------------------------------+
| Hostengine Process                                                         |
+============================================================================+
| Memory            | 75536.0 KB                                             |
| CPU Utilization   | 0.00 %                                                 |
+-------------------+--------------------------------------------------------+

You can see the memory usage and CPU utilization of the Hostengine process.