Crash工具介绍和常见命令使用_crash工具使用-程序员宅基地
技术标签： linux Linux debug
1. 介绍
本文主要介绍crash工具的使用以及常用的命令。crash工具，常用来分析内核的coredump以及应用的coredump，功能非常强大。
使用crash分析内核crash情况，需要准备以下内容：
- 内核crash时生成的coredump文件或者raw data
- 运行内核对应的符号表vmlinux
- crash 工具
2. crash基本使用
2.1 解释命令
使用crash工具解析内核crash产生的raw data，命令如下：
crash cluster-vmlinux ramdump@0x57400000
解析成功后，会进入crash工具cmdline模式，如下图所示：
使用crash工具解析内核crash产生的coredump文件，命令如下：
crash vmlinux vmcore
解析成功后，也会进入crash工具的cmdline模式
2.2 crash工具基本输出
KERNEL: cluster-vmlinux [TAINTED]
DUMPFILES: /var/tmp/ramdump_elf_Tp0C6d [temporary ELF header]
ramdump
CPUS: 6
DATE: Mon Apr 18 11:33:58 CST 2022 //panic时间
UPTIME: 01:28:40 //panic时，已运行时间
LOAD AVERAGE: 1.31, 1.32, 1.29
TASKS: 134 //panic时，进程总数
NODENAME: x9h_ms //硬件信息
RELEASE: 4.14.61 //内核版本信息
VERSION: #1 SMP PREEMPT Fri Apr 15 14:25:47 UTC 2022
MACHINE: aarch64 (unknown Mhz)
MEMORY: 544 MB //Linux系统内存数
PANIC: "sysrq: SysRq : Trigger a crash" //panic原因
PID: 2496 //panic进程号
COMMAND: "sh" //panic进程名词
TASK: ffff800023c5b800 [THREAD_INFO: ffff800023c5b800] //panic进程task和thread_info信息
CPU: 5 //系统cpu核数
STATE: TASK_RUNNING (SYSRQ) //panic时任务状态
2.2.1 crash常用命令
- help
该命令是查看crash工具的帮助信息，譬如支持的命令、具体某个命令的使用方法等。
crash> help
* extend log rd task
alias files mach repeat timer
ascii foreach mod runq tree
bpf fuser mount search union
bt gdb net set vm
btop help p sig vtop
dev ipcs ps struct waitq
dis irq pte swap whatis
eval kmem ptob sym wr
exit list ptov sys q
crash version: 8.0.0++ gdb version: 10.2
For help on any command above, enter "help <command>".
For help on input options, enter "help input".
For help on output options, enter "help output".
//查看某个命令使用方法
crash> help ps
NAME
ps - display process status information
SYNOPSIS
ps [-k|-u|-G|-y policy] [-s] [-p|-c|-t|-[l|m][-C cpu]|-a|-g|-r|-S|-A]
[pid | task | command] ...
DESCRIPTION
This command displays process status for selected, or all, processes
in the system. If no arguments are entered, the process data is
is displayed for all processes. Specific processes may be selected
by using the following identifier formats:
pid a process PID.
task a hexadecimal task_struct pointer.
command a command name. If a command name is made up of letters that
are all numerical values, precede the name string with a "\".
If the command string is enclosed within "'" characters, then
the encompassed string must be a POSIX extended regular expression
that will be used to match task names.
The process list may be further restricted by the following options:
-k restrict the output to only kernel threads.
-u restrict the output to only user tasks.
-G display only the thread group leader in a thread group.
-y policy restrict the output to tasks having a specified scheduling policy
expressed by its integer value or by its (case-insensitive) name;
multiple policies may be entered in a comma-separated list:
0 or NORMAL
1 or FIFO
2 or RR
3 or BATCH
4 or ISO
5 or IDLE
6 or DEADLINE
The process identifier types may be mixed. For each task, the following
items are displayed:
1. the process PID.
2. the parent process PID.
3. the CPU number that the task ran on last.
4. the task_struct address or the kernel stack pointer of the process.
(see -s option below)
5. the task state (RU, IN, UN, ZO, ST, TR, DE, SW, WA, PA, ID, NE).
6. the percentage of physical memory being used by this task.
7. the virtual address size of this task in kilobytes.
8. the resident set size of this task in kilobytes.
9. the command name.
The default output shows the task_struct address of each process under a
column titled "TASK". This can be changed to show the kernel stack
pointer under a column titled "KSTACKP".
-s replace the TASK column with the KSTACKP column.
On SMP machines, the active task on each CPU will be highlighted by an
angle bracket (">") preceding its information. If the crash variable
"offline" is set to "hide", the active task on an offline CPU will
be highlighted by a "-" preceding its information.
Alternatively, information regarding parent-child relationships,
per-task time usage data, argument/environment data, thread groups,
or resource limits may be displayed:
-p display the parental hierarchy of selected, or all, tasks.
-c display the children of selected, or all, tasks.
-t display the task run time, start time, and cumulative user
and system times.
-l display the task's last-run timestamp value, using either the
task_struct's last_run value, the task_struct's timestamp value
or the task_struct's sched_entity last_arrival value, whichever
applies, of selected, or all, tasks; the list is sorted with the
most recently-run task (with the largest timestamp) shown first,
followed by the task's current state.
-m similar to -l, but the timestamp value is translated into days,
hours, minutes, seconds, and milliseconds since the task was
last run on a cpu.
-C cpus only usable with the -l or -m options, dump the timestamp data
in per-cpu blocks, where the cpu[s] can be specified as "1,3,5",
"1-3", "1,3,5-7,10", "all", or "a" (shortcut for "all").
-a display the command line arguments and environment strings of
selected, or all, user-mode tasks.
-g display tasks by thread group, of selected, or all, tasks.
-r display resource limits (rlimits) of selected, or all, tasks.
-S display a summary consisting of the number of tasks in a task state.
-A display only the active task on each cpu.
EXAMPLES
Show the process status of all current tasks:
crash> ps
PID PPID CPU TASK ST %MEM VSZ RSS COMM
> 0 0 3 c024c000 RU 0.0 0 0 [swapper]
> 0 0 0 c0dce000 RU 0.0 0 0 [swapper]
0 0 1 c0fa8000 RU 0.0 0 0 [swapper]
> 0 0 2 c009a000 RU 0.0 0 0 [swapper]
1 0 1 c0098000 IN 0.0 1096 476 init
2 1 1 c0090000 IN 0.0 0 0 [kflushd]
3 1 1 c000e000 IN 0.0 0 0 [kpiod]
4 1 3 c000c000 IN 0.0 0 0 [kswapd]
5 1 1 c0008000 IN 0.0 0 0 [mdrecoveryd]
253 1 2 fbc4c000 IN 0.0 1088 376 portmap
- bt命令
该命令是查看进程栈信息的，可以通过pid/cpu等选项指定相关信息
crash> bt
PID: 2496 TASK: ffff800023c5b800 CPU: 5 COMMAND: "sh"
#0 [ffff000013573800] __crash_kexec at ffff00000817cee8
#1 [ffff000013573890] (null) at 108142390
#2 [ffff000013573920] psci_sys_reset at ffff000008965ae0
#3 [ffff000013573940] machine_restart at ffff000008085b94
#4 [ffff000013573960] emergency_restart at ffff0000080fddc4
#5 [ffff000013573970] panic at ffff0000080d8c04
#6 [ffff000013573a50] die at ffff00000808ae10
#7 [ffff000013573a90] __do_kernel_fault at ffff00000809feb8
#8 [ffff000013573ac0] do_page_fault at ffff00000809ffdc
#9 [ffff000013573b30] do_translation_fault at ffff0000080a0390
#10 [ffff000013573b40] do_mem_abort at ffff00000808130c
#11 [ffff000013573d20] el1_ia at ffff000008083050
PC: ffff0000086095b0 [sysrq_handle_crash+32]
LR: ffff00000860959c [sysrq_handle_crash+12]
SP: ffff000013573d30 PSTATE: 60400145
X29: ffff000013573d30 X28: ffff800023c5b800 X27: ffff000008cc1000
X26: 0000000000000040 X25: 0000000000000124 X24: 0000000000000000
X23: 0000000000000004 X22: ffff000009217000 X21: ffff000009217f10
X20: 0000000000000063 X19: ffff00000919e000 X18: 0000000000000010
X17: 000000000049b4d8 X16: ffff000008277b80 X15: ffffffffffffffff
X14: ffff0000892fc7a7 X13: ffff0000092fc7b5 X12: ffff00000919e000
X11: ffff00000917ae78 X10: ffff000008434f78 X9: 00000000ffffffd0
X8: 0000000000000015 X7: 6767697254203a20 X6: 00000000000004a3
X5: 0000000000000000 X4: 0000000000000000 X3: 0000000000000000
X2: ffff80002731efb8 X1: 0000000000000000 X0: 0000000000000001
#12 [ffff000013573d30] sysrq_handle_crash at ffff0000086095ac
#13 [ffff000013573d40] __handle_sysrq at ffff000008609b5c
#14 [ffff000013573d80] write_sysrq_trigger at ffff00000860a100
#15 [ffff000013573da0] proc_reg_write at ffff0000082ea22c
#16 [ffff000013573dc0] __vfs_write at ffff00000827760c
#17 [ffff000013573e40] vfs_write at ffff000008277900
#18 [ffff000013573e80] sys_write at ffff000008277bc4
#19 [ffff000013573ff0] el0_svc_naked at ffff000008083abc
PC: 0000ffff9e5aaeac LR: 0000000000408cc4 SP: 0000ffffeea390d0
X29: 0000ffffeea390d0 X28: 0000000000000000 X27: 0000000000000000
X26: 00000000071a3cc5 X25: 000000000049b000 X24: 00000000071a6372
X23: 0000000000000000 X22: 0000000000000001 X21: 00000000071a6370
X20: 0000000000000002 X19: 0000000000000001 X18: 00000000000002c8
X17: 000000000049b4d8 X16: 0000ffff9e5aae80 X15: 0000ffff9e4ebde0
X14: 0000ffff9e4f92c8 X13: 000000000000270f X12: 0101010101010101
X11: 0000000000000000 X10: 0101010101010101 X9: fffffffffffffff0
X8: 0000000000000040 X7: 7f7f7f7f7f7f7f7f X6: 0080000080808080
X5: 0000000000000000 X4: 00000000071a0063 X3: 0000ffff9e63f190
X2: 0000000000000002 X1: 00000000071a6370 X0: 0000000000000001
ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 80000000
- set
该命令获取crash的进程信息
crash> set
PID: 2496
COMMAND: "sh"
TASK: ffff800023c5b800 [THREAD_INFO: ffff800023c5b800]
CPU: 5
STATE: TASK_RUNNING (SYSRQ)
- sym
该命令查看符号和符号对应的地址信息，可以根据符号给出对应的地址，也可以根据给出的地址给出对应的符号
crash> sym proc_reg_write
ffff0000082ea1d0 (t) proc_reg_write /linux/fs/proc/inode.c: 224
crash> sym ffff0000082ea1d0
ffff0000082ea1d0 (t) proc_reg_write /linux/fs/proc/inode.c: 224
crash> sym __log_buf
ffff0000092ff020 (b) __log_buf
crash> sym ffff0000092ff020
ffff0000092ff020 (b) __log_buf
- ps
查看系统的进程信息，类似linux的ps
crash> ps
PID PPID CPU TASK ST %MEM VSZ RSS COMM
> 0 0 0 ffff000009186300 RU 0.0 0 0 [swapper/0]
> 0 0 1 ffff800025ba9c00 RU 0.0 0 0 [swapper/1]
> 0 0 2 ffff800025baaa00 RU 0.0 0 0 [swapper/2]
> 0 0 3 ffff800025bab800 RU 0.0 0 0 [swapper/3]
> 0 0 4 ffff800025bac600 RU 0.0 0 0 [swapper/4]
0 0 5 ffff800025bad400 RU 0.0 0 0 [swapper/5]
1 0 2 ffff800025b70000 IN 0.1 1824 1276 init
2 0 2 ffff800025b70e00 IN 0.0 0 0 [kthreadd]
4 2 0 ffff800025b72a00 ID 0.0 0 0 [kworker/0:0H]
6 2 0 ffff800025b74600 ID 0.0 0 0 [mm_percpu_wq]
7 2 0 ffff800025b75400 IN 0.0 0 0 [ksoftirqd/0]
8 2 5 ffff800025b76200 ID 0.0 0 0 [rcu_preempt]
9 2 1 ffff800025b77000 ID 0.0 0 0 [rcu_sched]
10 2 0 ffff800025ba8000 ID 0.0 0 0 [rcu_bh]
11 2 0 ffff800025ba8e00 IN 0.0 0 0 [migration/0]
12 2 0 ffff800025bae200 IN 0.0 0 0 [cpuhp/0]
13 2 1 ffff800025baf000 IN 0.0 0 0 [cpuhp/1]
14 2 1 ffff800025be8000 IN 0.0 0 0 [migration/1]
...
- vtop
根据一个虚拟地址转换成对应的物理地址
crash> vtop ffff800025065400
VIRTUAL PHYSICAL
ffff800025065400 65065400
PAGE DIRECTORY: ffff00000938b000
PGD: ffff00000938b800 => 673f8803
PUD: ffff8000273f8000 => 673f7803
PMD: ffff8000273f7940 => f8000065000f11
PAGE: 65000000 (2MB)
PTE PHYSICAL FLAGS
f8000065000f11 65000000 (VALID|SHARED|AF|NG|PXN|UXN|DIRTY)
PAGE PHYSICAL MAPPING INDEX CNT FLAGS
ffff7e0000941940 65065000 dead000000000400 0 0 fffc00000000000
- ptov
根据一个物理地址，转存成对应的内核态虚拟地址
crash> ptov 65065400
VIRTUAL PHYSICAL
ffff800025065400 65065400
- struct
查看结构体信息，如成员偏移，某个结构体变量内存中的值等
crash> struct thread_info -o
struct thread_info {
[0] unsigned long flags;
[8] mm_segment_t addr_limit;
[16] int preempt_count;
}
SIZE: 24
crash> struct thread_info -o -x
struct thread_info {
[0x0] unsigned long flags;
[0x8] mm_segment_t addr_limit;
[0x10] int preempt_count;
}
SIZE: 0x18
crash> struct thread_info 0xffff000009186300
struct thread_info {
flags = 32,
addr_limit = 281474976710655,
preempt_count = 65537
}
crash> task_struct.thread_info 0xffff000009186300 -p
thread_info = {
flags = 32,
addr_limit = 281474976710655,
preempt_count = 65537
},
- dis
反汇编，根据给出的PC地址，反编译出汇编源码，一般结合bt提供的栈信息和寄存器信息和反编译的汇编源码信息分析。
crash> bt
PID: 2496 TASK: ffff800023c5b800 CPU: 5 COMMAND: "sh"
#0 [ffff000013573800] __crash_kexec at ffff00000817cee8
#1 [ffff000013573890] (null) at 108142390
#2 [ffff000013573920] psci_sys_reset at ffff000008965ae0
#3 [ffff000013573940] machine_restart at ffff000008085b94
#4 [ffff000013573960] emergency_restart at ffff0000080fddc4
#5 [ffff000013573970] panic at ffff0000080d8c04
#6 [ffff000013573a50] die at ffff00000808ae10
#7 [ffff000013573a90] __do_kernel_fault at ffff00000809feb8
#8 [ffff000013573ac0] do_page_fault at ffff00000809ffdc
#9 [ffff000013573b30] do_translation_fault at ffff0000080a0390
#10 [ffff000013573b40] do_mem_abort at ffff00000808130c
#11 [ffff000013573d20] el1_ia at ffff000008083050
PC: ffff0000086095b0 [sysrq_handle_crash+32]
LR: ffff00000860959c [sysrq_handle_crash+12]
SP: ffff000013573d30 PSTATE: 60400145
X29: ffff000013573d30 X28: ffff800023c5b800 X27: ffff000008cc1000
X26: 0000000000000040 X25: 0000000000000124 X24: 0000000000000000
X23: 0000000000000004 X22: ffff000009217000 X21: ffff000009217f10
X20: 0000000000000063 X19: ffff00000919e000 X18: 0000000000000010
X17: 000000000049b4d8 X16: ffff000008277b80 X15: ffffffffffffffff
X14: ffff0000892fc7a7 X13: ffff0000092fc7b5 X12: ffff00000919e000
X11: ffff00000917ae78 X10: ffff000008434f78 X9: 00000000ffffffd0
X8: 0000000000000015 X7: 6767697254203a20 X6: 00000000000004a3
X5: 0000000000000000 X4: 0000000000000000 X3: 0000000000000000
X2: ffff80002731efb8 X1: 0000000000000000 X0: 0000000000000001
#12 [ffff000013573d30] sysrq_handle_crash at ffff0000086095ac
#13 [ffff000013573d40] __handle_sysrq at ffff000008609b5c
#14 [ffff000013573d80] write_sysrq_trigger at ffff00000860a100
#15 [ffff000013573da0] proc_reg_write at ffff0000082ea22c
#16 [ffff000013573dc0] __vfs_write at ffff00000827760c
#17 [ffff000013573e40] vfs_write at ffff000008277900
#18 [ffff000013573e80] sys_write at ffff000008277bc4
#19 [ffff000013573ff0] el0_svc_naked at ffff000008083abc
PC: 0000ffff9e5aaeac LR: 0000000000408cc4 SP: 0000ffffeea390d0
X29: 0000ffffeea390d0 X28: 0000000000000000 X27: 0000000000000000
X26: 00000000071a3cc5 X25: 000000000049b000 X24: 00000000071a6372
X23: 0000000000000000 X22: 0000000000000001 X21: 00000000071a6370
X20: 0000000000000002 X19: 0000000000000001 X18: 00000000000002c8
X17: 000000000049b4d8 X16: 0000ffff9e5aae80 X15: 0000ffff9e4ebde0
X14: 0000ffff9e4f92c8 X13: 000000000000270f X12: 0101010101010101
X11: 0000000000000000 X10: 0101010101010101 X9: fffffffffffffff0
X8: 0000000000000040 X7: 7f7f7f7f7f7f7f7f X6: 0080000080808080
X5: 0000000000000000 X4: 00000000071a0063 X3: 0000ffff9e63f190
X2: 0000000000000002 X1: 00000000071a6370 X0: 0000000000000001
ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 80000000
//查看栈数据
crash> bt -f 2
PID: 2 TASK: ffff800179d30000 CPU: 1 COMMAND: "kthreadd"
#0 [ffff00000a56bd90] __switch_to at ffff000008086690
ffff00000a56bd90: ffff00000a56bdb0 ffff000008ea5fe8
ffff00000a56bda0: ffff80017feae280 ffff800179d6e580
#1 [ffff00000a56bdb0] __schedule at ffff000008ea5fe4
ffff00000a56bdb0: ffff00000a56be40 ffff000008ea6708
ffff00000a56bdc0: ffff800179d30000 ffff000009a2bda0
ffff00000a56bdd0: ffff000009b8df60 ffff8000f4276600
ffff00000a56bde0: 0000000000000000 ffff000008108b68
ffff00000a56bdf0: 0000000000000001 ffff000008108000
ffff00000a56be00: 0000000000000000 0000000000000000
ffff00000a56be10: ffff000008109f54 ffff00000a56be60
ffff00000a56be20: ffff000008109e80 80838e611e66f600
ffff00000a56be30: ffff000000000004 80838e611e66f600
#2 [ffff00000a56be40] schedule at ffff000008ea6704
ffff00000a56be40: ffff00000a56be60 ffff000008109f54
ffff00000a56be50: ffff8000f4276628 ffff000008109e80
#3 [ffff00000a56be60] kthreadd at ffff000008109f50
//查看异常栈
crash> bt -e
PID: 1803 TASK: ffff8001794fab80 CPU: 5 COMMAND: "pvr_defer_free"
KERNEL-MODE EXCEPTION FRAME AT: ffff00000e7dbbc0
PC: ffff00000876d444 [DeviceMemSet+140]
LR: ffff0000087609c0 [_ZeroPageArray+120]
SP: ffff00000e7dbd00 PSTATE: 20c00145
X29: ffff00000e7dbd00 X28: ffff80015bc9fb00 X27: 00000000000186a0
X26: 0000000000000000 X25: 00e8000000000f0f X24: 0000000000000080
X23: ffff80012d0db000 X22: ffff00001fb7c000 X21: 0000000000000000
X20: 0000000000080000 X19: ffff00001fbaf000 X18: 0000ffffac000bcc
X17: 0000ffff97c22ec8 X16: ffff00000818ded0 X15: 0000ffffac000bc8
X14: 0140000000000000 X13: ffff00001fbfc000 X12: 0000000000000000
X11: 0000000000000000 X10: 0000000000000040 X9: 0040000000000041
X8: 0040000000000001 X7: 0000000000000001 X6: 000000017fffd7e8
X5: ffff8001398feb98 X4: ffff8001398feb98 X3: ffff00001fbfbfff
X2: 0000000000000000 X1: 0000000000000000 X0: ffff00001fbfc000
- irq
查看系统的中断信息
crash> irq
IRQ IRQ_DESC/_DATA IRQACTION NAME
0 (unused) (unused)
1 ffff800025807400 (unused)
2 ffff800025807800 (unused)
3 ffff800025807a00 (unused)
4 ffff800025807c00 ffff80002588df00 "arch_timer"
5 ffff800025807e00 (unused)
6 ffff800024c7e400 (unused)
7 ffff800024c7e600 ffff800024fcc100 "ttyS0"
8 ffff800024c7e800 (unused)
9 ffff800024c7ea00 ffff800024b10680 "30b00000.i2c"
10 ffff800024c7ec00 ffff800024efc200 "30b10000.i2c"
11 ffff800024c7ee00 ffff800024c72f00 "30b30000.i2c"
...
- files
查看进程打开的文件
crash> files
PID: 2496 TASK: ffff800023c5b800 CPU: 5 COMMAND: "sh"
ROOT: / CWD: /home/root
FD FILE DENTRY INODE TYPE PATH
0 ffff800023772500 ffff8000246089c0 ffff8000240b2f28 CHR /dev/ttyS0
1 ffff800024de9500 ffff8000239b29c0 ffff80002399e508 REG /proc/sysrq-trigger
2 ffff800023772500 ffff8000246089c0 ffff8000240b2f28 CHR /dev/ttyS0
10 ffff8000236a3b00 ffff8000245ff300 ffff800025095668 CHR /dev/tty
11 ffff800023772500 ffff8000246089c0 ffff8000240b2f28 CHR /dev/ttyS0
crash> foreach files -R /dev/console //结合foreach命令搜索打开/dev/console文件的进程
PID: 2331 TASK: ffff800024a1b800 CPU: 2 COMMAND: "messagecenter"
ROOT: / CWD: /
FD FILE DENTRY INODE TYPE PATH
1 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
2 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
PID: 2460 TASK: ffff800024f8aa00 CPU: 5 COMMAND: "adbd"
ROOT: / CWD: /
FD FILE DENTRY INODE TYPE PATH
1 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
2 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
PID: 2462 TASK: ffff80002421f000 CPU: 2 COMMAND: "usb ffs open"
ROOT: / CWD: /
FD FILE DENTRY INODE TYPE PATH
1 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
2 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
PID: 2463 TASK: ffff80002421aa00 CPU: 5 COMMAND: "server socket"
ROOT: / CWD: /
FD FILE DENTRY INODE TYPE PATH
1 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
2 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
PID: 2464 TASK: ffff800024a1aa00 CPU: 5 COMMAND: "->transport"
ROOT: / CWD: /
FD FILE DENTRY INODE TYPE PATH
1 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
2 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
PID: 2465 TASK: ffff800024a1c600 CPU: 5 COMMAND: "<-transport"
ROOT: / CWD: /
FD FILE DENTRY INODE TYPE PATH
1 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
2 ffff800022ec1100 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
PID: 2478 TASK: ffff800024a1d400 CPU: 1 COMMAND: "start_getty"
ROOT: / CWD: /
FD FILE DENTRY INODE TYPE PATH
0 ffff800023772c00 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
1 ffff800023772c00 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
2 ffff800023772c00 ffff8000245ff3c0 ffff800025095928 CHR /dev/console
- dev
查看系统注册的设备信息
crash> dev
CHRDEV NAME CDEV OPERATIONS
1 mem ffff8000253f5b00 memory_fops
2 pty ffff800024a6ed80 tty_fops
3 ttyp ffff8000249e3280 tty_fops
4 /dev/vc/0 ffff00000934b7a8 console_fops
4 tty ffff80002487e180 tty_fops
4 ttyS ffff800024f02e80 tty_fops
5 /dev/tty ffff00000934a2d0 tty_fops
5 /dev/console ffff00000934a338 console_fops
5 /dev/ptmx ffff00000934a4b0 ptmx_fops
...
BLKDEV NAME GENDISK OPERATIONS
259 blkext (none)
7 loop ffff800024e85000 lo_fops
8 sd (none)
9 md ffff800023db8800 md_fops
31 mtdblock (none)
...
- mount
查看系统挂载的文件系统
crash> mount
MOUNT SUPERBLK TYPE DEVNAME DIRNAME
ffff800025b4c000 ffff800025812000 rootfs rootfs /
ffff800023d0c000 ffff800023db9000 ext4 /dev/root /
ffff800023d0c1c0 ffff8000250b8000 devtmpfs devtmpfs /dev
ffff800023e0c000 ffff800025815000 proc proc /proc
ffff800023e0c1c0 ffff800023e60000 sysfs sysfs /sys
ffff800023e0c380 ffff800025817000 debugfs debugfs /sys/kernel/debug
ffff800023e0c540 ffff80002403f800 configfs configfs /sys/kernel/config
ffff800023e0c700 ffff800023e60800 tmpfs tmpfs /run
ffff800023e0c8c0 ffff800023e61000 tmpfs tmpfs /var/volatile
ffff8000250a8540 ffff8000234e4000 devpts devpts /dev/pts
ffff8000250a8700 ffff800023e61000 tmpfs tmpfs /var/lib
ffff800023e0ca80 ffff80002403f800 configfs none /config
ffff800023e0cc40 ffff800023e62000 functionfs adb /dev/usb-ffs/adb
ffff800023d0c380 ffff800024e19800 tracefs tracefs /sys/kernel/debug/tracing
ffff8000250a88c0 ffff8000234e6800 ext4 /dev/block/by-name/userdata /data
- kmem
查看内核态下内存的使用情况，如slab信息、vmalloc信息等
crash> kmem -i
PAGES TOTAL PERCENTAGE
TOTAL MEM 120690 471.4 MB ----
FREE 76395 298.4 MB 63% of TOTAL MEM
USED 44295 173 MB 36% of TOTAL MEM
SHARED 7202 28.1 MB 5% of TOTAL MEM
BUFFERS 649 2.5 MB 0% of TOTAL MEM
CACHED 10890 42.5 MB 9% of TOTAL MEM
SLAB 7869 30.7 MB 6% of TOTAL MEM
TOTAL HUGE 0 0 ----
HUGE FREE 0 0 0% of TOTAL HUGE
TOTAL SWAP 0 0 ----
SWAP USED 0 0 0% of TOTAL SWAP
SWAP FREE 0 0 0% of TOTAL SWAP
COMMIT LIMIT 60345 235.7 MB ----
COMMITTED 43807 171.1 MB 72% of TOTAL LIMIT
crash> kmem -s //查看kmalloc信息
CACHE OBJSIZE ALLOCATED TOTAL SLABS SSIZE NAME
ffff800024ada180 136 3052 3060 102 4k ext4_groupinfo_1k
ffff800023c96000 144 3 28 1 4k ext4_groupinfo_4k
ffff800024164480 1408 2 46 2 32k UDPv6
ffff800024164300 232 0 0 0 4k tw_sock_TCPv6
ffff800024164180 304 0 0 0 8k request_sock_TCPv6
ffff800024164000 2112 5 75 5 32k TCPv6
ffff8000241c7e00 224 233 882 49 4k nf_conntrack_expect
ffff8000241c7c80 256 0 0 0 8k nf_conntrack
ffff8000241c7b00 312 0 0 0 8k ashmem_area_cache
ffff8000241c7980 3312 0 0 0 32k kcopyd_job
ffff8000241c7800 120 0 0 0 4k dm_rq_target_io
ffff8000241c7680 72 0 0 0 4k isp1760_qtd
ffff8000241c7500 112 13 216 6 4k cfq_io_cq
ffff8000241c7380 240 14 102 6 4k cfq_queue
ffff8000241c7200 216 0 0 0 4k bsg_cmd
ffff8000241c7080 896 1 17 1 16k mqueue_inode_cache
ffff8000241c6f00 656 0 0 0 16k v9fs_inode_cache
...
crash> kmem -S //查看所有slab对象信息
CACHE OBJSIZE ALLOCATED TOTAL SLABS SSIZE NAME
ffff800024ada180 136 3052 3060 102 4k ext4_groupinfo_1k
CPU 0 KMEM_CACHE_CPU:
ffff7dffbfecbfb0
CPU 0 SLAB:
(empty)
CPU 0 PARTIAL:
(empty)
CPU 1 KMEM_CACHE_CPU:
ffff7dffbfee4fb0
CPU 1 SLAB:
SLAB MEMORY NODE TOTAL ALLOCATED FREE
ffff7e0000721f80 ffff80001c87e000 0 30 22 8
FREE / [ALLOCATED]
[ffff80001c87e000]
[ffff80001c87e088]
...
crash> kmem -p //查看page信息
PAGE PHYSICAL MAPPING INDEX CNT FLAGS
ffff7e0000000000 40000000 0 0 0 0
ffff7e0000000040 40001000 0 0 0 0
ffff7e0000000080 40002000 0 0 0 0
ffff7e00000000c0 40003000 0 0 0 0
ffff7e0000000100 40004000 0 0 0 0
ffff7e0000000140 40005000 0 0 0 0
...
- foreach
遍历系统上task的信息，如查看系统上处于UNINTERRUPT状态的task的栈信息
crash> foreach UN bt
PID: 561 TASK: ffff8000e3d33a00 CPU: 0 COMMAND: "kworker/u12:1"
#0 [ffff00002038b5f0] __switch_to at ffff000008086690
#1 [ffff00002038b610] __schedule at ffff000008ea5fe4
#2 [ffff00002038b6a0] schedule at ffff000008ea6704
#3 [ffff00002038b6c0] wait_transaction_locked at ffff0000083b5d8c
#4 [ffff00002038b730] add_transaction_credits at ffff0000083b602c
#5 [ffff00002038b7a0] start_this_handle at ffff0000083b624c
#6 [ffff00002038b840] jbd2__journal_start at ffff0000083b6a50
#7 [ffff00002038b8a0] __ext4_journal_start_sb at ffff0000083564d8
#8 [ffff00002038b8f0] ext4_writepages at ffff0000083760cc
#9 [ffff00002038ba40] do_writepages at ffff000008225400
#10 [ffff00002038bac0] __writeback_single_inode at ffff0000082ee774
#11 [ffff00002038bb10] writeback_sb_inodes at ffff0000082ef180
#12 [ffff00002038bc10] __writeback_inodes_wb at ffff0000082ef4b0
#13 [ffff00002038bc60] wb_writeback at ffff0000082ef828
#14 [ffff00002038bd10] wb_workfn at ffff0000082f0400
#15 [ffff00002038bdd0] process_one_work at ffff000008101950
#16 [ffff00002038be20] worker_thread at ffff000008101c6c
#17 [ffff00002038be70] kthread at ffff000008108c9c
PID: 814 TASK: ffff8001793f8000 CPU: 0 COMMAND: "saftey_heart_be"
#0 [ffff00000c053c10] __switch_to at ffff000008086690
#1 [ffff00000c053c30] __schedule at ffff000008ea5fe4
#2 [ffff00000c053cc0] schedule at ffff000008ea6704
#3 [ffff00000c053ce0] schedule_timeout at ffff000008eaa428
#4 [ffff00000c053d80] msleep at ffff000008178d64
#5 [ffff00000c053da0] saftey_hb_thread at ffff000008ad925c
#6 [ffff00000c053e70] kthread at ffff000008108c9c
PID: 1767 TASK: ffff800020776580 CPU: 0 COMMAND: "AVDemuxer::Read"
#0 [ffff000015dc3780] __switch_to at ffff000008086690
#1 [ffff000015dc37a0] __schedule at ffff000008ea5fe4
#2 [ffff000015dc3830] schedule at ffff000008ea6704
#3 [ffff000015dc3850] wait_transaction_locked at ffff0000083b5d8c
#4 [ffff000015dc38c0] add_transaction_credits at ffff0000083b602c
#5 [ffff000015dc3930] start_this_handle at ffff0000083b624c
#6 [ffff000015dc39d0] jbd2__journal_start at ffff0000083b6a50
#7 [ffff000015dc3a30] __ext4_journal_start_sb at ffff0000083564d8
#8 [ffff000015dc3a80] ext4_da_write_begin at ffff000008378448
#9 [ffff000015dc3c10] generic_perform_write at ffff00000821263c
#10 [ffff000015dc3ca0] __generic_file_write_iter at ffff000008215884
#11 [ffff000015dc3cf0] ext4_file_write_iter at ffff000008362760
#12 [ffff000015dc3d80] __vfs_write at ffff0000082b5180
#13 [ffff000015dc3e30] vfs_write at ffff0000082b53e0
#14 [ffff000015dc3e70] sys_write at ffff0000082b56f8
#15 [ffff000015dc3ff0] el0_svc_naked at ffff000008083af4
PC: 0000ffffa8eaee60 LR: 0000ffffa8eaee48 SP: 0000ffff93ffcdc0
X29: 0000ffff93ffcdc0 X28: 0000000000008000 X27: 0000000002faf080
X26: 0000000000000000 X25: 0000000000000000 X24: 0000000000001000
X23: 0000ffff84004de0 X22: 0000000000001000 X21: 0000ffff84004de0
X20: 0000000000001000 X19: 000000000000001f X18: 0000ffffa8f5ba70
X17: 0000ffffa8e50398 X16: 0000ffffa9b00420 X15: 0000074895555910
X14: 0000000000000000 X13: 52e6647d9bae90b9 X12: 936c08cc82e6992c
X11: 2de7fcffe5393767 X10: 41a928c963510c66 X9: 44377654e95c11b2
X8: 0000000000000040 X7: 042d1f3b698dddd1 X6: 0000000000000000
X5: 0000ffff93ffede0 X4: 00000000ffffffbb X3: 0000000000000000
X2: 0000000000001000 X1: 0000ffff84004de0 X0: 000000000000001f
ORIG_X0: 000000000000001f SYSCALLNO: 40 PSTATE: 80000000
...
- p
打印变量或者表达式的值
crash> p jiffies
jiffies = $1 = 4298283231
crash> p vm_event_states
PER-CPU DATA TYPE:
struct vm_event_state vm_event_states;
PER-CPU ADDRESSES:
[0]: ffff80017fe91ab8
[1]: ffff80017feaaab8
[2]: ffff80017fec3ab8
[3]: ffff80017fedcab8
[4]: ffff80017fef5ab8
[5]: ffff80017ff0eab8
//查看cpu1对应的变量的值
crash> p vm_event_states:1
per_cpu(vm_event_states, 1) = $3 = {
event = {318552, 739584, 0, 0, 558108, 2889268, 0, 0, 0, 0, 0, 0, 0, 4189877, 38713, 21935, 255, 2454551, 259, 0, 27971, 34455, 0, 63883, 0, 0, 0, 0, 62080, 233, 2, 1, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4576, 0, 0, 0, 46844, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 622, 34, 48, 0, 0, 0, 0, 0, 2119, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0}
}
//查看数组元素
crash> p &page_wait_table
$4 = (wait_queue_head_t (*)[256]) 0xffff000009a07d80 <page_wait_table>
crash> p page_wait_table[4]
$5 = {
lock = {
{
rlock = {
raw_lock = {
owner = 6378,
next = 6378
}
}
}
},
head = {
next = 0xffff000009a07de8 <page_wait_table+104>,
prev = 0xffff000009a07de8 <page_wait_table+104>
}
}
- list
遍历由指针或者链表连接起来的内容
//查看挂在panic_notifier_list的notify
crash> p &panic_notifier_list
$3 = (struct atomic_notifier_head *) 0xffff000009b8a670 <panic_notifier_list>
crash> struct atomic_notifier_head
struct atomic_notifier_head {
spinlock_t lock;
struct notifier_block *head;
}
SIZE: 16
crash> struct notifier_block
struct notifier_block {
notifier_fn_t notifier_call;
struct notifier_block *next;
int priority;
}
SIZE: 24
crash> list atomic_notifier_head.head 0xffff000009b8a670 -s notifier_block.next
ffff000009b8a670
next = 0xffff000009afaad0 <sdrv_wdt_panic_event_nb>,
ffff000009afaad0
next = 0xffff000009a3b540 <trace_panic_notifier>,
ffff000009a3b540
next = 0xffff000009a322a0 <rcu_panic_block>,
ffff000009a322a0
next = 0xffff000009a3ad50 <panic_block>,
ffff000009a3ad50
next = 0xffff000009aa5b80 <crashdump_panic_event_nb>,
ffff000009aa5b80
next = 0xffff000009a1b190 <kernel_offset_notifier>,
ffff000009a1b190
next = 0xffff000009a1b588 <cpu_hwcaps_notifier>,
ffff000009a1b588
next = 0xffff000009a1d3b0 <mem_limit_notifier>,
ffff000009a1d3b0
next = 0xffff000009afd460 <heartbeat_panic_nb>,
ffff000009afd460
next = 0x0,
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
智能推荐
什么是内部类？成员内部类、静态内部类、局部内部类和匿名内部类的区别及作用？_成员内部类和局部内部类的区别-程序员宅基地
文章浏览阅读3.4k次，点赞8次，收藏42次。一、什么是内部类？or 内部类的概念内部类是定义在另一个类中的类；下面类TestB是类TestA的内部类。即内部类对象引用了实例化该内部对象的外围类对象。public class TestA{ class TestB {}}二、 为什么需要内部类？or 内部类有什么作用？1、 内部类方法可以访问该类定义所在的作用域中的数据，包括私有数据。2、内部类可以对同一个包中的其他类隐藏起来。3、 当想要定义一个回调函数且不想编写大量代码时，使用匿名内部类比较便捷。三、 内部类的分类成员内部_成员内部类和局部内部类的区别
分布式系统_分布式系统运维工具-程序员宅基地
文章浏览阅读118次。分布式系统要求拆分分布式思想的实质搭配要求分布式系统要求按照某些特定的规则将项目进行拆分。如果将一个项目的所有模板功能都写到一起，当某个模块出现问题时将直接导致整个服务器出现问题。拆分按照业务拆分为不同的服务器，有效的降低系统架构的耦合性在业务拆分的基础上可按照代码层级进行拆分(view、controller、service、pojo)分布式思想的实质分布式思想的实质是为了系统的..._分布式系统运维工具
用Exce分析l数据极简入门_exce l趋势分析数据量-程序员宅基地
文章浏览阅读174次。1.数据源准备2.数据处理step1：数据表处理应用函数：①VLOOKUP函数; ② CONCATENATE函数终表：step2：数据透视表统计分析（1） 透视表汇总不同渠道用户数, 金额（2）透视表汇总不同日期购买用户数，金额（3）透视表汇总不同用户购买订单数，金额step3：讲第二步结果可视化, 比如, 柱形图（1）不同渠道用户数, 金额（2）不同日期..._exce l趋势分析数据量
宁盾堡垒机双因素认证方案_horizon宁盾双因素配置-程序员宅基地
文章浏览阅读3.3k次。堡垒机可以为企业实现服务器、网络设备、数据库、安全设备等的集中管控和安全可靠运行，帮助IT运维人员提高工作效率。通俗来说，就是用来控制哪些人可以登录哪些资产（事先防范和事中控制），以及录像记录登录资产后做了什么事情（事后溯源）。由于堡垒机内部保存着企业所有的设备资产和权限关系，是企业内部信息安全的重要一环。但目前出现的以下问题产生了很大安全隐患：密码设置过于简单，容易被暴力破解；为方便记忆，设置统一的密码，一旦单点被破，极易引发全面危机。在单一的静态密码验证机制下，登录密码是堡垒机安全的唯一_horizon宁盾双因素配置
谷歌浏览器安装（Win、Linux、离线安装）_chrome linux debian离线安装依赖-程序员宅基地
文章浏览阅读7.7k次，点赞4次，收藏16次。Chrome作为一款挺不错的浏览器，其有着诸多的优良特性，并且支持跨平台。其支持（Windows、Linux、Mac OS X、BSD、Android），在绝大多数情况下，其的安装都很简单，但有时会由于网络原因，无法安装，所以在这里总结下Chrome的安装。Windows下的安装：在线安装：离线安装：Linux下的安装：在线安装：离线安装：..._chrome linux debian离线安装依赖
烤仔TVの尚书房 | 逃离北上广？不如押宝越南“北上广”-程序员宅基地
文章浏览阅读153次。中国发达城市榜单每天都在刷新，但无非是北上广轮流坐庄。北京拥有最顶尖的文化资源，上海是“摩登”的国际化大都市，广州是活力四射的千年商都。GDP和发展潜力是衡量城市的数字指...
随便推点
java spark的使用和配置_使用java调用spark注册进去的程序-程序员宅基地
文章浏览阅读3.3k次。前言spark在java使用比较少，多是scala的用法，我这里介绍一下我在项目中使用的代码配置详细算法的使用请点击我主页列表查看版本jar版本说明spark3.0.1scala2.12这个版本注意和spark版本对应，只是为了引jar包springboot版本2.3.2.RELEASEmaven<!-- spark --> <dependency> <gro_使用java调用spark注册进去的程序
汽车零部件开发工具巨头V公司全套bootloader中UDS协议栈源代码，自己完成底层外设驱动开发后，集成即可使用_uds协议栈 源代码-程序员宅基地
文章浏览阅读4.8k次。汽车零部件开发工具巨头V公司全套bootloader中UDS协议栈源代码，自己完成底层外设驱动开发后，集成即可使用，代码精简高效，大厂出品有量产保证。:139800617636213023darcy169_uds协议栈 源代码
AUTOSAR基础篇之OS(下)_autosar 定义了 5 种多核支持类型-程序员宅基地
文章浏览阅读4.6k次，点赞20次，收藏148次。AUTOSAR基础篇之OS(下)前言首先，请问大家几个小小的问题，你清楚：你知道多核OS在什么场景下使用吗？多核系统OS又是如何协同启动或者关闭的呢？AUTOSAR OS存在哪些功能安全等方面的要求呢？多核OS之间的启动关闭与单核相比又存在哪些异同呢？。。。。。。今天，我们来一起探索并回答这些问题。为了便于大家理解，以下是本文的主题大纲：[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-JCXrdI0k-1636287756923)(https://gite_autosar 定义了 5 种多核支持类型
VS报错无法打开自己写的头文件_vs2013打不开自己定义的头文件-程序员宅基地
文章浏览阅读2.2k次，点赞6次，收藏14次。原因：自己写的头文件没有被加入到方案的包含目录中去，无法被检索到，也就无法打开。将自己写的头文件都放入header files。然后在VS界面上，右键方案名，点击属性。将自己头文件夹的目录添加进去。_vs2013打不开自己定义的头文件
【Redis】Redis基础命令集详解_redis命令-程序员宅基地
文章浏览阅读3.3w次，点赞80次，收藏342次。此时，可以将系统中所有用户的 Session 数据全部保存到 Redis 中，用户在提交新的请求后，系统先从Redis 中查找相应的Session 数据，如果存在，则再进行相关操作，否则跳转到登录页面。此时，可以将系统中所有用户的 Session 数据全部保存到 Redis 中，用户在提交新的请求后，系统先从Redis 中查找相应的Session 数据，如果存在，则再进行相关操作，否则跳转到登录页面。当数据量很大时，count 的数量的指定可能会不起作用，Redis 会自动调整每次的遍历数目。_redis命令
URP渲染管线简介-程序员宅基地
文章浏览阅读449次，点赞3次，收藏3次。URP的设计目标是在保持高性能的同时，提供更多的渲染功能和自定义选项。与普通项目相比，会多出Presets文件夹，里面包含着一些设置，包括本色，声音，法线，贴图等设置。全局只有主光源和附加光源，主光源只支持平行光，附加光源数量有限制，主光源和附加光源在一次Pass中可以一起着色。URP：全局只有主光源和附加光源，主光源只支持平行光，附加光源数量有限制，一次Pass可以计算多个光源。可编程渲染管线：渲染策略是可以供程序员定制的，可以定制的有：光照计算和光源，深度测试，摄像机光照烘焙，后期处理策略等等。_urp渲染管线