杰霸霸博客

主题

Yarn常用命令 ​

1. 查看任务信息 ​

1.1 列出所有Application ​

格式： yarn application -list

sh
[jack@hadoop102 ~]$ yarn application -list
2024-01-22 14:57:28,227 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at hadoop103/192.168.101.103:8032
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0
                Application-Id        Application-Name        Application-Type          User         Queue                 State           Final-State           Progress                           Tracking-URL

1.2 过滤任务信息 ​

格式：yarn application -list -appStates
根据Application状态过滤任务信息, 任务所有状态：ALL、NEW、NEW_SAVING、SUBMITTED、ACCEPTED、RUNNING、FINISHED、FAILED、KILLED)

sh
[jack@hadoop102 ~]$ yarn application -list -appStates FINISHED
2024-01-22 15:02:42,357 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at hadoop103/192.168.101.103:8032
Total number of applications (application-types: [], states: [FINISHED] and tags: []):5
                Application-Id        Application-Name        Application-Type          User         Queue                 State           Final-State           Progress                           Tracking-URL
application_1705399039702_0001              word count               MAPREDUCE          jack       default              FINISHED             SUCCEEDED               100%    http://hadoop102:19888/jobhistory/job/job_1705399039702_0001
application_1705399039702_0003              word count               MAPREDUCE          jack       default              FINISHED             SUCCEEDED               100%    http://hadoop102:19888/jobhistory/job/job_1705399039702_0003
application_1705399039702_0002              word count               MAPREDUCE          jack       default              FINISHED             SUCCEEDED               100%    http://hadoop102:19888/jobhistory/job/job_1705399039702_0002
application_1705399039702_0005         QuasiMonteCarlo               MAPREDUCE          jack       default              FINISHED             SUCCEEDED               100%    http://hadoop102:19888/jobhistory/job/job_1705399039702_0005
application_1705399039702_0004              word count               MAPREDUCE          jack       default              FINISHED             SUCCEEDED               100%    http://hadoop102:19888/jobhistory/job/job_1705399039702_0004

2. kill掉Application ​

格式：yarn application -kill <ApplicationId>

sh
[jack@hadoop102 ~]$ yarn application -kill application_1612577921195_0001
2024-01-22 15:03:44,657 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.101.103:8032
Application application_1612577921195_0001 has already finished

3. yarn logs查看日志 ​

格式：yarn logs -applicationId <ApplicationId>

3.1 查询Application日志： ​

sh
[jack@hadoop102 ~]$ yarn logs -applicationId application_1705399039702_0003

3.2 查询Container日志 ​

格式：yarn logs -applicationId <ApplicationId> -containerId <ContainerId>

sh
[jack@hadoop102 ~]$ yarn logs -applicationId application_1612577921195_0001 -containerId container_1612577921195_0001_01_000001

4. 查看尝试运行的任务 ​

4.1 列出所有Application尝试的列表 ​

格式：yarn applicationattempt -list <ApplicationId>

sh
[jack@hadoop102 ~]$ yarn applicationattempt -list application_1612577921195_0001
2021-02-06 10:26:54,195 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of application attempts :1
         ApplicationAttempt-Id	               State	                    AM-Container-Id	                       Tracking-URL
appattempt_1612577921195_0001_000001	            FINISHED	container_1612577921195_0001_01_000001	http://hadoop103:8088/proxy/application_1612577921195_0001/

4.2 打印ApplicationAttemp状态 ​

格式： yarn applicationattempt -status <ApplicationAttemptId>

sh
[jack@hadoop102 ~]$ yarn applicationattempt -status appattempt_1612577921195_0001_000001
2021-02-06 10:27:55,896 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Application Attempt Report : 
	ApplicationAttempt-Id : appattempt_1612577921195_0001_000001
	State : FINISHED
	AMContainer : container_1612577921195_0001_01_000001
	Tracking-URL : http://hadoop103:8088/proxy/application_1612577921195_0001/
	RPC Port : 34756
	AM Host : hadoop104
	Diagnostics :

5. 查看容器 ​

5.1 列出所有Container ​

格式： yarn container -list <ApplicationAttemptId>

sh
[jack@hadoop102 ~]$ yarn container -list appattempt_1612577921195_0001_000001
2021-02-06 10:28:41,396 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total number of containers :0
                  Container-Id	          Start Time	         Finish Time	               State	                Host	   Node Http Address

5.2 打印Container状态 ​

格式： yarn container -status <ContainerId> [jack@hadoop102 ~]$ yarn container -status container_1612577921195_0001_01_000001 2021-02-06 10:29:58,554 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032 Container with id 'container_1612577921195_0001_01_000001' doesn't exist in RM or Timeline Server.

:::tip
只有在任务跑的途中才能看到container的状态信息
:::

## 6. 查看节点状态
### 6.1 列出所有节点
格式： `yarn node -list -all`
```sh
[jack@hadoop102 ~]$ yarn node -list -all
2021-02-06 10:31:36,962 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Total Nodes:3
         Node-Id	     Node-State	Node-Http-Address	Number-of-Running-Containers
 hadoop103:38168	        RUNNING	   hadoop103:8042	                           0
 hadoop102:42012	        RUNNING	   hadoop102:8042	                           0
 hadoop104:39702	        RUNNING	   hadoop104:8042	                           0

7. 刷新配置 ​

重新加载队列配置：yarn rmadmin -refreshQueues

sh
[jack@hadoop102 ~]$ yarn rmadmin -refreshQueues
2021-02-06 10:32:03,331 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8033

8. 查看队列 ​

格式： yarn queue -status <QueueName> 打印队列信息

sh
[jack@hadoop102 ~]$ yarn queue -status default
2021-02-06 10:32:33,403 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.10.103:8032
Queue Information : 
Queue Name : default
	State : RUNNING
	Capacity : 100.0%
	Current Capacity : .0%
	Maximum Capacity : 100.0%
	Default Node Label expression : <DEFAULT_PARTITION>
	Accessible Node Labels : *
	Preemption : disabled
	Intra-queue Preemption : disabled