HDFS的shell操作

格式：hadoop fs 具体命令 或者 hdfs dfs 具体命令

一、文件上传

1. -moveFromLocal 从本地剪切粘贴到HDFS

[jack@hadoop102 bin]$ hdfs dfs -moveFromLocal /opt/software/斗破苍穹.txt  /user/jack/input/story/doupo.txt

2. -copyFromLocal：从本地文件系统中拷贝文件到HDFS路径去

[jack@hadoop102 bin]$ hdfs dfs -copyFromLocal /opt/software/server.log  /user/jack/input/log/server1.log

3. -put：等同于copyFromLocal，生产环境更习惯用put

[jack@hadoop102 bin]$ hdfs dfs -put /opt/software/catalog.json  /user/jack/input/json/catalog.json

4. -appendToFile：追加一个文件到已经存在的文件末尾

[jack@hadoop102 bin]$ hadoop fs -put /opt/software/鬼吹灯上.txt /user/jack/input/story/gcd.txt
[jack@hadoop102 bin]$ hadoop fs -appendToFile /opt/software/鬼吹灯下.txt /user/jack/input/story/gcd.txt

二、文件下载

1. -copyToLocal：从HDFS拷贝到本地

[jack@hadoop102 bin]$ hdfs dfs -copyToLocal /user/jack/input/story/doupo.txt  /tmp/doupo.txt

2. -get：等同于copyToLocal，生产环境更习惯用get

[jack@hadoop102 bin]$ hdfs dfs -get /user/jack/input/story/gcd.txt  /tmp/gcd.txt

三、其他操作命令

1. -ls: 显示目录信息

[jack@hadoop102 bin]$ hdfs dfs -ls /user/jack/input/story/
Found 2 items
-rw-r--r--   3 jack supergroup     253829 2024-01-16 22:27 /user/jack/input/story/doupo.txt
-rw-r--r--   3 jack supergroup    2350218 2024-01-16 22:48 /user/jack/input/story/gcd.txt

2. -cat：显示文件内容

[jack@hadoop102 bin]$ hdfs dfs -cat /user/jack/input/story/doupo.txt
Found 2 items
-rw-r--r--   3 jack supergroup     253829 2024-01-16 22:27 /user/jack/input/story/doupo.txt
-rw-r--r--   3 jack supergroup    2350218 2024-01-16 22:48 /user/jack/input/story/gcd.txt

3. -chgrp、-chmod、-chown：和Linux文件系统中的用法一样，修改文件所属权限

[jack@hadoop102 bin]$ hdfs dfs -chmod 666 /user/jack/input/story/doupo.txt

4. -mkdir：创建路径

[jack@hadoop102 bin]$ hdfs dfs -mkdir /user/jack/input/story/

5. -cp：从HDFS的一个路径拷贝到HDFS的另一个路径

[jack@hadoop102 bin]$ hdfs dfs -cp /user/jack/input/story/doupo.txt  /user/jack/input/story/doupo1.txt

6. -mv：在HDFS目录中移动文件

[jack@hadoop102 bin]$ hdfs dfs -mv /user/jack/input/story/doupo1.txt   /user/jack/input/story/bak/doupo.bak

7. -tail：显示一个文件的末尾1kb的数据

[jack@hadoop102 bin]$ hdfs dfs -tail /user/jack/input/story/doupo.bak

8. -rm：删除文件或文件夹

[jack@hadoop102 bin]$ hdfs dfs -rm /user/jack/input/story/doupo.bak

9. -rm -r：递归删除目录及目录里面内容

[jack@hadoop102 bin]$ hdfs dfs -rm -r   /user/jack/input/story/bak
Deleted /user/jack/input/story/bak

10. -du统计文件夹的大小信息

## 第一个是总大小, 第二个是三个副本总大小  第三个是目录名
[jack@hadoop102 bin]$ hdfs dfs -du -h /
473.0 K  1.4 M   /story
3.3 M    15.4 M  /tmp
31.5 M   94.4 M  /user

11. -setrep：设置HDFS中文件的副本数量

[jack@hadoop102 bin]$  hdfs dfs -setrep 10   /user/jack/input/story/doupo.txt
Replication 10 set: /user/jack/input/story/doupo.txt