您现在的位置是:首页 > 数据库 > 大数据

大数据学习第(3)话-hdfs常用操作

yuyong2019-02-26大数据

简介环境介绍:操作系统:CentOS7 x64 *4(1台namenode,3台datanode)Hadoop版本:3.1.2Java版本:1.8.0 x64常用命令:执行“hdfs dfs”或者“hadoop fs”便可以弹出如下命令提示[root@nnode1

环境介绍:

操作系统:CentOS7 x64 *4(1台namenode,3台datanode)

Hadoop版本:3.1.2

Java版本:1.8.0  x64

常用命令:

参考文档:hadoop.apache.org/docs/r3.1.2/hadoop-project-dist/hadoop-common/FileSystemShell.html

执行“hdfs dfs”或者“hadoop fs”便可以弹出如下命令提示

[root@nnode101 ~]# hdfs dfs
Usage: hadoop fs [generic options]
        [-appendToFile <localsrc> ... <dst>]
        [-cat [-ignoreCrc] <src> ...]
        [-checksum <src> ...]
        [-chgrp [-R] GROUP PATH...]
        [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
        [-chown [-R] [OWNER][:[GROUP]] PATH...]
        [-copyFromLocal [-f] [-p] [-l] [-d] [-t <thread count>] <localsrc> ... <dst>]
        [-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
        [-count [-q] [-h] [-v] [-t [<storage type>]] [-u] [-x] [-e] <path> ...]
        [-cp [-f] [-p | -p[topax]] [-d] <src> ... <dst>]
        [-createSnapshot <snapshotDir> [<snapshotName>]]
        [-deleteSnapshot <snapshotDir> <snapshotName>]
        [-df [-h] [<path> ...]]
        [-du [-s] [-h] [-v] [-x] <path> ...]
        [-expunge]
        [-find <path> ... <expression> ...]
        [-get [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
        [-getfacl [-R] <path>]
        [-getfattr [-R] {-n name | -d} [-e en] <path>]
        [-getmerge [-nl] [-skip-empty-file] <src> <localdst>]
        [-head <file>]
        [-help [cmd ...]]
        [-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [-e] [<path> ...]]
        [-mkdir [-p] <path> ...]
        [-moveFromLocal <localsrc> ... <dst>]
        [-moveToLocal <src> <localdst>]
        [-mv <src> ... <dst>]
        [-put [-f] [-p] [-l] [-d] <localsrc> ... <dst>]
        [-renameSnapshot <snapshotDir> <oldName> <newName>]
        [-rm [-f] [-r|-R] [-skipTrash] [-safely] <src> ...]
        [-rmdir [--ignore-fail-on-non-empty] <dir> ...]
        [-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
        [-setfattr {-n name [-v value] | -x name} <path>]
        [-setrep [-R] [-w] <rep> <path> ...]
        [-stat [format] <path> ...]
        [-tail [-f] <file>]
        [-test -[defsz] <path>]
        [-text [-ignoreCrc] <src> ...]
        [-touch [-a] [-m] [-t TIMESTAMP ] [-c] <path> ...]
        [-touchz <path> ...]
        [-truncate [-w] <length> <path> ...]
        [-usage [cmd ...]]
Generic options supported are:
-conf <configuration file>        specify an application configuration file
-D <property=value>               define a value for a given property
-fs <file:///|hdfs://namenode:port> specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.
-jt <local|resourcemanager:port>  specify a ResourceManager
-files <file1,...>                specify a comma-separated list of files to be copied to the map reduce cluster
-libjars <jar1,...>               specify a comma-separated list of jar files to be included in the classpath
-archives <archive1,...>          specify a comma-separated list of archives to be unarchived on the compute machines
The general command line syntax is:
command [genericOptions] [commandOptions]


命令选项含义示例
[-appendToFile <localsrc> ...   <dst>]从本地文件系统追加到目标文件系统,本地文件系统可以多个hdfs dfs -appendToFile append_text.txt append_text.txt /temp.sh
[-cat [-ignoreCrc] <src> ...]查看指定文件内容,可以同时查看多个文件hdfs dfs -cat /temp.sh  或者  hdfs dfs -cat /user/root/input/*
[-checksum <src> ...]返回文件的校验信息hdfs dfs -checksum input/*
[-chgrp [-R] GROUP PATH...]更改文件的属组,-R为递归修改hdfs dfs -chgrp root /temp.sh
[-chmod [-R] <MODE[,MODE]... |   OCTALMODE> PATH...]更改文件的权限,-R为递归修改hdfs dfs -chmod 777 input/core-site.xml
[-chown [-R] [OWNER][:[GROUP]] PATH...]更改文件的所有者,操作用户必须是超级用户,-R为递归修改hdfs dfs -chown root:root /temp.sh
[-copyFromLocal [-f] [-p] [-l] [-d] [-t   <thread count>] <localsrc> ... <dst>]复制本地文件到目标文件系统,-p保留权限等相关信息,-f如果存在则覆盖,-l延迟持久化,-d直接创建而跳过创建后缀为._COPYING_的临时文件hdfs dfs -copyFromLocal -p -f jdk-8u201-linux-x64.tar.gz /
[-copyToLocal [-f] [-p] [-ignoreCrc]   [-crc] <src> ... <localdst>]复制HDFS文件系统中的文件到本地文件系统,-p保留权限等相关信息,-f如果存在则覆盖hdfs dfs -copyToLocal -p -f /jdk-8u201-linux-x64.tar.gz ./
[-count [-q] [-h] [-v] [-t [<storage   type>]] [-u] [-x] [-e] <path> ...]统计文件或目录的目录数,大小,所在路径,-h人类可读格式显示,-v显示标题行hdfs dfs -count -h -v /user/root/input
[-cp [-f] [-p | -p[topax]] [-d]   <src> ... <dst>]将文件从源复制到目标,若源为多个则目标为目录,-f覆盖,-p保持属性hdfs dfs -cp /temp.sh input/cp_temp.sh
[-createSnapshot <snapshotDir>   [<snapshotName>]]创建快照
[-deleteSnapshot <snapshotDir>   <snapshotName>]删除快照
[-df [-h] [<path> ...]]显示磁盘空间使用情况,-h人类可读格式显示hdfs dfs -df -h /
[-du [-s] [-h] [-v] [-x] <path>   ...]显示指定文件或指定目录或指定目录下的文件的大小,-h友好显示,-v标题hdfs dfs -du -h -v /jdk-8u201-linux-x64.tar.gz /user/root/input
[-expunge]

[-find <path> ...   <expression> ...]查找与指定表达式匹配的所有文件,-iname不区分大小写,-name区分大小写,-print0不换行以空格分隔,-print换行显示hdfs dfs -find / -iname *XmL* -print0
[-get [-f] [-p] [-ignoreCrc] [-crc]   <src> ... <localdst>]将文件复制到本地文件系统,-f存在则覆盖,-p保留相关属性信息hdfs dfs -get -f -p /jdk-8u201-linux-x64.tar.gz ./
[-getfacl [-R] <path>]显示文件或目录的访问控制列表,-R递归列出所有文件和目录hdfs dfs -getfacl -R /jdk-8u201-linux-x64.tar.gz
[-getfattr [-R] {-n name | -d} [-e en]   <path>]

[-getmerge [-nl] [-skip-empty-file]   <src> <localdst>]

[-head <file>]

[-help [cmd ...]]

[-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S]   [-r] [-u] [-e] [<path> ...]]

[-mkdir [-p] <path> ...]

[-moveFromLocal <localsrc> ...   <dst>]

[-moveToLocal <src>   <localdst>]

[-mv <src> ... <dst>]

[-put [-f] [-p] [-l] [-d]   <localsrc> ... <dst>]

[-renameSnapshot <snapshotDir>   <oldName> <newName>]

[-rm [-f] [-r|-R] [-skipTrash] [-safely]   <src> ...]

[-rmdir [--ignore-fail-on-non-empty]   <dir> ...]

[-setfacl [-R] [{-b|-k} {-m|-x   <acl_spec>} <path>]|[--set <acl_spec> <path>]]

[-setfattr {-n name [-v value] | -x name}   <path>]

[-setrep [-R] [-w] <rep>   <path> ...]

[-stat [format] <path> ...]

[-tail [-f] <file>]

[-test -[defsz] <path>]

[-text [-ignoreCrc] <src> ...]

[-touch [-a] [-m] [-t TIMESTAMP ] [-c]   <path> ...]

[-touchz <path> ...]

[-truncate [-w] <length>   <path> ...]

[-usage [cmd ...]]


很赞哦! ()

文章排行

本栏推荐

栏目更新