博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
带你认识spark安装包的目录结构
阅读量:6382 次
发布时间:2019-06-23

本文共 4557 字,大约阅读时间需要 15 分钟。

 

 

 

 

 

  其实啊,将spark的压缩包解压好,就已经是spark的最简易安装了。

其实啊,想说的是,只要将spark这压缩包解压好,就已经是spark的最简易安装了。

 

[spark@sparksinglenode spark-1.6.1-bin-hadoop2.6]$ lltotal 1408  drwxr-xr-x. 2 spark spark    4096 Feb 27  2016 bin             (可执行)-rw-r--r--. 1 spark spark 1343562 Feb 27  2016 CHANGES.txtdrwxr-xr-x. 2 spark spark    4096 Feb 27  2016 conf            (配置文件)drwxr-xr-x. 3 spark spark    4096 Feb 27  2016 data       (例子里用到的一些数据) drwxr-xr-x. 3 spark spark 4096 Feb 27 2016 ec2 drwxr-xr-x. 3 spark spark 4096 Feb 27 2016 examples     (自带的例子一些源代码)  drwxr-xr-x. 2 spark spark 4096 Feb 27 2016 lib        (jar包) -rw-r--r--. 1 spark spark 17352 Feb 27 2016 LICENSE drwxr-xr-x. 2 spark spark 4096 Feb 27 2016 licenses -rw-r--r--. 1 spark spark 23529 Feb 27 2016 NOTICE drwxr-xr-x. 6 spark spark 4096 Feb 27 2016 python drwxr-xr-x. 3 spark spark 4096 Feb 27 2016 R         -rw-r--r--. 1 spark spark 3359 Feb 27 2016 README.md     (包含一些入门的spark说明) -rw-r--r--. 1 spark spark 120 Feb 27 2016 RELEASE drwxr-xr-x. 2 spark spark 4096 Feb 27 2016 sbin        (集群启停,因为spark有自带的集群环境) [spark@sparksinglenode spark-1.6.1-bin-hadoop2.6]$ pwd /usr/local/spark/spark-1.6.1-bin-hadoop2.6 [spark@sparksinglenode spark-1.6.1-bin-hadoop2.6]$

 

 

 

[spark@sparksinglenode bin]$ pwd/usr/local/spark/spark-1.6.1-bin-hadoop2.6/bin[spark@sparksinglenode bin]$ lltotal 92-rwxr-xr-x. 1 spark spark 1099 Feb 27  2016 beeline-rw-r--r--. 1 spark spark  932 Feb 27  2016 beeline.cmd-rw-r--r--. 1 spark spark 1910 Feb 27  2016 load-spark-env.cmd-rw-r--r--. 1 spark spark 2143 Feb 27  2016 load-spark-env.sh-rwxr-xr-x. 1 spark spark 3459 Feb 27  2016 pyspark-rw-r--r--. 1 spark spark 1486 Feb 27  2016 pyspark2.cmd-rw-r--r--. 1 spark spark 1000 Feb 27  2016 pyspark.cmd-rwxr-xr-x. 1 spark spark 2384 Feb 27  2016 run-example-rw-r--r--. 1 spark spark 2682 Feb 27  2016 run-example2.cmd-rw-r--r--. 1 spark spark 1012 Feb 27  2016 run-example.cmd-rwxr-xr-x. 1 spark spark 2858 Feb 27  2016 spark-class-rw-r--r--. 1 spark spark 2365 Feb 27  2016 spark-class2.cmd-rw-r--r--. 1 spark spark 1010 Feb 27  2016 spark-class.cmd-rwxr-xr-x. 1 spark spark 1049 Feb 27  2016 sparkR-rw-r--r--. 1 spark spark 1010 Feb 27  2016 sparkR2.cmd-rw-r--r--. 1 spark spark  998 Feb 27  2016 sparkR.cmd-rwxr-xr-x. 1 spark spark 3026 Feb 27  2016 spark-shell-rw-r--r--. 1 spark spark 1528 Feb 27  2016 spark-shell2.cmd-rw-r--r--. 1 spark spark 1008 Feb 27  2016 spark-shell.cmd-rwxr-xr-x. 1 spark spark 1075 Feb 27  2016 spark-sql-rwxr-xr-x. 1 spark spark 1050 Feb 27  2016 spark-submit-rw-r--r--. 1 spark spark 1126 Feb 27  2016 spark-submit2.cmd-rw-r--r--. 1 spark spark 1010 Feb 27  2016 spark-submit.cmd[spark@sparksinglenode bin]$

 

 

  拿bin目录而言。比如

[spark@sparksinglenode bin]$ pwd/usr/local/spark/spark-1.6.1-bin-hadoop2.6/bin[spark@sparksinglenode bin]$ ./spark-shell log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.Using Spark's repl log4j profile: org/apache/spark/log4j-defaults-repl.propertiesTo adjust logging level use sc.setLogLevel("INFO")Welcome to      ____              __     / __/__  ___ _____/ /__    _\ \/ _ \/ _ `/ __/  '_/   /___/ .__/\_,_/_/ /_/\_\   version 1.6.1      /_/Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60)Type in expressions to have them evaluated.Type :help for more information.Spark context available as sc.17/03/24 04:37:19 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)17/03/24 04:37:21 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)17/03/24 04:37:36 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.017/03/24 04:37:36 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException17/03/24 04:37:43 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)17/03/24 04:37:43 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)17/03/24 04:37:53 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.017/03/24 04:37:53 WARN ObjectStore: Failed to get database default, returning NoSuchObjectExceptionSQL context available as sqlContext.scala>

 

 

 

 

  拿example而言,比如

 

   例子,(官网给的)

http://spark.apache.org/docs/latest/quick-start.html

 

 

 

 

 

 

 

 

 

转载地址:http://kwzha.baihongyu.com/

你可能感兴趣的文章
redis数据结构存储SDS设计细节(redis的设计与实现笔记)
查看>>
数学之美系列二十四 -- 谈谈动态规划
查看>>
【内存溢出】Maven编译时内存溢出的问题解决方式
查看>>
【C++注意事项】1 数据类型及类型转换
查看>>
重建二叉树
查看>>
【Android】17.1 Bound Services基本概念
查看>>
让vc程序不显示任务栏图标
查看>>
cocos2d-x CCCallFuncN中node CCCallFuncND中data
查看>>
[Luogu P2973&BZOJ 1778][USACO10HOL]赶小猪DOtP(高斯消元+期望)
查看>>
深入浅出REST
查看>>
window.location.reload(false);window.location.reload(true);history.Go(0)区别
查看>>
[速记]关于字符串数组+字符串常量+结束符号'\0'
查看>>
[php审计实战篇]BlueCms v1.6 Union注入
查看>>
【Excle数据透视表】如何在数据透视表顶部显示列总计数据
查看>>
SpringMVC,Mybatis,FreeMarker连接mycat示例(一)
查看>>
[NOIP2003普及组]麦森数(快速幂+高精度)
查看>>
sqlzoo需要知道的那些事
查看>>
git for windows配置SSH key
查看>>
互联网创业三件事:钱、人和项目
查看>>
十亿美金公司不常有 一亿美金公司花常开 | 雷锋网
查看>>