4 Nobi

尚未进行身份认证

我要认证

暂无相关简介

等级
TA的排名 2w+

oozie-远程debug参数说明

早期版本4.3.0以及之前版本1. oozie调试Spark应用程序在SparkOpts中添加:sparkOpts=--conf "spark.driver.extraJavaOptions=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,syspend=y,address=8000"2. oozie调试Mapper进行调度相应的Action...

2018-10-27 10:28:11

Cloudera Manager 安装提示: jdk x86_64 2000:1.6.0_31-fcs 没有安装源

手动安装对应版本的jdk1.6即可! 下载jdk1.6地址: 下载地址

2018-09-09 00:39:37

Cloudera Manager 安装提示: 卡在正在获取安装锁..

解决方案rm -rf /tmp/scm_prepare_node.*rm -rf /tmp/.scm_prepare_node.lock

2018-09-09 00:31:14

Clouder-Manager安装 JDBC driver cannot be found. Unable to find the JDBC database jar on host : slave1

解决方案 1. 将 mysql-connector-java-5.1.30-bin.jar放置在/usr/share/java 2. 务必将包重命名mysql-connector-java.jar

2018-09-04 23:44:41

Linux:解决虚拟机可以ping通,但是不能SSH远程登录的问题

问题分析首先,Linux可以互相ping通,说明Linux的网络配置没有问题,那么就专注解决SSH的问题。可能原因Linux防火墙没有关闭,导致SSH连接失败(关闭防火墙)LinuxSSH服务没有启动,查看是否启动SSH服务SSH服务启动后,仍然连接不上,需要确认是否是SSH服务得端口问题,核对连接端口( /etc/ssh/sshd_config)...

2018-09-04 12:02:02

Hadoop Yarn RM启动失败提示:transport error 202: bind failed:地址已占用

问题原因集群开启了远程Debug,当这个端口被占用时,出现此问题导致RM启动失败主要启动参数transport=dt_socket,server=y,suspend=n,address=*

2018-09-04 11:54:37

CDH 版本下载库

CDH 版本下载库https://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh_package_tarball_57.html

2018-09-03 15:43:21

Spark DataFrame:value $ is not a member of StringContext df.select($"name", $"age" + 1).show()

错误信息Error:(29, 15) value $ is not a member of StringContext df.select($"name", $"age" + 1).show()解决方案··· //在创建SparkSession之后 import sparkSession.implicits._ ···...

2018-08-31 14:13:42

Can't connect to X11 window server using 'localhost:12.0' as the valu e of the DISPLAY variable.

错误描述at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method) at sun.awt.X11GraphicsEnvironment.access$200(X11GraphicsEnvironment.java:65) at sun.awt.X11GraphicsEnvironment$1.run(...

2018-08-23 16:24:12

oozie on spark2:Failing Oozie Launcher, main() threw exception, null

问题详情iling Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, nulljava.lang.NullPointerException at scala.collection.mutable.ArrayOps$ofRef$.newBu...

2018-08-22 09:12:22

oozie on spark2:java.lang.IllegalArgumentException: Attempt to add jar multiple times to the distrib

错误信息详情2018-08-20 19:55:03,088 [uber-SubtaskRunner] ERROR org.apache.spark.SparkContext - Error initializing SparkContext.java.lang.IllegalArgumentException: Attempt to add (hdfs://hdh116:8020/us...

2018-08-22 09:07:12

oozie on spark2:基于Yarn运行Spark2时出现的错误说明(scala does not exist; make sure Spark is built.)

Oozie adds all of the jars in the Oozie Spark sharelib to the DistributedCache such that all jars will be present in the current working directory of the YARN container (as well as in the container cl...

2018-08-21 18:41:34

oozie on spark2:发布oozie任务时出现错误

错误信息variable [spark] cannot be resolved解决方案将变量名中不包含特殊字符

2018-08-21 09:38:25

oozie on spark2:部署Oozie4.3.0出现部署错误

错误详情Exception in thread "main" java.lang.NoClassDefFoundError: javax/servlet/Filter at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java...

2018-08-21 09:36:52

oozie on spark2:运行Oozie中自带的Yarn出现问题

运行Oozie中自带的Yarn出现问题:Error: Could not load YARN classes. This copy of Spark may not have been compiled with YARN support.Run with --help for usage help or --verbose for debug outputIntercepting...

2018-08-21 09:35:41

ZK-节点信息说明

节点存储内容说明null(节点中保存的数据信息)cZxid = 0x100000052(创建节点的事务ID)ctime = Fri Jun 22 17:50:30 CST 2018(节点创建时间)mZxid = 0x100000052(最后一次更新节点的事务ID)mtime = Fri Jun 22 17:50:30 CST 2018(节点更新时间)pZxid = 0x10000...

2018-08-11 14:20:14

HBase-错误记录

1. windows远程开发HBase提示:java.net.UnknownHostException: unknown host: master解决方案:C:\WINDOWS\system32\drivers\etc\hosts中添加IP1 hostName1IP2 hostName2IP3 hostName32. 在将HBase添加协处理器时,出现错误ERR...

2018-08-11 14:18:20

HBase-基础命令总结

1. HBase获取多行数据get 表名,行键,{COLUMN=>"列族名:列名",VERSIONS=>版本数}2. HBase修改VERSIONSalter '表名',NAME=>'',VERSION=>数字版本3. HBase删除一个列族//表下线disable 表明//删除列族alter '表名',{NAME=>'列...

2018-08-11 14:15:58

ES-基础知识总结

学习目标熟悉ES中index,shard,type,document,倒排索引等概念以及Client API。作用高度可扩展的开源全文搜索与分析引擎快速地、近实时的对大数据进行存储、搜索与分析用来支撑有负责的数据搜索需求的企业级应用一个分布式可扩展的实时搜索和分析引擎一切的设计都是为了提高搜索的性能(为了提高搜索能力,在插入或者更新方面可能比较慢)特点分布...

2018-08-11 14:14:03

ES-基础命令总结

1. 创建索引POST http://localhost:9200/movies创建成功提示:{ "acknowledged": true}2. 查看索引的状态//浏览器中执行http://ds0:9200/_cat/indices?v3. 添加索引信息(更新索引信息=》相同的索引,相同的类型,相同的ID)PUT http://ds0:...

2018-08-11 14:12:20

查看更多

勋章 我的勋章
  • 持之以恒
    持之以恒
    授予每个自然月内发布4篇或4篇以上原创或翻译IT博文的用户。不积跬步无以至千里,不积小流无以成江海,程序人生的精彩需要坚持不懈地积累!