您当前的位置: 首页 >  ide

段智华

暂无认证

  • 0浏览

    0关注

    1232博文

    0收益

  • 0浏览

    0点赞

    0打赏

    0留言

私信
关注
热门博文

IDEA环境安装及运行

段智华 发布时间:2018-01-15 11:10:41 ,浏览量:0

IDEA环境安装及运行

(1)windows 10 + idea 运行spark 提示 Winutils异常:

解决办法:

1,建立目录 C:\hadoop\bin,将网上下载的winutils.exe文件放在此目录下。(网上提供的下载地址https://github.com/srccodes/hadoop-common-2.2.0-bin)

2,环境变量配置:HADOOP_HOME 配置为 C:\hadoop

3,注销或重启windows 10,Winutils问题解决。

18/01/15 10:04:17 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
	at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)
	at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)
	at org.apache.hadoop.util.Shell.(Shell.java:363)
	at org.apache.hadoop.util.StringUtils.(StringUtils.java:79)
	at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:104)
	at org.apache.hadoop.security.Groups.(Groups.java:86)
	at org.apache.hadoop.security.Groups.(Groups.java:66)
	at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280)
	at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271)
	at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:248)
	at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:763)
	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:748)
	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:621)
	at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2373)
	at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2373)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2373)
	at org.apache.spark.SparkContext.(SparkContext.scala:295)
	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313)
	at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
	at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
	at com.dt.spark.cores.DatasetOps$.main(DatasetOps.scala:27)
	at com.dt.spark.cores.DatasetOps.main(DatasetOps.scala)
(2) idea +git 

   在IDEA 中使用Git 可参考 https://www.cnblogs.com/zbw911/p/6206689.html

git下载提示

15:31	Couldn't save uncommitted changes.
		Tried to save uncommitted changes in stash before Update, but failed with an error.
		stash file://D:/WorkSpace/test1.0: unable to auto-detect email address (got 'lenovo@duanzhihua.(none)')
		Cannot save the current index state
解决:

1,进入git的目录 

C:\Users\lenovo>cd C:\git\bin

2,设置用户名 密码即可

C:\Users\lenovo>
C:\Users\lenovo>cd C:\git\bin

C:\git\bin>git config --global user.email "duanzhihua@189.cn"

C:\git\bin>git config --global user.name "duanzhihua"

C:\git\bin>

关注
打赏
1659361485
查看更多评论
立即登录/注册

微信扫码登录

0.0667s