0

From the Ambari GUI we can not start the Spark service. So we want to start it by command line as the following:

[spark@mas01 spark2]$ ./sbin/start-thriftserver.sh --master yarn-client --executor-memory 512m --hiveconf hive.server2.thrift.port=10015
starting org.apache.spark.sql.hive.thriftserver.HiveThriftServer2, logging to /var/log/spark2/spark-spark-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-master01.octopus9.com.out

But from the log we get the error:

Warning: Master yarn-client is deprecated since 2.0. Please use master "yarn" with specified deploy mode instead.

Please advice what is wrong with the syntax? 

[spark@mas01 spark2]$ more /var/log/spark2/spark-spark-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-master01.octopus9.com.out
Spark Command: /usr/jdk64/jdk1.8.0_112/bin/java -Dhdp.version=2.6.0.3-8 -cp /usr/hdp/2.6.0.3-8/spark2/conf/:/usr/hdp/2.6.0.3-8/spark2/jars/*:/usr/hdp/current/hadoop-client/conf/ -Xmx10000m org.apache.spark.deploy.SparkSubmit --master yarn-client --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --name Thrift JDBC/ODBC Server --executor-memory 512m spark-internal -- hiveconf hive.server2.thrift.port=10015
========================================
Warning: Master yarn-client is deprecated since 2.0. Please use master "yarn" with specified deploy mode instead.
U880D
  • 1,146
yael
  • 13,106

1 Answers1

0

Before Spark 2.0.0, it was possible to use yarn-client or yarn-cluster and which is now deprecated. When you use the deprecated master URLs, you will get the mentioned warning.

The only proper master URL is now yarn, e.g. ./bin/spark-submit --master yarn .... You can then use --deploy-mode to choose between Spark client (default) or Spark cluster modes.

U880D
  • 1,146