Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions dev/deps/spark-deps-hadoop-3-hive-2.3
Original file line number Diff line number Diff line change
Expand Up @@ -68,16 +68,16 @@ gcs-connector/hadoop3-2.2.31/shaded/gcs-connector-hadoop3-2.2.31-shaded.jar
gmetric4j/1.0.10//gmetric4j-1.0.10.jar
gson/2.13.2//gson-2.13.2.jar
guava/33.4.8-jre//guava-33.4.8-jre.jar
hadoop-aliyun/3.4.2//hadoop-aliyun-3.4.2.jar
hadoop-annotations/3.4.2//hadoop-annotations-3.4.2.jar
hadoop-aws/3.4.2//hadoop-aws-3.4.2.jar
hadoop-azure-datalake/3.4.2//hadoop-azure-datalake-3.4.2.jar
hadoop-azure/3.4.2//hadoop-azure-3.4.2.jar
hadoop-client-api/3.4.2//hadoop-client-api-3.4.2.jar
hadoop-client-runtime/3.4.2//hadoop-client-runtime-3.4.2.jar
hadoop-cloud-storage/3.4.2//hadoop-cloud-storage-3.4.2.jar
hadoop-huaweicloud/3.4.2//hadoop-huaweicloud-3.4.2.jar
hadoop-shaded-guava/1.4.0//hadoop-shaded-guava-1.4.0.jar
hadoop-aliyun/3.4.3//hadoop-aliyun-3.4.3.jar
hadoop-annotations/3.4.3//hadoop-annotations-3.4.3.jar
hadoop-aws/3.4.3//hadoop-aws-3.4.3.jar
hadoop-azure-datalake/3.4.3//hadoop-azure-datalake-3.4.3.jar
hadoop-azure/3.4.3//hadoop-azure-3.4.3.jar
hadoop-client-api/3.4.3//hadoop-client-api-3.4.3.jar
hadoop-client-runtime/3.4.3//hadoop-client-runtime-3.4.3.jar
hadoop-cloud-storage/3.4.3//hadoop-cloud-storage-3.4.3.jar
hadoop-huaweicloud/3.4.3//hadoop-huaweicloud-3.4.3.jar
hadoop-shaded-guava/1.5.0//hadoop-shaded-guava-1.5.0.jar
hive-beeline/2.3.10//hive-beeline-2.3.10.jar
hive-cli/2.3.10//hive-cli-2.3.10.jar
hive-common/2.3.10//hive-common-2.3.10.jar
Expand Down
2 changes: 1 addition & 1 deletion docs/building-spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ You can enable the `yarn` profile and specify the exact version of Hadoop to com

Example:

./build/mvn -Pyarn -Dhadoop.version=3.4.1 -DskipTests clean package
./build/mvn -Pyarn -Dhadoop.version=3.4.3 -DskipTests clean package

## Building With Hive and JDBC Support

Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@
<slf4j.version>2.0.17</slf4j.version>
<log4j.version>2.25.3</log4j.version>
<!-- make sure to update IsolatedClientLoader whenever this version is changed -->
<hadoop.version>3.4.2</hadoop.version>
<hadoop.version>3.4.3</hadoop.version>
<!-- SPARK-41247: When updating `protobuf.version`, also need to update `protoVersion` in `SparkBuild.scala` -->
<protobuf.version>4.33.5</protobuf.version>
<protoc-jar-maven-plugin.version>3.11.4</protoc-jar-maven-plugin.version>
Expand Down
4 changes: 2 additions & 2 deletions resource-managers/kubernetes/integration-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,8 +136,8 @@ properties to Maven. For example:

mvn integration-test -am -pl :spark-kubernetes-integration-tests_2.13 \
-Pkubernetes -Pkubernetes-integration-tests \
-Phadoop-3 -Dhadoop.version=3.4.0 \
-Dspark.kubernetes.test.sparkTgz=spark-4.1.0-SNAPSHOT-bin-example.tgz \
-Phadoop-3 -Dhadoop.version=3.4.3 \
-Dspark.kubernetes.test.sparkTgz=spark-4.2.0-SNAPSHOT-bin-example.tgz \
-Dspark.kubernetes.test.imageTag=sometag \
-Dspark.kubernetes.test.imageRepo=docker.io/somerepo \
-Dspark.kubernetes.test.namespace=spark-int-tests \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ private[hive] object IsolatedClientLoader extends Logging {
case e: RuntimeException if e.getMessage.contains("hadoop") =>
// If the error message contains hadoop, it is probably because the hadoop
// version cannot be resolved.
val fallbackVersion = "3.4.2"
val fallbackVersion = "3.4.3"
logWarning(log"Failed to resolve Hadoop artifacts for the version " +
log"${MDC(HADOOP_VERSION, hadoopVersion)}. We will change the hadoop version from " +
log"${MDC(HADOOP_VERSION, hadoopVersion)} to " +
Expand Down