Feb 10, 2019 — setAppName(s"BigDataETL - Check if file exists") val sc = new SparkContext(sparkConf) // Create FileSystem object from Hadoop Configuration ...
spark-check-if-file-exists
For a file in HDFS, you can use the hadoop way of doing this: val conf = sc.hadoopConfiguration val fs = org.apache.hadoop.fs. 939c2ea5af
spark check file exists, spark check if file exists s3, spark sql check if file exists, spark session check if file exists
Comments