java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties

GitHub | borisclemencon | 5 months ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    IAM Role not taken into account

    GitHub | 5 months ago | borisclemencon
    java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties
  2. 0

    `aws_iam_role` not being used

    GitHub | 7 months ago | Gauravshah
    java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties (respectively).

    Root Cause Analysis

    1. java.lang.IllegalArgumentException

      AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties

      at com.databricks.spark.redshift.S3Credentials.initialize()
    2. com.databricks.spark
      AWSCredentialsUtils$$anonfun$load$1.apply
      1. com.databricks.spark.redshift.S3Credentials.initialize(S3Credentials.java:67)
      2. com.databricks.spark.redshift.AWSCredentialsUtils$.com$databricks$spark$redshift$AWSCredentialsUtils$$loadFromURI(AWSCredentialsUtils.scala:60)
      3. com.databricks.spark.redshift.AWSCredentialsUtils$$anonfun$load$1.apply(AWSCredentialsUtils.scala:48)
      4. com.databricks.spark.redshift.AWSCredentialsUtils$$anonfun$load$1.apply(AWSCredentialsUtils.scala:48)
      4 frames
    3. Scala
      Option.getOrElse
      1. scala.Option.getOrElse(Option.scala:121)
      1 frame
    4. com.databricks.spark
      RedshiftRelation.buildScan
      1. com.databricks.spark.redshift.AWSCredentialsUtils$.load(AWSCredentialsUtils.scala:48)
      2. com.databricks.spark.redshift.RedshiftRelation.buildScan(RedshiftRelation.scala:89)
      2 frames
    5. org.apache.spark
      DataSourceStrategy$$anonfun$pruneFilterProject$1.apply
      1. org.apache.spark.sql.execution.datasources.DataSourceStrategy$$anonfun$8.apply(DataSourceStrategy.scala:260)
      2. org.apache.spark.sql.execution.datasources.DataSourceStrategy$$anonfun$8.apply(DataSourceStrategy.scala:260)
      3. org.apache.spark.sql.execution.datasources.DataSourceStrategy$$anonfun$pruneFilterProject$1.apply(DataSourceStrategy.scala:303)
      3 frames