org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to yarn/localhost@LOCALREALM

Apache's JIRA Issue Tracker | Roman Shaposhnik | 4 years ago
  1. 0

    Here's what I'm observing on a fully distributed cluster deployed via Bigtop from the RC0 2.0.3-alpha tarball: {noformat} 528077-oozie-tucu-W@mr-node] Error starting action [mr-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to yarn/localhost@LOCALREALM at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.<init>(AbstractDelegationTokenIdentifier.java:68) at org.apache.hadoop.mapreduce.v2.api.MRDelegationTokenIdentifier.<init>(MRDelegationTokenIdentifier.java:51) at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getDelegationToken(HistoryClientService.java:336) at org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getDelegationToken(MRClientProtocolPBServiceImpl.java:210) at org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:240) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729) Caused by: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to yarn/localhost@LOCALREALM at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:378) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.<init>(AbstractDelegationTokenIdentifier.java:66) ... 12 more ] {noformat} This is submitting a mapreduce job via Oozie 3.3.1. The reason I think this is a Hadoop issue rather than the oozie one is because when I hack /etc/krb5.conf to be: {noformat} [libdefaults] ticket_lifetime = 600 default_realm = LOCALHOST default_tkt_enctypes = des3-hmac-sha1 des-cbc-crc default_tgs_enctypes = des3-hmac-sha1 des-cbc-crc [realms] LOCALHOST = { kdc = localhost:88 default_domain = .local } [domain_realm] .local = LOCALHOST [logging] kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmin.log default = FILE:/var/log/krb5lib.log {noformat} The issue goes away. Now, once again -- the kerberos auth is NOT configured for Hadoop, hence it should NOT pay attention to /etc/krb5.conf to begin with.

    Apache's JIRA Issue Tracker | 4 years ago | Roman Shaposhnik
    org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to yarn/localhost@LOCALREALM
  2. 0

    This might not be a bug. Here is the description. Any workarounds are appreciated. I am only able to execute hadoop commands using principals which are in the default realm. hadoop.security.auth_to_local seems to be ignored. Attached is a log of everything done. Here is overview of the configuration and some troubleshooting tests: # created and tested a principal using the KDC instead of AD and confirmed all OK hadoop org.apache.hadoop.security.HadoopKerberosName george@EC2.INTERNAL Name: george@EC2.INTERNAL to george # fails to use with principal from AD, seems to ignore rules in hadoop.security.auth_to_local hadoop org.apache.hadoop.security.HadoopKerberosName george@CLOUDSECURE.LOCAL Exception in thread "main" org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to george@CLOUDSECURE.LOCAL at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:378) at org.apache.hadoop.security.HadoopKerberosName.main(HadoopKerberosName.java:74) # note: ip-10-151-51-135.ec2.internal has Win 2008 R2 + AD DS with 1 forest, and defines all user accounts used for authentication /etc/krb5.conf [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] default_realm = EC2.INTERNAL dns_lookup_realm = false dns_lookup_kdc = false max_life = 1d max_renewable_life = 7d ticket_lifetime = 24h renew_lifetime = 7d forwardable = true default_tgs_enctypes = aes256-cts aes128-cts arcfour-hmac des3-hmac-sha1 des-hmac-sha1 des-cbc-md5 des-cbc-crc default_tkt_enctypes = aes256-cts aes128-cts arcfour-hmac des3-hmac-sha1 des-hmac-sha1 des-cbc-md5 des-cbc-crc [realms] EC2.INTERNAL = { kdc = ip-10-191-70-81.ec2.internal admin_server = ip-10-191-70-81.ec2.internal default_domain = EC2.INTERNAL } CLOUDSECURE.LOCAL = { kdc = ip-10-151-51-135.ec2.internal:88 admin_server = ip-10-151-51-135.ec2.internal:749 default_domain = EC2.INTERNAL } [domain_realm] .ec2.internal = EC2.INTERNAL ec2.internal = EC2.INTERNAL cat /etc/hadoop/conf.cloudera.hdfs1/core-site.xml <?xml version="1.0" encoding="UTF-8"?> <!--Autogenerated by Cloudera CM on 2013-10-06T10:16:50.792Z--> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://ip-10-191-70-81.ec2.internal:8020</value> </property> <property> <name>fs.trash.interval</name> <value>1</value> </property> <property> <name>hadoop.security.authentication</name> <value>kerberos</value> </property> <property> <name>hadoop.rpc.protection</name> <value>authentication</value> </property> <property> <name>hadoop.security.auth_to_local</name> <value>RULE:[1:$1@$0](.*@\QEC2.INTERNAL\E$)s/@\QEC2.INTERNAL\E$// RULE:[2:$1@$0](.*@\QEC2.INTERNAL\E$)s/@\QEC2.INTERNAL\E$// RULE:[1:$1@$0](.*@\QCLOUDSECURE.LOCAL\E$)s/@\QCLOUDSECURE.LOCAL\E$// RULE:[2:$1@$0](.*@\QCLOUDSECURE.LOCAL\E$)s/@\QCLOUDSECURE.LOCAL\E$// DEFAULT</value> </property> </configuration>

    Cloudera Open Source | 3 years ago | Daniel Rule
    org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to george@CLOUDSECURE.LOCAL
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    run dmlc yarn error, "failure to login"

    GitHub | 1 year ago | robbine
    java.io.IOException: failure to login
  5. 0

    Kerberos Authentication Error - When loading Hadoop Config Files from SharedPath

    Stack Overflow | 5 months ago | Padmanabhan Vijendran
    java.io.IOException: Login failure for name@XX.XX.COM from keytab \\NASdrive\name.keytab: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM

    1 unregistered visitors
    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule

      No rules applied to yarn/localhost@LOCALREALM

      at org.apache.hadoop.security.authentication.util.KerberosName.getShortName()
    2. Apache Hadoop Auth
      KerberosName.getShortName
      1. org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:378)
      1 frame
    3. Hadoop
      AbstractDelegationTokenIdentifier.<init>
      1. org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.<init>(AbstractDelegationTokenIdentifier.java:66)
      1 frame
    4. hadoop-mapreduce-client-common
      MRDelegationTokenIdentifier.<init>
      1. org.apache.hadoop.mapreduce.v2.api.MRDelegationTokenIdentifier.<init>(MRDelegationTokenIdentifier.java:51)
      1 frame
    5. hadoop-mapreduce-client-hs
      HistoryClientService$HSClientProtocolHandler.getDelegationToken
      1. org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getDelegationToken(HistoryClientService.java:336)
      1 frame
    6. hadoop-mapreduce-client-common
      MRClientProtocolPBServiceImpl.getDelegationToken
      1. org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getDelegationToken(MRClientProtocolPBServiceImpl.java:210)
      1 frame
    7. hadoop-yarn-api
      MRClientProtocol$MRClientProtocolService$2.callBlockingMethod
      1. org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:240)
      1 frame
    8. Hadoop
      Server$Handler$1.run
      1. org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
      2. org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
      3. org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735)
      4. org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731)
      4 frames
    9. Java RT
      Subject.doAs
      1. java.security.AccessController.doPrivileged(Native Method)
      2. javax.security.auth.Subject.doAs(Subject.java:396)
      2 frames
    10. Hadoop
      Server$Handler.run
      1. org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
      2. org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729)
      2 frames