Eyeglass Access Zone Failover and Dual Delegation
This technical note covers Dual Delegation, answers key questions, how to set it up and how it works, with DNS.
- DNS Forwarding is untested and not recommended. Name server delegation is the recommended method to configure your DNS servers.
- Do not disable recursive queries on your DNS server. This is untested and not recommended.
Let's review Access Zone failover with Eyeglass at a high level before explaining how “Dual Delegation” works. It involves SmartConnect zones applied to IP pools failing over. This means creating an alias on the target Isilon that existed on the source cluster IP pool. Once this is completed, DNS delegation records are based on NS or name server records that point at the Subnet Service IP servicing the IP pool involved in the failover.
The IP Pools are updated to forward SmartConnect zone lookups to the newly active clusters subnet Service ip with the newly created IP alias on the IP pool.
Skip the reading and watch the video 5M on how to setup Dual Delegation with Microsoft DNS (other DNS vendors work as well, the concept is the same).
Continue reading below for details and how it works.
Eyeglass automates this process during failover BUT the source cluster SmartConnect Zone is renamed (Preserved for failback operations), and leaves a simple breadcrumb zone that remains that is prefixed with igls-original-whatever-the-zone-name-was. This rename operation has a second benefit, in that the source cluster Service IP will no longer answer any queries that are sent to the source cluster Subnet Service IP.
(see diagram below) DNS resolution BEFORE failover where typically on only a single NS record points at Primary cluster.
(see diagram below) DNS resolution AFTER manually updating the NS record to point at the Service IP of the secondary cluster.
- Can two NS records be added to the delegation to use Primary and Secondary Subnet Service IP ahead of time to simplify failover and remove the DNS manual step post failover?
- Answer: Yes, this is possible and removes yet another step in the failover when using Access Zone Failover
- Can dual delegation be used without Eyeglass?
- Answer: no without Eyeglass, this configuration can not be used. This is because the Primary cluster SmartConnect Zone needs to be manually removed or renamed, Eyeglass turns this into an Atomic automated process during failover when the Secondary cluster file system is made writeable (the only time you need this functionality)
- Will Dual Delegation support SMB and NFS failovers?
- Answer: Yes, both protocols benefit from this feature with SMB handling this better since retrying failed mounts requires clicking on the drive letter or accessing the UNC again to look up the name to ip address. NFS clients will require unmount and remount since they mount an ip address after name resolution has completed, even if used in the /etc/fstab file with an FQDN. Script Engine can now be focused entirely on host side automation without needing and DNS updates
How to setup the dual delegation?
Simple, delegate the SmartConnect Zone with two NS records #1 to primary cluster and #2 the Secondary cluster SSIP that answers DNS queries for the IP pool.
Let's review how this works. The following two diagrams show the DNS setup before and after failover with Eyeglass.
How does this work with DNS?
- The DNS server can issue queries for SmartConnect Zone userdata.ad1.test to either Name server record.
- If a query is sent to the Secondary Cluster before failover the Isilon answers the query code 5 or Refused.
NOTE: apply igls-original-<production cluster smartconnect name>, dual delegation requires a SmartConnect name to exist on the target IP pool. We recommend the syntax above. Also NOTE detection of the igls-original prefix on an Access Zone pool will update the DR dashboard with FAILEDOVER state.
- This tells the DNS server to re-issue to the second name server record to satisfy the query.
- Since the Primary cluster is configured for this SmartConnect Zone, it will answer the query from one of IP addresses in the IP pool as expected.
- The DNS server returns the IP address provided to the client that issued the query.
- Same as above except Eyeglass has disabled the Primary cluster SmartConnect Zone name with prefix igls-original-xxxx, and the Primary cluster will respond with DNS return code 5 Refused
- DNS server re-issues the query to the Secondary cluster SSIP to get the query answered.
- The above all assumes a TTL of 0 on NS records to avoid caching name to ip addresses which works with the exception of Linux mount commands.
- If a real-DR event occurs versus a controlled failover where both clusters are reachable, the last step would be to ensure the Primary cluster is not ip reachable (if partial disaster), and prevent this cluster from coming up again post DR Event, so that it does not answer DNS queries again or simple remove the entry from DNS.. This is standard practice but mentioned for completeness.
Let's review the wireshark traces below of a failover from a DNS view of a Linux Client.
- Linux Client 172.31.1.101 issues query to userdata.ad1.test
- DNS server source sends query to Primary cluster SSIP (no longer the active cluster)
- Primary Cluster (ip 172.31.1.200) with igl-original renamed Smartconnect Zone answers the query from DNS server (172.16.80.6)
- DNS Server re-issues query to 2nd NS record in this case the Secondary cluster SSIP
- Secondary cluster responds with new IP address from the target failover over IP pool mapped by Eyeglass for failover
- Client mount can now succeed using new ip address on the Secondary cluster IP pool to mount shares or exports
DNS Return codes
DNS Return Message
DNS Response Code
DNS Query completed successfully
DNS Query Format Error
Server failed to complete the DNS request
Domain name does not exist.
Function not implemented
The server refused to answer for the query
Name that should not exist, does exist
RRset that should not exist, does exist
Server not authoritative for the zone
Name not in zone
Copyright 2017 Superna LLC
Copyright Superna LLC