Preparing your Clusters for Eyeglass Assisted Access Zone Failover
The following steps described in this section are required to prepare your system for the Eyeglass Assisted Access Zone Failover:
This is required to avoid Eyeglass requiring direct access Active Directory to synchronize the Service Principal Names (SPN) for production or DR clusters computer accounts.
- Service principal names are used by Kerberos authentication and machine accounts and an New SPN name pair is created each time a new SmartConnect Zone Alias is created.
- Mapping of SmartConnect Zones between source and target cluster.
This is required so that Eyeglass can create SmartConnect Zone names and aliases on your DR cluster automatically in the event of a DR failover.
- Update Isilon sudoer file for Eyeglass Service Account.
This is required when the Eyeglass Service Account is being used to execute CLI commands that require root privileges (see the following section for detail).
Update Isilon sudoer file for Eyeglass Service Account
Eyeglass Access Zone Failover requires some CLI commands that must run with root level access. Many customers also run the cluster in STIG or compliance mode for Smartlock WORM features. Root user account is not allowed to login and run commands. The “SPN machine account maintenance before and after cluster failover” command requires elevated permission to allow this user permissions across the cluster nodes. See “Eyeglass Service account guide for minimum permissions” for details on how to add sudo privileges to the Eyeglass cluster service account.
Active Directory Machine Account Service Principal Name (SPN) Delegation
What is an SPN?
It’s used in Kerberos authentication from clients to network services for file serving. It's formed from SmartConnect Zones and has two forms: the NTLM netbios name and Kerberos name URL format.
When a client connects to \\data.example.com\sharename, the SPN for this authentication request to active directory uses the SPN name to authenticate. Kerberos is the default SPN request for authentication and it uses the URL based request to the domain.
What’s the risk if I don’t fix SPN’s?
Without SPN values set on the cluster machine accounts, Kerberos authentication will fail, but many Windows clients will fall back on NTLM authentication automatically (NTLM fallback can be disabled in the domain for higher level of security). NTLM is a legacy authentication protocol and considered less secure than Kerberos.
The Eyeglass solution aims at removing manual steps wherever possible. All prerequisites ensure manual steps are not required during a DR event. The Eyeglass solution also has a goal to remove dependencies between groups within IT, to reduce the potential for communication issues impacting DR failover.
SPN Delegation is a one time setup setup, that achieves simplified DR automation, and is required to use Eyeglass Access Zone failover feature. SPNs related to Source Cluster Zone and SmartConnect Zone AD providers will be deleted to avoid SPN collision in AD. During normal operating conditions, Eyeglass will audit SPN’s on source and destination clusters to insure they are correct and will remediate prior to any failover.
The steps outlined in “How to - Delegation of Cluster Machine Accounts with Active Directory” are required for each cluster machine account, for each AD provider that is added to each cluster. Example four different AD providers for different domains will require four delegations to be created, as per below, to each machine account name. Typically the cluster name is used when the cluster joins an active directory.
This one time setup avoids the requirement for failover operations to require ADS administrative permissions to successfully failover, and have SPN source and destination cluster values managed by Eyeglass. This reduce the risk of SPN authentication failures and ensures proper cluster self management of SPN fails required for proper Kerberos authentication for SmartConnect zones and aliases.
Note: Superna Eyeglass only manages SPN related to HOST. SPNs related to HDFS or NFS are not updated and will need to be manually repaired post failover.
Configure Eyeglass Subnet IP Pool Mapping Hints
This section covers why they are configured and how to configure mappings between IP pools for failover. IP pools serve data from SmartConnect names, the IP pool used to serve the name on failover is predetermined using ip pool mapping hints.
Zone Aliases for Failover Overview
Access Zone failover depends on dual DNS delegation to ensure no steps are required in DNS during a failover. The target cluster Subnet IP Pools require a SmartConnect Zone name set in the Onefs UI (must be in the UI and not an alias).
This SmartConnect name is not mounted or accessed on the DR cluster but it is required to configure the 2nd name server record in DNS to be setup.
The diagram below explains how we recommend SmartConnect Zone names to be entered to simplify the failover of a SmartConnect Zone name.
We recommend entering the source IP pool SmartConnect Zone name entered into the OneFS UI on the target mapped IP pool by applying a prefix of “igls-original-<source cluster SmartConnect zone name>
NOTE: The target ip pool MUST have a SmartConnect value. The recommended value is as shown above or dual delegation responses will not function as expected. Blank or no SmartConnect name is NOT supported (no validation in DR Dashboard for checking if target cluster SmartConnect name is set correctly).
This simplifies failover visually in onefs UI and will rename on failover without an extra alias being created for failover purposes. It swaps the name from one side to the other during failover.
When and how to NOT failover an IP pool SmartConnect name using hint: igls-ignore
The hint applied to an IP pool tells Eyeglass to not process this SmartConnect name and Aliases found on this IP pool.
Syntax of Igls-ignore:
This mapping hint can also be made unique using igls-ignore-xxxx where -xxx can be unique value to self document the purpose of the ignore. Examples below.
- Igls-ignore-repl (documents ignore on SmartConnect ip pool used for SyncIQ)
- Igls-ignore-dfsprod (documents prod hint for DFS pool used for DFS clients)
- Igls-ignore-mgmtclst1 (documents management FQDN pool used to manage the cluster)
When to apply ignore hints:
- For SyncIQ IP pools that are used for target host.
- For DFS IP pools so that no DNS updates are done for DFS target folders, also avoids SPN updates needed.
- For IP used for cluster management, and when Clusters are added to Eyeglass with this FQDN, it's required to apply an ignore hint so that Eyeglass will not lose access to the cluster during failover or failback. This is also validated in the Zone Readiness screen and checked to make sure the FQDN used for cluster add has an igls-ignore hint applied. This is a blocking condition for failover.
How to create Mapping Hints for IP pools between source and target clusters following best practise naming convention
Subnet:Pool Failover mapping between the Source and Target Cluster is done to ensure that data is accessed from the correct SmartConnect Zone IP and node pool on the Target cluster. Mapping of IP address Pools should be completed after installation and will be audited by Eyeglass as part of the Failover Readiness validation. This is done using a SmartConnect Zone alias.
The hint alias on the IP Pool is of the form igls-xxx, where “xxx” can be any string. We recommend numbers to keep it simple. Example; igls-01-pool-name-prod is self explanatory name and the DR mapping would be igls-01-pool-name-dr.
NOTE: We also recommend this syntax to avoid SPN collision. Eyeglass does not inject these hints but an admin could run ISI commands and inject them, no harm if they are present in AD computer account. For example “igls-01-prod” and “igls-01-DR” are matching hints since only “igls-xx” needs to match. Therefore the syntax form of “igls-xx-some-unique-string” with the trailing string “some-unique-string” allow them to be made unique and still match.
Eyeglass requires that the user decide which network pools are partnered during failover. Create the identical alias hint on the source network pools and their target network pools.
- A hint is a pre-fixed zone alias that instructs Eyeglass which source cluster network pool should failover to a specific target SmartConnect subnet pool. Eyeglass will detect when hints are missing and raises an alarm to correct it.
- An ignore hint is used to identify the SmartConnect Zone(s) used for SyncIQ replication. It is Isilon best practice to have a dedicated SmartConnect Zone for this purpose, and avoid using this zone name to mount data with clients. During failover there is no need to failover the SmartConnect Zone used for SyncIQ. Eyeglass needs to know which zone name should be ignored during failover and readiness job assessment of Access Zones and SmartConnect Zones.
To add the mapping alias to the Isilon cluster, ssh to the cluster and login as root, execute the following command: (note subnet and pool names are case sensitive)
- get list of pools “isi network list pools -v”
- isi network modify pool --name=<subnet:poolname> --add-zone-alias=<hint>
In the example below, we will execute the command on the source and target cluster to map pools to each other:
OneFS 7 Example igls hints for user data
- Prod Cluster isi network modify pool --name=subnet0:exampleProd --add-zone-aliases igls-01-prod
- DR Cluster isi network modify pool --name=subnet0:exampleDR --add-zone-aliases igls-01-dr
OneFS 8 Example igls hints for user data
Data Pool mapping example - Prod
Prod cluster data access IGLS hint applied to an Access Zone IP pool example. NOTE: hint used to match is igls-marketing-marketingprod where marketingprod is used to identify the cluster the hint is applied. The marketingprod is not used to match the pools.
Data Pool mapping example - DR
DR cluster data access IGLS hint applied to an Access Zone IP pool example. NOTE: hint used to match is igls-marketing-marketingdr where marketingdr is used to identify the cluster the hint is applied. The marketingdr is not used to match the pools
IGLS examples for ignore option on IP Pools
In the example below, we will execute the command on the source and target cluster to pools that will be ignored for failover and are dedicated to SyncIQ replication or for DFS dedicated IP pools in the Access Zone :
OneFS 7 Example igls hints for ignore SyncIQ pool
- Prod Cluster isi network modify pool --name=subnet0:siqProd --add-zone-aliases igls-ignore
- DR Cluster isi network modify pool --name=subnet0:siqDR --add-zone-aliases igls-ignore
Example: Mapping with hints configured correctly will display the mapping
OneFS 8 Example igls hints for ignore SyncIQ pool
SyncIQ Ignore hint replication Pool.
This Pool is used by SyncIQ for replication (restrict source and or target host for replication. The igls-ignore- is used to ignore the pool the prod8 makes the hint unique and identifies the cluster the hint is applied using the cluster name prod.
DFS Ignore example - Prod
This is another ignore hint used for a prod cluster pool that protects DFS mounted data in the Access Zone. This IP pool and its SmartConnect names should not failover and uses an ignore hint igls-ignore-dfs01 where igls-ignore- is used to match and dfs01 is to make the hint unique.
DFS Ignore example - DR
This is another ignore hint used for a DR cluster pool that protects DFS mounted data in the Access Zone. This IP pool and its SmartConnect names should not failover and uses an ignore hint igls-ignore-dfs02 where igls-ignore- is used to match and dfs02 is to make the hint unique.
Hot-Cold Replication Topology Mapping Examples
Example: Single Access Zone, Single IP Pool for Access, Single IP Pool for SyncIQ
Example: Multiple Access Zone, Multiple IP Pool for Access, Single IP Pool for SyncIQ
Example: Single Access Zone containing DFS, Single IP Pool for Access, Single IP Pool for SyncIQ
Copyright Superna LLC