Dear All, My DAG IP was not replying for some reason then I moved all DBs on 1 Ex server and deleted copies from others and tried to delete other exchange members from DAGroup ( I thought to delete this DAG and then create another one). Then deletion was not successful as I thought Servers were partially deleted I think. Unfortunately I Evict 2 of the nodes from Fail Over Cluster then removed from DAG. I DIDNT/COULDNT DELETE ANY DAG. I created another DAG to get others into that but that is also giving almost same error.

I think of getting rid of that error and get those servers back into this DAG. May be deleting from ADSI or metadata (but I don't know how).

Now the ping of DAG is back.

Then tried to add Exchange Servers in DAG again but that was giving error:

The following servers in the Windows Failover Cluster are not in Active Directory: exchange-server. This is usually the result of an incomplete membership change (add or remove) of the database availabilty group.
Click here for help... http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.140).aspx?v=14.2.342.0&t=exchgf1&e=ms.exch.err.Ex280EB9 (NO SOLUTION ON THIS LINK)

Warning:
The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-09-27_06-08-53.119_add-databaseavailabiltygroupserver.log".

Exchange Management Shell command attempted:
Add-DatabaseAvailabilityGroupServer -MailboxServer 'EXCHANGE-SERVER' -Identity 'DAG'

I am pasting that log file here for further details.

add-databaseavailabiltygroupserver started on machine EXCHANGE-SERVER.
[2013-09-27T06:08:53] add-dagserver started
[2013-09-27T06:08:53] commandline:         $scriptCmd = {& $wrappedCmd @PSBoundParameters }
[2013-09-27T06:08:53] Option 'Identity' = 'DAG'.
[2013-09-27T06:08:53] Option 'MailboxServer' = 'EXCHANGE-SERVER'.
[2013-09-27T06:08:53] Option 'DatabaseAvailabilityGroupIpAddresses' = ''.
[2013-09-27T06:08:53] Option 'WhatIf' = ''.
[2013-09-27T06:08:53] Process: w3wp w3wp.exe:5608.
[2013-09-27T06:08:53] User context = 'NT AUTHORITY\SYSTEM'.
[2013-09-27T06:08:53]   Member of group 'Everyone'.
[2013-09-27T06:08:53]   Member of group 'BUILTIN\Users'.
[2013-09-27T06:08:53]   Member of group 'NT AUTHORITY\SERVICE'.
[2013-09-27T06:08:53]   Member of group 'CONSOLE LOGON'.
[2013-09-27T06:08:53]   Member of group 'NT AUTHORITY\Authenticated Users'.
[2013-09-27T06:08:53]   Member of group 'NT AUTHORITY\This Organization'.
[2013-09-27T06:08:53]   Member of group 'BUILTIN\IIS_IUSRS'.
[2013-09-27T06:08:53]   Member of group 'LOCAL'.
[2013-09-27T06:08:53]   Member of group 'IIS APPPOOL\MSExchangePowerShellAppPool'.
[2013-09-27T06:08:53]   Member of group 'BUILTIN\Administrators'.
[2013-09-27T06:08:53] Updated Progress 'Validating the parameters.' 2%.
[2013-09-27T06:08:53] Working
[2013-09-27T06:08:53] Mailbox server: value passed in = EXCHANGE-SERVER, mailboxServer.Name = EXCHANGE-SERVER, mailboxServer.Fqdn = EXCHANGE-SERVER.DOMAIN.COM
[2013-09-27T06:08:53] LogClussvcState: clussvc is Running on EXCHANGE-SERVER.DOMAIN.COM. Exception (if any) = none
[2013-09-27T06:08:53] The IP addresses for the DAG are (blank means DHCP): 10.xx.x.xx
[2013-09-27T06:08:53] Looking up IP addresses for DAG.
[2013-09-27T06:08:53]   QICTDAG = [ 10.xx.x.xx ].
[2013-09-27T06:08:53] Looking up IP addresses for exchange-server.
[2013-09-27T06:08:53]   exchange-2 = [ 10.xx.x.xx ].
[2013-09-27T06:08:53] Looking up IP addresses for EXCHANGE-SERVER.DOMAIN.COM.
[2013-09-27T06:08:53]   EXCHANGE-SERVER.DOMAIN.COM = [ 10.xx.x.xx ].
[2013-09-27T06:08:53] DAG DAG has 1 servers:
[2013-09-27T06:08:53] DAG DAG contains server EXCHANGE-SERVER.
[2013-09-27T06:08:53] Updated Progress 'Checking if Mailbox server 'EXCHANGE-SERVER' is in a database availability group.' 4%.
[2013-09-27T06:08:53] Working
[2013-09-27T06:08:53] Mailbox server 'EXCHANGE-SERVER' is already a member of cluster 'DAG'.
[2013-09-27T06:08:53] DumpClusterTopology: Opening remote cluster DAG.
[2013-09-27T06:09:14] DumpClusterTopology: Failed opening with Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(DAG.DOMAIN.COM) failed with 0x6ba. Error: The RPC server is unavailable"' failed.. ---> System.ComponentModel.Win32Exception: The RPC server is unavailable
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Cluster.ClusApi.AmCluster.OpenByName(AmServerName serverName)
   at Microsoft.Exchange.Management.SystemConfigurationTasks.DumpClusterTopology..ctor(String nameCluster, ITaskOutputHelper output)
[2013-09-27T06:09:14] Dumping the cluster by connecting to: DAG.
[2013-09-27T06:09:14] DumpClusterTopology: Not dumping, since the cluster could not be contacted.
[2013-09-27T06:09:14] DumpClusterTopology: Opening local cluster.
[2013-09-27T06:09:14] Dumping the cluster by connecting to: .
[2013-09-27T06:09:14] The cluster's name is: DAG.
[2013-09-27T06:09:14] Groups
[2013-09-27T06:09:14]     group: Available Storage [not a CMS]
[2013-09-27T06:09:14]         OwnerNode: EXCHANGE-SERVER1.DOMAIN.COM
[2013-09-27T06:09:14]         State: Offline
[2013-09-27T06:09:14]     group: Cluster Group [Cluster Main Group]
[2013-09-27T06:09:14]         OwnerNode: EXCHANGE2-SERVER.DOMAIN.COM
[2013-09-27T06:09:14]         State: Failed
[2013-09-27T06:09:14]             Resource: File Share Witness (\\WITNESS-SERVER.DOMAIN.COM\DAG.DOMAIN.COM) [Online, type = File Share Witness, PossibleOwners = EXCHANGE-SERVER,EXCHANGE3-SERVER,EXCHANGE4-SERVER ]
[2013-09-27T06:09:14]             Resource: Cluster Name [Offline, type = Network Name, PossibleOwners = EXCHANGE-2,EXCHANGE1-SERVER,EXCHANGE2-SERVER ]
[2013-09-27T06:09:14]                 NetName = [DAG]
[2013-09-27T06:09:14]             Resource: Cluster IP Address [Offline, type = IP Address, PossibleOwners = EXCHANGE-2,EXCHANGE1-SERVER,EXCHANGE2-SERVER ]
[2013-09-27T06:09:14]                 Address = [10.xx.x.xx]
[2013-09-27T06:09:14]                     EnableDhcp = [0]
[2013-09-27T06:09:14]                     Network = [Cluster Network 1]
[2013-09-27T06:09:14] Nodes
[2013-09-27T06:09:14]     node: EXCHANGE-2.DOMAIN.COM [ state = Up ]
[2013-09-27T06:09:14]     node: EXCHANGE1-SERVER.DOMAIN.COM [ state = Up ]
[2013-09-27T06:09:14]     node: EXCHANGE2-SERVER.DOMAIN.COM [ state = Up ]
[2013-09-27T06:09:14] Subnets
[2013-09-27T06:09:14]     Name(Cluster Network 3), Mask(10.xx.x.x/24), Role(ClusterNetworkRoleInternalAndClient)
[2013-09-27T06:09:14]         NIC 10.xx.x.xx on Node EXCHANGE2-SERVER in State=Up
[2013-09-27T06:09:14]         NIC 10.xx.x.xx on Node exchange1-server in State=Up
[2013-09-27T06:09:14]     Name(Cluster Network 2), Mask(10.10.10.0/24), Role(ClusterNetworkRoleInternalUse)
[2013-09-27T06:09:14]         NIC 10.10.10.x on Node EXCHANGE2-SERVER in State=Up
[2013-09-27T06:09:14]         NIC 10.10.10.x on Node EXCHANGE-2 in State=Up
[2013-09-27T06:09:14]         NIC 10.10.10.20 on Node exchange1-server in State=Up
[2013-09-27T06:09:14]     Name(Cluster Network 1), Mask(10.xx.x.0/24), Role(ClusterNetworkRoleInternalAndClient)
[2013-09-27T06:09:14]         NIC 10.xx.x.xx on Node EXCHANGE-2 in State=Up
[2013-09-27T06:09:14] Opening the cluster on nodes [exchange2-server].
[2013-09-27T06:09:14] Other mailbox servers in the DAG are already members of cluster 'DAG'
[2013-09-27T06:09:14] The server EXCHANGE-2 belongs to the same cluster as the other servers (DAG).
[2013-09-27T06:09:14] Successfully resolved the servers based on the stopped servers list.
[2013-09-27T06:09:14] The following servers are in the StartedServers list (The list is the StartedServers property of the DAG in AD):
[2013-09-27T06:09:14] The following servers are in the StoppedServers list:
[2013-09-27T06:09:14] Verifiying that the members of database availability group 'DAG' are also members of the cluster.
[2013-09-27T06:09:14] Verifying that the members of cluster 'DAG' are also members of the database availability group.
[2013-09-27T06:09:14] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-09-27_06-08-53.119_add-databaseavailabiltygroupserver.log".
[2013-09-27T06:09:14] WriteError! Exception = Microsoft.Exchange.Management.Tasks.DagTaskServersInClusterNotInAd: The following servers in the Windows Failover Cluster are not in Active Directory: exchange1-server. This is usually the result of an incomplete membership change (add or remove) of the database availabilty group.
[2013-09-27T06:09:14] Updated Progress 'Done!' 100%.
[2013-09-27T06:09:14] COMPLETED
add-databaseavailabiltygroupserver explicitly called CloseTempLogFile().

• Edited by Az Lashari Friday, September 27, 2013 3:12 PM

Please check to see if the computer object was created in AD for the second DAG, make sure that all the members are out of the DAG before deleting it.

Dear, new DAG is in AD but I am unable to add or delete members in it and I am getting that error mentioned in my post.

In current situation I have only one Server in each DAG. I moved all DBs to server which is live. If I will remove that DAG then what will happen all clients will still work normally or not?

Please confirm.

It seen like you're having some conflict base on the log, verify the status of "EXCHANGE-2"
[2013-09-27T06:09:14] Other mailbox servers in the DAG are already members of cluster 'DAG'
[2013-09-27T06:09:14] The server EXCHANGE-2 belongs to the same cluster as the other servers (DAG).

This saying that the DAG is in the stoppedservers list, verify this too.
[2013-09-27T06:09:14] The following servers are in the StoppedServers list:
[2013-09-27T06:09:14] Verifying that the members of database availability group 'DAG' are also members of the cluster.
[2013-09-27T06:09:14] Verifying that the members of cluster 'DAG' are also members of the database availability group.

This saying that "exchange1-server" is not in AD.
[2013-09-27T06:09:14] WriteError! Exception = Microsoft.Exchange.Management.Tasks.DagTaskServersInClusterNotInAd: The following servers in the Windows Failover Cluster are not in Active Directory: exchange1-server. This is usually the result of an incomplete membership change (add or remove) of the database availability group.

You mention that you evicted two nodes from the cluster, were you using cluster manager? if that so then it cause conflict in Exchange.

Please only use EMC when making config changes in the DAG so you don't cause any conflict to Exchange.

Removing the last the Active Member from the DAG will not cause any issue, DAG only provide HA, once you rebuild the DAG you can add all the member back and configure it they will replicate again, also one server can't be in two DAG at the same time and having one server in one DAG will not provide anything.

Please confirm the above.

It seen like you're having some conflict base on the log, verify the status of "EXCHANGE-2"
 [2013-09-27T06:09:14] Other mailbox servers in the DAG are already members of cluster 'DAG'
 [2013-09-27T06:09:14] The server EXCHANGE-2 belongs to the same cluster as the other servers (DAG).


 This saying that the DAG is in the stoppedservers list, verify this too.
 [2013-09-27T06:09:14] The following servers are in the StoppedServers list:
 [2013-09-27T06:09:14] Verifying that the members of database availability group 'DAG' are also members of the cluster.
 [2013-09-27T06:09:14] Verifying that the members of cluster 'DAG' are also members of the database availability group.


 This saying that "exchange1-server" is not in AD.
 [2013-09-27T06:09:14] WriteError! Exception = Microsoft.Exchange.Management.Tasks.DagTaskServersInClusterNotInAd: The following servers in the Windows Failover Cluster are not in Active Directory: exchange1-server. This is usually the result of an incomplete membership change (add or remove) of the database availability group.

You mention that you evicted two nodes from the cluster, were you using cluster manager? if that so then it cause conflict in Exchange.

Please only use EMC when making config changes in the DAG so you don't cause any conflict to Exchange.

Removing the last the Active Member from the DAG will not cause any issue, DAG only provide HA, once you rebuild the DAG you can add all the member back and configure it they will replicate again, having one server in one DAG will not provide anything.

• Edited by Chaen Fong Sunday, September 29, 2013 6:31 PM Typo

Yes one server removed properly from EMC other 2 were removed from Cluster Manager.

How can I delete those two stopped servers from list. If these are removed properly then the problem will be resolved.

Need advise to clear those records.

Remove database copy from the server before proceeding:

Remove-DatabaseAvailabilityGroupServer -Identity DAG-Name -MailboxServer MBX-Server-Name

This is the last server having all the DBs copies on it - If I will remove this one from DAG then users will be ok or disconnected from server. Then will delete the DAG.

Please advise.

Dear All,

Glad to inform you that I found one entry of a server which was not member of currently working DAG in Failover Cluster then evicted that node from there.

Then removed server from new DAG.

Then removed member server from currently working DAG.

Created New DAG and added required members in it. Cluster server on those servers automatically came from disabled to automatic and started.

And all worked fine.

I would like to thank you all for your kind help.

• Marked as answer by Az Lashari 5 hours 23 minutes ago