Hi all, looking for a suggestion on how I might be able to resolve this indexing issue.

We've got a two node Exchange 2013 DAG. On the primary copy of the database, the index keeps failing and restarting its crawl. On the secondary copy, the node is showing that the index is Failed and Suspended.

We'd stopped the search services and blown away the index folder for the DB on the mounted copy and let the crawl resume, but after a while we'll see it fail and then resume still. Looking at the High Availability > Seeding logs, event 1112 is getting logged every few minutes or more:

Automatic Reseed Manager has detected a content indexing error for database 'DB06'. The catalog may be corrupted and will be reseeded automatically. contentIndexError: The content index of old schema version 1. Newer version 12 is available.

Indexes on all other databases are healthy.

So I need to figure out the best possible way to recover the indexes for this database. I can't fail over to the secondary node with a failed index, but it looks like I could force it if I have to. I haven't yet tried deleting the index on the secondary copy, and my suspicion is trying to reseed off the mounted copy will just be bad for the secondary copy. And the schema error concerns me too.

Any suggestions on how best to resolve this?

I have seen this in my test lab few times. This should fix it...

To resolve the issue, do the following:

1. Create a new Active Directory group that is named "ContentSubmitters" and then grant Admistrators and NetworkService full access to the group. This is a dummy group and should be used as a placeholder only. You might want to add a description so that the group is not removed.
2. Force or wait for Active Directory replication.
3. Restart the following services:
   • Microsoft Exchange Search
   • Microsoft Exchange Search Host Controller

Reference: Content Index status of all or most of the mailbox databases in the environment shows "Failed"

Then recreate the Index by following manual kick off procedure -

Rebuild an Index on Exchange 2013 for Specific Databases

• Edited by Amit TankMVP, Moderator Friday, May 16, 2014 4:49 PM Added Rebuild link
• Proposed as answer by Amit TankMVP, Moderator Friday, May 16, 2014 4:58 PM
• Marked as answer by cara chenMicrosoft contingent staff Monday, May 26, 2014 7:40 AM

I have seen this in my test lab few times. This should fix it...

To resolve the issue, do the following:

1. Create a new Active Directory group that is named "ContentSubmitters" and then grant Admistrators and NetworkService full access to the group. This is a dummy group and should be used as a placeholder only. You might want to add a description so that the group is not removed.
2. Force or wait for Active Directory replication.
3. Restart the following services:
   • Microsoft Exchange Search
   • Microsoft Exchange Search Host Controller

Reference: Content Index status of all or most of the mailbox databases in the environment shows "Failed"

Then recreate the Index by following manual kick off procedure -

Rebuild an Index on Exchange 2013 for Specific Databases

• Edited by Amit TankMVP, Moderator Friday, May 16, 2014 4:49 PM Added Rebuild link
• Proposed as answer by Amit TankMVP, Moderator Friday, May 16, 2014 4:58 PM
• Marked as answer by cara chenMicrosoft contingent staff Monday, May 26, 2014 7:40 AM

Just did a double-check and we've confirmed that the ContentSubmitters group is present with the correct permissions. So it doesn't look like that's where the problem lies, unfortunately.

Hummm, do you see any Search related error in the event log? Can you post them here?

The only error I'm seeing is event 1009 in the Application log:

Log Name:      Application
Source:        MSExchangeFastSearch
Date:          5/16/2014 1:11:54 PM
Event ID:      1009
Task Category: General
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      XMB2.domain.COM
Description:
The indexing of mailbox database DB06 encountered an unexpected exception. Error details: Microsoft.Exchange.Search.Core.Abstraction.OperationFailedException: The component operation has failed. ---> Microsoft.Exchange.Search.Core.Abstraction.OperationFailedException: The component operation has failed. ---> Microsoft.Exchange.Search.Fast.FastDocumentTimeoutException: The Content Submission Service returned failure for the document: Document timeout expired
. ---> System.TimeoutException: Document timeout expired

   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Search.Fast.FastFeeder.EndSubmitDocument(IAsyncResult asyncResult)
   at Microsoft.Exchange.Search.Mdb.CrawlerFeeder.DocumentCompleteCallback(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Search.Core.Common.Executable.EndExecute(IAsyncResult asyncResult)
   at Microsoft.Exchange.Search.Engine.SearchFeedingController.ExecuteComplete(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Search.Core.Common.Executable.EndExecute(IAsyncResult asyncResult)
   at Microsoft.Exchange.Search.Engine.SearchRootController.ExecuteComplete(IAsyncResult asyncResult)

Other than that, as it crawls the index before failing, I see it cycling through tmp files, two or three at a time. Each one changes out and it keeps going until it fails. I don't recall seeing this happen with indexes before, this is pretty new to me.

Hi,

I recommend you try the following method to resolve the issue:

1. Checked and found that the Ipv6 was disabled in the registry but not in the NIC properties. Disabled it and rebooted the servers in sequence.
2. Disabled indexing on the failing database using Set-MailboxDatabase -Identity "DB06" -IndexEnabled $false.
3. Stopped the Exchange search service & Host controller service.
4. Renamed the Catalog index folder.
5. Started the services again.
6. Enabled the indexing on the database using Set-MailboxDatabase -Identity "DB06" -IndexEnabled $True
7. Reseeded the Catalog using the cmdlet Update-MailboxDatabaseCopy -Identity DB06 \MBXServer -CatalogOnly

Hope this helps!

Thanks.

I am having the exact same problem.  I have stopped the services and deleted the .Single folder and then restarted the services but it always fails maybe 20 minutes into the process with the Document Timeout Expired message you show above.  Did you ever resolve this?