Exchange 2007 SP3 not rebuilding search index
I have two Exchange 2007 SP, one on Server 2003-64, the other on Server 2008R2.
Both of them have developed problems related to the search index (users not able to search properly), so I decided to recreate the Search-Index.
I checked with
Test-ExchangeSearch -identity xy@dom.at and got a False/Searchtime=-1
I checked if the index was enabled (Get-MailboxDatabase | Format-Table Name,IndexEnabled) and got "True" for all DBS
I Triggered the rebuild with
ResetSearchIndex.ps1 "WILH-PD1"
I watched the catalogue beeing deleted, I got the 108, the 109 and finally after a couple of hours I got the 110.
The index was not recreated.
I read all those posts concerning Searchindex-Problems in Exchange 2007
I patched the HOSTS file (the
crl.microsoft.com-thingy), I reinstalled the Searchfunction from an ExchangeSP2 (in SP3 it's missing), I reregister the
regsvr32 msftepxy.dll, I manually deleted the catalogue-files, I rebooted,
repatched both to SP3 Rollup7
The CatalogueData ist still empty (just those 129kb sitting there), I get 110s, but no CPU peak hitting, nothing beeing indexed.
I Have previously installed DPM 2010, from some vague hints that DPM might be the culprit I stopped DPM-protection and deinstalled DPM agent on one Exchange, but nothing, no index.
Any ideas?
thx
Gerhard
May 24th, 2012 2:49am
did you checked whether filer pack were registered for exchange search indexer
http://support.microsoft.com/kb/944516
Free Windows Admin Tool Kit Click here and download it now
May 24th, 2012 1:58pm
sure. both 1.0 and 2.0
it changed..a bit...I have now instead of 129kb in the CI-files 101MB...but it's still completly useless...should be several tons of gigabytes....stores are 400 GB, so the indexes should be around 40GB I'd guess....
May 24th, 2012 5:40pm
Hi,
After you run resetindexsearch.ps1,please restart Microsoft
Search (Exchange) Service.
Also you can try to manually rebuild the index catalog. Detail steps you can refer to the article below:
How to Rebuild the Full-Text Index Catalog
http://technet.microsoft.com/en-us/library/aa995966(EXCHG.80).aspx
If that do not work, then please follow the steps from my post in the following thread to try to solve the problem.
Cannot load Exchange Search Index service
http://social.technet.microsoft.com/Forums/en-US/exchangesvrgeneral/thread/b5c06df7-4f90-48f8-8aa3-fb137ea0b7e8/
How to Diagnose Exchange Search Issues
http://technet.microsoft.com/en-us/library/bb123701.aspx
For outlook side, I recommend you to download Windows Desktop search again and re-install.
Description of Windows Desktop Search 3.01 and the Multilingual User Interface Pack for Windows Desktop Search 3.01
http://support.microsoft.com/kb/917013
Extending Windows Desktop Search
http://technet.microsoft.com/en-us/library/cc725753(WS.10).aspx
By the way, waht is the services pack for your Exchange Server? Please run ExBPA to have a health scan of the Exchange Server.Xiu Zhang
TechNet Community Support
Free Windows Admin Tool Kit Click here and download it now
May 25th, 2012 5:20am
Dear Xiu Zhang,
thanks a lot for your expertise. Unfortunatly I did all those steps already (with the exception of the desktop Search issues, which are completly unrelated to the question in concern).
The Servicepack installed is SP3 (the topic name was maybe not clear enough), additional i have RU7 installed. To my mind that's pretty the last patch-level available.
ExBPA is without issues, but it gives nearly no indicators about the searchfunction anyhow, doesn't ist?
About your post: you have one point where I want you to share the informational background:
You recommend to reregister some dlls (msfte.dll) and finally
find the installation package
msfte.msi on the Exchange installation CD. The
default location is \Setup\ServerRoles\Mailbox\
As far as I can see: this recommendation
is probably breaking the Exchange Server installation. The msfte.msi is not part of the SP3-package (!!!), and the SP2-msfte seems to be the utterly wrong thing to install.
Can you recheck your recommendation in light of this information?
May 25th, 2012 7:04am
Hi,
Can you find msfte.msi from you original Exchange setup media?Xiu Zhang
TechNet Community Support
Free Windows Admin Tool Kit Click here and download it now
May 28th, 2012 1:57am
The Server in question is a Server 2008R2, so the initial Exchange Setup was a SP3 (which was a download).
I have a SP2 installation as well (also a download). I don't have a MX2007 RTM-setup media.
May 28th, 2012 5:29am
Hi,
I have setup lab and found that we do not have msfte.msi from Exchange 2007 SP3 media.
So I think we may have to follow the steps in article below to diagnose the search related issue.
How to Diagnose Exchange Search Issues
http://technet.microsoft.com/en-us/library/bb123701(v=exchg.80).aspxXiu Zhang
TechNet Community Support
Free Windows Admin Tool Kit Click here and download it now
May 28th, 2012 11:19pm
Dear Xiu Zhang,
Question: is there no need for the msfte.msi or is it missing by accident?
Back to the Problem: to Diagnose the Search Issues I did the following:
I triggered the recreation of the Searchindex:
[PS] ...Scripts>ResetSearchIndex.ps1 -force "GRND-PDA1"
WARNUNG: Warten auf Beendigung des Diensts "Microsoft Exchange-Suchindizierung
(MSExchangeSearch)"...
MSExchangeSearch service stopped
Deleting catalog for SERVERMAILB2\GRND-PD1\GRND-PDA1
removing: E:\PostfachData\GRND-PD1\CatalogData-3cad7a73-9207-4346-af9b-fba494cc2660-cc64dd2d-2428-4f12-bba2-79d6d34c4d27
MSExchangeSearch service Started
In the Application log I can see Event 109 and Event 108, the CI-Data is deleted, the services are restarted.
After short delay the cpu peaks to 50%, I get msftesql running with high usage.
In PerfMon I check Full crawl Mode Status: it's 1I check some other counters, please be aware that I have a german installation and I translated those counters on the fly:
I check "Mailboxes remaing for full crawl" -> it's 7 (which is correct)
I check "Documents pending" which is slowly decreasing (from 30.000 downwards)
I check" Documents succesfully indexed" -> thats ZERO
I check "Documents with errors in indexing" which is climbing steadily (just reaching the 17.000)
Finally the crawl stops.
I get an event 110 in applog.
The perfmon counter drops to 0
"remaining Mailboxes" is down to 0
Successful indexed: 512 Documents
Documents with errors in indexing: 29.088
Cpu is down at normal 2%
THe CatalogDatafolder has files inside, but the size is very small (5,29 MB) for a store of 20 GB (should be 2 GB or something, shouldn't it?)
Searching is not working, TestExchange still delivers FALSE!
[PS] C:\Program Files\Microsoft\Exchange Server\Scripts>Test-ExchangeSearch -identity sauhammel@zfg.at
ResultFound
SearchTime
-----------
----------
False
-1
That's it basically, to my mind the Counter "Documents with errors in indexing" (caution, this is my translation, original name might differ!) pinpoints the problem. The Documents are NOT indexed....how to find the problem? Is the Filterpack not working?
br
Gerhard
May 29th, 2012 2:41pm
Hi,
The Exchange Search engine has been update to MSSearch Engine 3.1 when you update to SP3, so that is by design.
Base on my research, we have the following known issue which is related to Exchange search after you upgrade to sp3.
XADM: Exchange Search Fails After Installing Exchange 2007 SP3
http://support.microsoft.com/kb/2469863
Besides, please try the following steps and then post the related detail event information here.
1. To ensure you can search items in your Exchange, stops Microsoft Search (Exchange) service, this will also stops Microsoft
Exchange Search Indexer service. Then try to search in OWA and see if string-matching search is working. (OWA may warning you about not able to search email body, but thats a false warning, as string matching also searches email
body.)
2. Re-enable two services, if you are still unable to see the index files (CatalogData), increase Diagnostic Logging for MSExchange Search Indexer to Expert (Both General and Configuration) at Server Configuration -> Mailbox
server -> Manage Diagnostic Logging Level.Xiu Zhang
TechNet Community Support
Free Windows Admin Tool Kit Click here and download it now
May 30th, 2012 1:18am
Dear Xui Zhang,
as I stated in my initial post, I already patched the HOSTS-file with the crl.microsoft-dummy entry (http://support.microsoft.com/kb/2469863)
The behaviour on the MX is different: inside the CatalogData something IS created. Before I patched the HOSTS-file the index files were stuck at 129 kb, with the patched crl.microsoft they increased in 48hrs up to 22 MB (on a store of 20 GB) - but this is
unsuccessful!
I activated the expert-diagnostiv level and restarted the creation of the index files (ResetSearchIndex.ps1 -force "GRND-PDA1")
while the crawl was running I monitor in perfmon some of the exchangecounters:
"Anzahl der ausstehenden Batches" -> Amount of remaining Batches(?) is slowly decreasing vom 2400 downwards
"Anzahl von Wiederholungsversuchen" ->Amount of retrys(?) is 2166
"Anzahl von Dokumenten mit Fehlern bei der Indizierung" -> Documents with indexing-errors (?) is increasing from 18.000 to 40.000
Even with logging to expert on the Search I'm getting only the 109, 108 and 110 events. Is there an additional file somewhere hidden?
br
Gerhard
May 30th, 2012 2:28am
Hi,
Then please try to collect trace from Extra and then send to
me.
1. Launch Microsoft Exchange Troubleshooting Assistant by running extra.
2. Skip welcome screen, Select a Task, select Trace Control.
3. Select Trace file location, leave others as default, then click Set manual trace tags.
4. Make sure you select all Trace Types
5. under Component to trace list, check ExchangeSearch (every tag on the right are checked)
6. under Component to trace list, find Store; on the right, search for CI, checked items with CIPS or Ci; then search for search, checked all items.
7. Click start tracing. (once enabled, you can close ExTra)
8. Tracing is now running in the background, you can use the Technet article to rebuild all
index (or the impacted databases if you have identified them):
http://technet.microsoft.com/en-us/library/aa995966(EXCHG.80).aspx
9. Wait for 10-20 minutes, make sure you have encounter ID 115 error or ID
110 information.
10. Stop tracing, on the screen where ExTra reports the trace has stopped, click Exit instead of closing window directly, you will then see a result summary; if you have previously closed ExTra, select Tracing Control again to stop the current tracing.
Xiu Zhang
TechNet Community Support
Free Windows Admin Tool Kit Click here and download it now
May 30th, 2012 2:46am
I downloaded the Microsoft Exchange Troubleshooting Assistant from http://www.microsoft.com/en-us/download/details.aspx?id=25213
I was not able to install the extra.msi, because I got the error-message that framework 1.1 is missing.
Framework 1.1 is not supported on Server 2008 R2. see http://blogs.msdn.com/b/servicing/archive/2010/06/21/supportability-of-net-framework-1-1-on-windows-7-and-windows-server-2008-r2.aspx
Please advise.
br
Gerhard
May 30th, 2012 5:03am
Hi,
I found the following information from the trace file:
Got error UnknownError (13905) in GetRetryTimeFromStateAndHResult
Setting retry time 30.05.2012 10:10:28 for HRESULT UnknownError (0x80043651)
MDB 3CAD7A73-9207-4346-AF9B-FBA494CC2660 - adding document 60 1-A6C36D-1-A6CFFB to failed documents with error 0x80043651 (UnknownError (0x80043651))
For error 13905&0x80043651,it means ERROR_IPSEC_IKE_AUTHORIZATION_FAILURE, SA establishment is not authorized.
Please try to follow the steps below to troubleshoot the issue:
Check to see if you are using a 3rd party disk defragmenter on the Exchange 2007 server. This could be Diskeeper or any other 3rd party defragmentation software. If you are using this type
of software, stop all the 3rd party defragmentation services, disable all those services, reboot the Exchange 2007 server.We have seen issues in the past where search results stop working as expected due to 3rd party defragmentation softwares.
Check to see if the CatalogData folder on all databases is being scanned by file-level antivirus OR on-demand antivirus. If it is, reconfigure both file-level and on-demand antivirus so it
is not scanning the CatalogData folder on all mailboxes, reboot the Exchange 2007 server.
For further information, click on the following link:
File-Level Antivirus Scanning on Exchange 2007: Exchange 2007 Help
http://technet.microsoft.com/en-us/library/bb332342(EXCHG.80).aspx
Check to see if you are using Mount Points.
If you are, check the following link to make sure that Mount Points are not causing Exchange 2007 Search to fail:
Mount point design and MSSearch
http://blogs.technet.com/b/timmcmic/archive/2010/07/11/mount-point-design-and-mssearch.aspx
Please ensure that all the Exchange Servers have the same service pack and rollup installed.
Install IFilters and register them follow ALL the steps below.
Install Filter Pack 1.0, which allows Exchange 2007 to index Office 2007 documents:
2007 Office System Converter: Microsoft Filter Pack
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=20109
Install the hotfix which addresses a known issue when search crawls a .vsd file.
960502 Description of the 2007 Office system hotfix package (Offfiltx.msp, Visfilter.msp): December 16, 2008
http://support.microsoft.com/default.aspx?scid=kb;EN-US;960502
Additional Information on the hotfix:
960166 Error message when a search process crawls a .vsd file on a Windows 64-bit operating system that is running the 2007 Office Filter Pack: "The filtering process has been terminated"
http://support.microsoft.com/default.aspx?scid=kb;EN-US;960166
Register the Filter Pack 1.0 IFilters. This is a manual process with Exchange 2007 but we have created a script to automate the process.:
944516 How to register Filter Pack IFilters with Exchange Server 2007
http://support.microsoft.com/default.aspx?scid=kb;EN-US;944516
Besides, please manually stop the Microsoft Exchange Search Indexer service and the Microsoft search (exchange) service. Delete all catalog folders, example "E:\Exchsrvr\SG01\CatalogData-d3b801ad-be13-4a0a-a138-c23ee55aabc9-418ab808-a7ed-44e2-b3e4-e7e56a9ff3eb"
and start the two services. Waited more than 1 day.
By the way, how many Exchange Servers in the network?
Can you search mails via outlook(online/cache mode) or OWA?Xiu Zhang
TechNet Community Support
Free Windows Admin Tool Kit Click here and download it now
May 31st, 2012 1:45am
Dear All,
this issue is still existing with my exchange 2007 as well...
I red this enitre post like 5 times and then followed all the steps:-
1) Delete the index
2) Recreate the index
3) regsvr32 msftepxy.dll
4) getting plenty of 108's in my event logand only one 109 ...
but the size of catalog for both my DB's is still 129 KB
My DB sizes are 634 GB and 534 GB respectively so roughly the catalog size should be approx 60 GB and 50 GB as they are like 10% of the actual size..but the size is 129 KB both the db's...
Please help or is this is bug or something....
the only thing which is different here is that my setup does include a Symantec AV that's it... but it should not pose a problem with indexing...
Apoorv Mehrotra
June 5th, 2012 8:05am
Hi Apoorv,
I have opened a support-case with Microsoft. The Tech-Team is on the case, but no findings so far.
Concerning the AV: on the productive Server I have a Symantec AV as well, on the Test-Server I don't, both of them are failing the Index-Rebuild.
br
Gerhard
Free Windows Admin Tool Kit Click here and download it now
June 5th, 2012 8:13am
Hi Gerhard,
well i am not that as lucky to have something in testing environment as all the machines are in production.
so whatever i am doing, i have to read 10 times before applying since the number of mailboxes i am handling are above 3000 and exchange setup is ccr (2 hub cas with nlb and MBX clustered on 2 nodes)
Please do share with me the findings of MS team and rectification procedure...
Thanks...Apoorv Mehrotra
June 5th, 2012 9:27am
Hi,
Any updates in this?
On a second note, i have noticed now that advanced search features are not showing / disappeared up in OWA
Can you please advice on this?
And yes btw, my catalog size is still 129 KB for both the db's!!!
ThanksApoorv Mehrotra
Free Windows Admin Tool Kit Click here and download it now
June 12th, 2012 5:59am
Nothing new. Tomorrow there'S the support-team meeting at Microsoft's (that'S now level 5 support or something) and there the topic is under discussion.
br
Gerhard
PS: if you have no index, then you won't have advanced search features to my mind....
June 12th, 2012 11:14am
Hello Gerhard,
Yes you are right, obviously when the indexes are not built there is no question of Advance search features available!
Level 5... Waao; till today i thought that highest level support if Level - III. Hope this gets resolved soon as major of Exchange usage is on OWA in my current setup.
Please do let me know if any developments.
Thanks...Apoorv Mehrotra
Free Windows Admin Tool Kit Click here and download it now
June 13th, 2012 1:29am
Also having the same issue on one of my clustered mailbox server nodes. I have tried all steps outlined (accept the ones that dont apply, all filter packs up to date etc), My index grows to 200MB and stops indexing. Searching on the affected stores returns
no results in OWA. It was working before it failed over to the secondary node last week due to an information store error. This weekend I am going to fail it back during maintenance and see if it works on the primary again where it was functioning. Not sure
what is going on with the secondary but seems to have the same sypmtoms. I am on SP3 rollup 5.Regards, Michael
June 19th, 2012 10:28am
Hi, Gerhard.
Where did you get the dip.exe tool ??Emiliano
Free Windows Admin Tool Kit Click here and download it now
June 28th, 2012 10:35am
Hi Emiliani,
I got it by Mail. Yes, I know, sounds like a joke...but it's not an official tool, so there is no download link.
I could mail it to you, on your own risk.
br
Gerhard
June 28th, 2012 10:40am
Well, I should try it first in a testing enviroment. Please send me a copy.
Emiliano
Free Windows Admin Tool Kit Click here and download it now
June 28th, 2012 10:41am
*ggg*...google lists 261.000 entries for
Emiliano Ogando. Which one are you?
June 28th, 2012 10:45am
LOL
ogandoe@gmail.com
Emiliano
Free Windows Admin Tool Kit Click here and download it now
June 28th, 2012 10:46am
Any update? I tried the dip command many times. It still does not work? please give any sugguestions.
Thanks,
June 29th, 2012 1:33pm
John, I followed the steps posted by Gerhard and now it is working, did you tried this ??
Place a copy of the dip.exe-tool in in the bin subdirectoryGo to an elevated command prompt in the bin subdirectoryUse dip.exe with the following command line:
dip /ADMIN_SETBOOL g FTE_VerifySignature 0 (This will disable signature verification for all dlls loaded by Search for msftesql.exe )Restart the Microsoft Search Indexer Service and Microsoft Search (Exchange) service (this will also restart the Microsoft Search (Exchange) ServiceRebuild the Catalog Index ( Either by stopping Microsoft Search Indexer Service & and Microsoft Search (Exchange) service and Moving the Catalog Folder and start both services back or using script)Monitor the Size of the Catalog folderEmiliano
Free Windows Admin Tool Kit Click here and download it now
June 29th, 2012 1:41pm
I tried this again. still no luck. any suggestion?
Thanks
June 29th, 2012 3:13pm
Hi Gerhard,
Thanks for the update, sorry was out of this discussion for some time.
So as per the discussion, the dip.exe tool is not an official release from MS and also as per your suggestion i should t/f the role to a virtual box then test...
Hmmm, well i do not have that luxury to do so as all the boxes in environment are production so don't know what to do. Don't want to take the risk as well since the tool is also not certified by MS. Anyway can you however email me the tool / link to download
the same...
apoorv.mehrotra@hotmail.com
Thanks...Apoorv Mehrotra
Free Windows Admin Tool Kit Click here and download it now
July 1st, 2012 12:54am
Hi Apoorv,
yes, I really would recommend you to test (and have a backup of your data)....and to build a test-environment, that's not to complicated and even cheap. Both Server OS and Exchange have a testing period of at least 30 days. For the hardware: come on, you
are running some clustered boxes, don't tell me that you dont have a server 2008 R2 capable of running the Hyper-V role. Just make a virtual box and test it. If you can't run it on the server just take a PC (with 3000 mailboxes that shouldn'T be to hard).
Just collect a Workstation for maintenance (tell them, teh are getting an upgrade!) and voila, there you have your testbed...*ggg*
br
Gerhard
July 4th, 2012 1:49pm
Is there anyway I can get the DIP.exe tool? I have the exact same issue with one of my clients Exchange 2007 servers. Thanks
ajohnson@nextstepnetworking.com
Free Windows Admin Tool Kit Click here and download it now
August 1st, 2012 11:00am
Gerhard, can you mail me the DIP tool? I would like to try it on the exchange server I am having the same issue with. Thanks
ajohnson@nextstepnetworking.com
August 2nd, 2012 11:41am
Just keep in mind that the dip-tool might not be applicable to your configuration/situation. I'd recommend to snapshotting your system or similar backupstrategies.
br
Gerhard
Free Windows Admin Tool Kit Click here and download it now
August 3rd, 2012 2:29am
I also have this same issue and none of the fixes have resolved the issue. I have two Windows 2003 R2 two node clusters that have Exchange 2007 SP 2 RU5 that do not have the issue. I have three
2003 R2 two node clusters that have Exchange 2007 SP3 RU7 that do have the issue. I have been working with Microsoft on this issue with no resolution. Rebuilding the indexes does not do anything. On one of my servers the index size on the Information stores
only grows to 257 k 19 files and stops never grows any larger. My two other clusters that have the issue, the IS indexes grow to 641 k 25 files and stops, it will not grow any larger. Most of my DB's are 80 - 100 Gb in Size. I suspect that this is related
to SP 3 and or RU7.
I have tried the dip.exe tool , the Host file fix. One of my servers I am getting an error when logging in that there was and issue with Microsoft SharePoint search indexer, which from what
I read about Microsoft switched to the same indexing engine that is used for SharePoint and imbedded it in the install of SP 3 for exchange.
Does any one have any new news on this issue and if there is a real fix?
September 17th, 2012 5:02pm
Nopes, did not work out...
Well i'v given up on this... Going to upgrade to 2K10 (project kicked off)
Well one of the MS consultant told me it could be the issue with your DB size since one of my DB is approx 1.2 TB and other is only 80 Gig's but failing on both dont know why ...
Let us seeApoorv Mehrotra
Free Windows Admin Tool Kit Click here and download it now
September 18th, 2012 12:59am
I is definintly an issue with SP3 and or RU7. I don't have any issues on the servers that have SP2 RU5. The DB's are approximately the same on each of my servers, less than 100Gb
September 20th, 2012 12:47pm