Part 2: Search issues

This is the rest of the story regarding Search issues

Problem: the customer has a server farm (the platform is one SQL, two WFE with MOSS 2007) and NLB running. When accessing one of the servers, the Search Box is missing. This is what the customer reported to me, and where my troubleshooting started.

Summary: Resolving a database schema mismatch after applying Security Update KB 956716

So, I had tried to reset the content index on the WFE in the server farm. But that crashed and I got into a real mess where I restarted the search services, reset the passwords on all search accounts, etc. Nothing helped me so that I could perform a Full crawl again. I got a lot of errors within Search Settings and finally what happened was that when I clicked on Full Crawl it only jumped back to "Idle", so nothing could trigger the Full crawl.

Then I discovered the real issue: on the WFE where Search box was disabled, a patch for SharePoint had been installed. No one knows why that patch was installed, probably it has been downloaded and installed along with some other updates. So, there was a mismatch in patches since the KB956716 only was installed on one server and not on the other (as you can see, it is divided into three instances - strange):

So, I ran the Sharepoint Configuration Wizard. That was successful, and the search box came back! Success I thought and ran a Full Crawl. Everything was working perfectly. The morning after I tested the search again... No results!! Hm, what had happened during the night?? Looked through Event Viewer and there were some error messages saying that there was a database error:

Event Type: ErrorEvent Source: Office Server SearchEvent Category: Gatherer Event ID: 10036Date: 18-03-2009Time: 14:00:07User: N/AComputer: Dxxxx001
Description:A database error occurred.
Source: Microsoft OLE DB Provider for SQL
ServerCode: 4060 occurred 1 time(s)
Description: Cannot open database "SharedServices2_Search_DB" requested by the login. The login failed.

So I installed the kb on the search server as well, but the first time the server hung and we had to restart it. Then I ran the kb installation again and success this time. Restarted the server. Ran the configuration wizard. and it was successful. Started CA and Search settings and tada, there was a new link, Search Administration which is the same link that you get when you install the infrastructure upgrade.Performed a full crawl.... and yes! It started!

Conclusion: always make sure your server farm has a "patch match" 🙂 And always run the Config Wizard after a patch has been installed, then all will work fine.