Thursday, May 24, 2012

vCenter Server 5 Service Fails

We had an issue with the vCenter Server 5 Service failing recently. Basically what happened was the VMware VirtualCenter Server service failed (out of the blue) with the following Informational Event ID 1000 logged in the Application log:

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Starting VMware VirtualCenter 5.0.0 build-623373

the message resource is present but the message is not found in the string/message table

Followed by another Info Event 1000:

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Log directory: C:\ProgramData\VMware\VMware VirtualCenter\Logs.

the message resource is present but the message is not found in the string/message table
And followed by an Error Event 1000 (upon it attempting to auto-restart the service):

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Failed to intialize VMware VirtualCenter. Shutting down...

the message resource is present but the message is not found in the string/message table
First try was to restart all the vCenter services - only the VMware VirtualCenter Server service was offline; all other services (like VMware VirtualCenter Management Webservices) were still online. This obviously failed with the same error as before.

All of the SQL services were verified online (these are kept on a separate server due to the size), and accessible. I ran across a similar Discussion thread in VMware Communities, which pointed to issues inside of the SQL Database. Based off of this, I decided the first order was to look at my SQL server to see what was going on there. I was able to log in, and verified that everything was up. Then I looked at my logs, and saw Event ID 17053:
C:\VCDB.ldf: Operating system error 112(There is not enough space on the disk.) encountered.
 And instantly I knew my problem. Some yahoo (namely the yahoo writing this article) must not've been paying attention when installing the VCDB database, and stuck it in the root of the C drive. Naturally, that yahoo had to fix his own problem...

So I went into Microsoft SQL Server Management Studio, and ran the following command:

alter database VCDB modify file ( name = vcdb , filename = 'E:\SQL\MDF\VCDB.mdf' )
alter database VCDB modify file ( name = vcdb_log , filename = 'E:\SQL\LDF\VCDB.ldf')
go
Then I Offlined the database (Right click, Tasks, Take Offline), and moved the files to their new homes. Then I Onlined the database, and verified that the path was correct for it. You can see the full process of how to move a SQL database here.


Once this was done, I went back to my vCenter Server, and was able to bring all my services online without incident, and was able to again go in and manage my vCenter 5 Server via vSphere Client.

------
Dustin Shaw
VCP