vCenter Server 5 Service Fails

We had an issue with the vCenter Server 5 Service failing recently. Basically what happened was the VMware VirtualCenter Server service failed (out of the blue) with the following Informational Event ID 1000 logged in the Application log:

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Starting VMware VirtualCenter 5.0.0 build-623373

the message resource is present but the message is not found in the string/message table

Followed by another Info Event 1000:

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Log directory: C:\ProgramData\VMware\VMware VirtualCenter\Logs.

the message resource is present but the message is not found in the string/message table

And followed by an Error Event 1000 (upon it attempting to auto-restart the service):

The description for Event ID 1000 from source VMware VirtualCenter Server cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Failed to intialize VMware VirtualCenter. Shutting down...

the message resource is present but the message is not found in the string/message table

First try was to restart all the vCenter services - only the VMware VirtualCenter Server service was offline; all other services (like VMware VirtualCenter Management Webservices) were still online. This obviously failed with the same error as before.

All of the SQL services were verified online (these are kept on a separate server due to the size), and accessible. I ran across a similar Discussion thread in VMware Communities, which pointed to issues inside of the SQL Database. Based off of this, I decided the first order was to look at my SQL server to see what was going on there. I was able to log in, and verified that everything was up. Then I looked at my logs, and saw Event ID 17053:

C:\VCDB.ldf: Operating system error 112(There is not enough space on the disk.) encountered.

 And instantly I knew my problem. Some yahoo (namely the yahoo writing this article) must not've been paying attention when installing the VCDB database, and stuck it in the root of the C drive. Naturally, that yahoo had to fix his own problem...

So I went into Microsoft SQL Server Management Studio, and ran the following command:

alter database VCDB modify file ( name = vcdb , filename = 'E:\SQL\MDF\VCDB.mdf' )
alter database VCDB modify file ( name = vcdb_log , filename = 'E:\SQL\LDF\VCDB.ldf')
go

Then I Offlined the database (Right click, Tasks, Take Offline), and moved the files to their new homes. Then I Onlined the database, and verified that the path was correct for it. You can see the full process of how to move a SQL database here.


Once this was done, I went back to my vCenter Server, and was able to bring all my services online without incident, and was able to again go in and manage my vCenter 5 Server via vSphere Client.

------
Dustin Shaw
VCP

vRanger 5.0 is released!

As of January 31, 2011, Quest vRanger 5.0 is now publicly available and out of beta.

There are now two versions of vRanger available: vRanger Standard Edition 5.0 and vRanger Pro 5.0. Pro buys you these additional features that people have been looking forward to: Replication, LAN Free, Catalog, and Linux FLR. Here's a look at them:

vReplicator is integrated into vRanger Pro
You no longer have to buy two separate products to do backups and replication if you choose to go the Quest route. You can use one single product for both.

Support for NFS and FTP Repositories
We already had support for CIFS and SFTP targets. The addition of NFS and FTP gives more flexibility on what you can use as a backup target. They didn't integrate credentials into the NFS support, however, so you will need to make sure that you lock down your NFS shares in other ways.

Linux File Level Restores
I know a number of people who are excited about this one. You've always been able to do FLR on Windows VMs, and now you can do the same with LinuxVMs.

Catalogs
You now have the option to catalog your file and directory structure during the backup process, rather than having to wait for it to propagate during your restore process. Awesome thing if you are always doing the type of restore where someone is sitting over your shoulder because they needed the file (that they deleted, coincidentally) yesterday; now they can't scream at you for it taking too long. My concern is that it might increase the backup window (one of the big selling points for vRanger for me is the speed) - I'm going to have to test it out and see.

LAN Free Restores over Fibre
Allows you do restores on your fibre network without touching the bandwidth on your lan.


What's Missing

Unfortunately, I notice several key features missing that were promised, that we will probably have to wait for vRanger 6.0 (current due date 2012) until they are released:

Direct-to-Target backups from ESXi (it has to backup using the vRanger box as a proxy) via a Virtual Appliance. You can currently do Direct-to-Target from ESX, but due to the lack of service console on ESXi, it will need a VA.

FlashRestore, which allows you to start running a VM before it finishes restoring. Very cool for those who are under a time crunch (wait, I believe that's the whole IT industry).


If you want to read more, or download a trial, you can from the Quest vRanger Website.

------
Dustin Shaw
VCP

Speed, Speed, Speed

Who would've ever thought you'd see network performance like this on an Windows box:

This is the networking graph I got off of a physical box running the inital push for Quest vConverter's Continuous Protection option. The blip in the middle was it switching from the C: volume to the D: volume. It just about knocked my socks off - I've never seen anything eat up the entire NIC on Windows.

I found out some really cool stuff with vConverter. With the Continuous Protection option, you can keep a full-image copy of your physical server on an NTFS Share, ESX(i) Server, or Hyper-V server. After the first push, it just copies block (or file level) changes (whichever option you want) to the target store. This way, if you ever lose your physical box, you can either immediately pop it online in your ESX/Hyper-V box, or do a V2P back to your original hardware once it's back up and running. Very cool stuff.

Even cooler - let's say you're looking at Offsite DR. You already have Virtual Servers (VMware or Microsoft) in a hardened data center, and are currently replicating your VMs at your main office offsite to there. Good for you! Now how do you push your physical boxes out there?

With vConverter, you can do your initial copy locally (say to an NTFS Share on a NAS device), then you can take it to your Datacenter, and upload it to your SAN that your ESX or Hyper-V boxes use. Then go into vConverter, do a "change target" and select your ESX or Hyper-V box, tell it where the data is, and your good to go. Now all you have to worry about getting across the WAN is your block changes. Now you really do have an Offsite DR solution.

------
Dustin Shaw

VCP

vConverter 5.1 is out!

Yesterday Quest released vConverter 5.1 onto their website for download. You can check out the product here.

For those of you familiar with their product, you'll know that vConverter 4.2 was the previous version that was out. They recently released vConverter 5.0 as an "early release preview" (AKA - Beta), but it had numerous issues. One tech that I spoke with said that the marketing team put version 5 online for download and issued a press release before they consulted with tech or engineering - sounds like many marketing guys that I know.

If you were one of the lucky ones that installed the "early release preview," then you'll have to uninstall it before you install vConverter 5.1.

The new features to look for in version 5.1 are:

P2V for Linux Servers: this was a big absent in the previous releases. I know many penguin fans who are excited about this one. It's a Cold Convert, so you have to be down when doing the conversion and have physical access to the box so that you can boot off a CD or PXE server (unless of course you have ILO or DRAC or other console access). This is unfortunate since VMware's vCenter Converter 4.x supports several Linux builds doing Hot Convert, but there are a number of other features that vConvert does buy you.

ESXi Support: This is a good thing since VMware is pushing everyone hard away from ESX onto ESXi. They are doing this by using a Virtual Appliance.

Hidden Partitions Support: This has recently become an issue with Windows 7 and Server 2008r2, keeping most converter programs from being able to work. I'm glad to see this added, as if you've been reading my blog you know that I recently ran into an issue caused from someone doing a conversion of a 2008r2 box using Acronis.

vConverter 5.1 download comes with a 30 day trial license that allows you to use Continuous Protection on up to 5 source machines, and limits you to 5 conversions (not including the Continuous Protection conversions).

------
Dustin Shaw
VCP