Wednesday, 24 December 2014

Office Arguments - Maximum VMDK size is NOT 2TB-512bytes

...if you want to use snapshots

Pop Quiz
Q: What's the maximum size VMDK you should create in vSphere 5.1 or earlier?

A: Most people that have studied for VCP will know the maximum VMDK size is 2TB minus 512 bytes. If you create a disk in the GUI, it allows you to choose 2TB, but it's smart enough to minus 512 bytes.

So technically that's the maximum VMDK size, but you should NOT create it that big.

Why?

If you plan to take snapshots, there's additional overhead you need to take into account. For a 2TB VMDK, there's up to 16GB of overhead for the snapshot. So in reality, the VMDK needs to be 2032GB to allow for that overhead.

If you have created a 2TB VMDK and attempt to take snapshots, you may get the error:  
File is larger than maximum file size supported (1012384)
or:
File <unspecified filename> is larger than the maximum size supported by the datastore '<unspecified datastore>.

As mentioned in KB 1012384, depending on the block size of the VMFS volume:

Maximum VMDK sizeMaximum OverheadMaximum size less overhead
256GB - 512B~ 2GB254GB
512GB - 512B~ 4GB508GB
1TB - 512B~ 8GB1016GB
2TB - 512B~ 16GB2032GB


If you plan to take snapshots, the maximum VMDK size is 2032GB. Or better still, embrace Decimal, and round it off to 2000GB.

The younger generation will know a TB as 1000GB (http://en.wikipedia.org/wiki/Terabyte). Older folks will still be used to a TB being 1024GB, now known as a Tebibyte TiB (http://en.wikipedia.org/wiki/Tebibyte). Hey VMware, please update all your doco from TB to TiB ;-)

Monday, 22 December 2014

Maximum Disks Per SCSI Controller is NOT 15

 Pop Quiz

Q: What's the maximum number of disks per SCSI Controller?

A: It depends.. On your VCP exam, you would have said 15. Correct.

Although if you want to clone or snapshot and quiesce a VM, the maximum is 7 disks per SCSI controller.

Each SCSI Controller can control 15 disks and the quiesced snapshots in Windows 2008 require one available slot per existing disk.

If you have more than 7 disks, the clone / quiesce part will fail, and you'll have the following errors in vCenter and the VM's vmware.log:

An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.

2014-12-22T00:19:16.188Z| vmx| ToolsBackup: not enough empty nodes (needed 8, found 7)
2014-12-22T00:19:16.188Z| vmx| ToolsBackup: changing quiesce state: IDLE -> DONE
2014-12-22T00:19:16.188Z| vmx| SnapshotVMXTakeSnapshotComplete done with snapshot 'clone-temp-1419207556192169': 0
2014-12-22T00:19:16.188Z| vmx| SnapshotVMXTakeSnapshotComplete: Snapshot 0 failed: Failed to quiesce the virtual machine. (40)


It's explained in http://kb.vmware.com/kb/2015181.

Just one more way snapshots can bring you pain.

It's listed as only affecting Windows 2008, but I'm not sure why other versions are exempt. At this time there's no mention of this affecting vSphere 5.5.

Updated 23/12/2014:  This affects Windows 2008 and above (ie: Win 2012), and is still an issue in vSphere 5.5. KB article will be updated.

For the lazy, see here:

Creating a quiesced snapshot of a Windows 2008 virtual machine fails with the error: Snapshot 0 failed: Failed to quiesce the virtual machine.

Creating a quiesced snapshot of a Windows 2008 virtual machine fails with the error: Snapshot 0 failed: Failed to quiesce the virtual machine. (2015181)


Symptoms

  • Cannot create a quiesced snapshot of a Windows 2008 virtual machine.
  • Creating a quiesced snapshot of a Windows 2008 virtual machine fails.
  • In the vmware.log file of the affected virtual machine, you see entries like this:

    XXXX-03-08T04:10:09.790Z| vmx| SnapshotVMXTakeSnapshotComplete done with snapshot 'test4': 0
    XXXX-03-08T04:10:09.790Z| vmx| SnapshotVMXTakeSnapshotComplete: Snapshot 0 failed: Failed to quiesce the virtual machine. (40).
    XXXX-07-01T15:30:43.244Z| vmx| ToolsBackup: not enough empty nodes (needed 9, found 6)


    where the values provided in the errors can vary.

Cause

Each SCSI Controller can control 15 disks and the quiesced snapshots in Windows 2008 require one available slot per existing disk.
This issue occurs if the virtual machine has more than seven disks attached to a single controller.

Resolution

To resolve this issue:
  1. Create a new thin virtual disk. This allows you to add a new SCSI Controller.
  2. Ensure the new thin virtual disk is attached to SCSI1:0 node.
  3. Retry creating the quiesced snapshot. You are now able to create the snapshot successfully.
Alternatively, for virtual machines with more than 7 disks on a single SCSI controller, instead of creating a new thin disk and adding this to a new SCSI controller (SCSI1:0):
  1. Shut down the virtual machine.
  2. Spread out the existing disks between multiple SCSI controllers.

    To do this:

    1. Right-click the virtual machine and click Edit Settings.
    2. Change the Virtual Device Node to your new desired SCSI controller.
  3. Power on the virtual machine.

    Note
    : Use this method if there is limited storage space or limited authority to create new disks.

Saturday, 20 December 2014

SRM - IP Customisation error


During an SRM (5.0) failover, a VM failed during the IP customisation failed at step 11. It was strange as we hadn't seen this error in quite a while.

11. Power On Priority 3 VMs     Error - Cannot complete customization, possibly due to a scripting runtime error or invalid script parameters (Error code: -1). IP settings may have been partially applied.

Looking through the SRM logs, they pointed to an error in C:\Windows\TEMP\vmware-imc\guestcust.log at the VM guest.

On the guest OS, looking at C:\Windows\TEMP\vmware-imc\guestcust.log:

Guestcust.log :
=================
 2014-12-04T13:12:07  INFO] GuestCustUtil invoked.
 2014-12-04T13:12:07  INFO] Params: setupNic -m 00:50:56:aa:aa:aa -s 10.10.10.145 -su 255.255.255.0
 2014-12-04T13:12:07 DEBUG] Rpci: Sending request='deployPkg.update.state 4 101 C:\Windows\TEMP\vmware-imc\guestcust.log@No error message found for error code -2147217406 (0x80041002)'

 2014-12-04T13:12:07 DEBUG] Rpci: Sent request='deployPkg.update.state 4 101 C:\Windows\TEMP\vmware-imc\guestcust.log@No error message found for error code -2147217406 (0x80041002)', reply='Unknown command', len=15, status=0

 2014-12-04T13:12:07 ERROR] Unable to set customization status in vmx.

 2014-12-04T13:12:07 ERROR] error number 80041002, No error message found for error code -2147217406 (0x80041002)
 2014-12-04T13:12:07  INFO] GuestCustUtil exiting.


Checking with VMware Support, they said this occurs when the WMI repository is corrupted. You have to rebuild the WMI repository.

For rebuilding the WMI repository, check out:

http://blogs.technet.com/b/askperf/archive/2009/04/13/wmi-rebuilding-the-wmi-repository.aspx

The solution looks pretty full on.

As always, be careful and contact VMware support for any issues like this.