A few weeks ago, we encountered a problem with the orchestrated creation of virtual machines where “VMM Error 801: VMM cannot find VirtualHardDisk <GUID>” was being reported. The Orchestrator runbook was terminating, failing to create the VM. The service request would fail in turn.
I did a quick check up on Orchestrator and Service Manager, no misconfiguration or issues were evident on the runbooks or the request offerings. This was expected since the error was being returned by VMM. I created several test VMs to get some insight into the problem and noticed that the creation was failing on only one of the Hyper-V farms in our environment. The GUID was always the same, it never changed between test runs.
Further checking showed that only one node on the cluster was causing the issue. Whenever a VM is created on this node, the creation would fail. Creating the virtual machine directly from the template in VMM fails as well with the same error. Since this node had the lightest load, it was given the best star rating for placement. The node was healthy otherwise and was not reporting any issues. I migrated all roles off the node and created a test VM, it was created successfully. The problem had to be caused by one the hosted VMs.
I checked the VMM logs for any errors that popped up when the roles were being migrated. VMM had failed to refresh one of the VMs.
The GUID was the same. I created another test VM on the node that now hosted the roles, the creation failed with the same error. VMM was unable to refresh the problematic VM and this was causing it to be unable to create any new virtual machines on the node hosting this VM. When I tried checking this virtual machine properties through VMM, the console was unable to display the properties and it crashed.
I checked the VM through Hyper-V Manager. It was running on a differencing VHD with a direct link to the root VHD.
This VM was supposed to have a snapshot which was taken around November 2013 when the application was updated. Hyper-V Manager was showing that the virtual machine had no snapshots though. It seemed VMM was having problems refreshing the virtual disks of this VM or viewing its properties when the snapshot configuration is missing. I backed up the VMM database and executed the following SQL query to remove orphaned objects as detailed in this Technet article.
BEGIN TRANSACTION T1
DECLARE custom_cursor CURSOR FOR
SELECT VHDId, VDriveId from
dbo.tbl_WLC_VDrive WHERE [VHDId] NOT IN
(SELECT VHDId from dbo.tbl_WLC_VHD WHERE VHDID IS NOT NULL)
DECLARE @VHDId uniqueidentifier
DECLARE @VDriveId uniqueidentifier
FETCH NEXT FROM custom_cursor INTO @VHDId, @VDriveId
WHILE(@@fetch_status = 0)
if(@VHDId is NOT NULL)
DELETE FROM dbo.tbl_WLC_VDrive
WHERE VDriveId = @VDriveId
FETCH NEXT FROM custom_cursor INTO @VHDId, @VDriveId
COMMIT TRANSACTION T1
The query ran successfully, however, it did not update any rows and the problem remained.
I shut down the VM thinking that a merge operation would take place but Hyper-V did not initiate any such operation. This VM is hosted on a Windows Server 2008 R2 Hyper-V farm by the way, versions prior to Server 2012 do not support online VHD merge.
The disks had to be manually merged. Disk editing allows you to manually merge a differencing VHD back to its parent with a simple procedure.
From within Hyper-V Manager, launch Edit Disk. Browse to the location of the differencing disk then choose the Merge option. Finally, choose the option to merge the differencing disk back to its parent disk. When the merge process was finished, a new virtual machine was created and the newly merged VHD was assigned to it.
Virtual Machine Manager was now able to successfully refresh the properties of the new VM and the orchestrated virtual machines creation returned to a fully operational state.
Caution: In case you have more than one snapshot and they are chained serially, be careful with the options during the merge wizard. Always merge a differencing VHD back to its immediate parent and not the root unless it is the last (oldest) AVHD in the chain. Start from the newest snapshot and work your way back.
2 responses to “VMM Error 801 and VM Creation Failure”
I have what seems to be a very similar issue but I am not able to fix this thus far.
I have found that via SCVMM I can no longer put any hosts into maintenance mode, the job fails.
I dug into trace logging and found a missing Virtual hard disk error like what you reference in your article.
Then through dumb luck I pinned this down to a single vm. I tried to connect to the virtual console of a specific vm and received the same error.
I have run the sql script that you posted, as does microsoft, but it does not delete any rows.
I have shutdown my vm, deleted all snapshots and presumably it merged the disk. It went from a differencing disk to a dynamically expanding VHD. But the error persists.
I dug around in the SCVMM database and found the Virtual hard disk GUID as a ParentVHDId in tbl_WLC_VHD. It doesnt seem to be orphaned per the logic of the database script. My vm functions ok and is not missing any volumes, unless it is missing an uneeded volume that was possibly just deleted from storage?
Any thoughts on where to do from here?
Some questions to be able to better know what the problem is and help you:
– What’s the error that you are receiving when you attempt to put VMs in maintenance mode? and is the same error as when you try to connect to the console?
– Can you migrate that VM to other hosts (if you have the capability)?
– Which version of VMM & Hyper-V are you running?
– If you connect to the console using Hyper-V manager, does it work?
– Everything works fine when the VM is on, meaning you can remote into it or the application(s) on it work as intended?
If the script does not update any rows then you are right, you do not have any orphaned VHDs. Check the properties of the virtual machine using Hyper-V manager and check the attached hard disk if it has any differencing disk(s).
Let me know the answers to the questions above and we can take it from there.