A few weeks ago, we encountered a problem with the orchestrated creation of virtual machines where “VMM Error 801: VMM cannot find VirtualHardDisk <GUID>” was being reported. The Orchestrator runbook was terminating, failing to create the VM. The service request would fail in turn.
I did a quick check up on Orchestrator and Service Manager, no misconfiguration or issues were evident on the runbooks or the request offerings. This was expected since the error was being returned by VMM. I created several test VMs to get some insight into the problem and noticed that the creation was failing on only one of the Hyper-V farms in our environment. The GUID was always the same, it never changed between test runs.
Further checking showed that only one node on the cluster was causing the issue. Whenever a VM is created on this node, the creation would fail. Creating the virtual machine directly from the template in VMM fails as well with the same error. Since this node had the lightest load, it was given the best star rating for placement. The node was healthy otherwise and was not reporting any issues. I migrated all roles off the node and created a test VM, it was created successfully. The problem had to be caused by one the hosted VMs.
I checked the VMM logs for any errors that popped up when the roles were being migrated. VMM had failed to refresh one of the VMs.
The GUID was the same. I created another test VM on the node that now hosted the roles, the creation failed with the same error. VMM was unable to refresh the problematic VM and this was causing it to be unable to create any new virtual machines on the node hosting this VM. When I tried checking this virtual machine properties through VMM, the console was unable to display the properties and it crashed.
I checked the VM through Hyper-V Manager. It was running on a differencing VHD with a direct link to the root VHD.
This VM was supposed to have a snapshot which was taken around November 2013 when the application was updated. Hyper-V Manager was showing that the virtual machine had no snapshots though. It seemed VMM was having problems refreshing the virtual disks of this VM or viewing its properties when the snapshot configuration is missing. I backed up the VMM database and executed the following SQL query to remove orphaned objects as detailed in this Technet article.