Environment
OpenStack Icehouse
1 x controller node = 4xCPU 8GB RAM
5 x compute node = 32xCPUs 64GB RAM

Hosting 5 development environments
Each is roughly 20 VMs of 'small' size


Problem
Pinging a VM will exhibit small periods of no response.
A traceroute displays a single hop to the VM at all times, but will also exhibit periods of being unable to connect.
When logged into a machine via ssh, typing is nearly impossible due to the frequent freezing
When logged onto the VM from being sat at the host, we still see the freezing.

The host displays a minimal load average of roughly 4.00 (over 32 cores)
The host displays a LOT of qemu-system-x86 processes, many of which are over 100%
The VMs themselves do not display any high loads in top


Attempts to resolve
A lot of time has been spent in attempting to debug the problem without any real steps forward.
We did attempt to enable the VHOST_NET_ENABLED flag in /etc/default/qemu-kvm as detailed here, https://blog.codecentric.de/en/2014/09/openstack-crime-story-solved-tcpdump-sysdig-iostat-episode-3/
but that's not helped.