Most of these steps are covered in the README file in the vmgl tar ball
Download VMGL from here
install libXaw, libXext, libjpeg, libXmu devel files for your distro. Warning: compile problems from missing libraries do not stop the Makefile and so keep an eye of out for them as the compile flies by.
On 64-bit, I had to download and patch tightvnc with the patches provided in extra/patches/Xvnc.patch and extra/patches/vncviewer.patch as the provided code would not compile. Everything went smoothly on 32-bit (Ubuntu).
The vmglext module for Xorg is not compiled, but comes as a 32-bit binary in the download. This will not work for 64-bit unless you patch Xorg.
On the host, run make and make install-host, on the guest run make and make install-guest. Requires sudo to install (if in a system path) and to change ldconfig to look for VMGL loadable modules.
Warning: Usermode network stack cripples performance, use Tap for now.
Section "Module" Load "vmglext" EndSection
to your /etc/X11/xorg.conf
Once you start kvm with the -vnc option, point your patched vncviewer to the appropriate display.
When you start vncviewer you will see something in the output like this: Set GLSTUB var in guest to point to port 7001
If you don't see the above message (or vncviewer fails to start) you are probably using an unpatched vncviewer.
In the guest then run: export GLSTUB=<host ip>:7001
As a test, running glxinfo and glxgears should work. The output of glxinfo should refer to the fact that it is using VMGL. If you seen an error like this CR Error(3458): Need to know glStub environment variable, then you haven't set your GLSTUB environment variable properly.
Using X forwarding
Login in to the guest from the host with ssh using either the -X or -Y options to allow X forwarding.
On the guest: run the stub-daemon program and then look in the file ~/.stub-daemon to find out what to set your GLSTUB environment variable to.
On the host: Set the GLSTUB variable appropriately and then run glxinfo or glxgears to test. See the VNC section above if you encounter errors.
Currently, the QEMU/KVM standard SDL viewer does not work with VMGL.