A family member had been running Goat Simulator on an old Q6600 + HD7770 machine, but found that the GoatZ DLC add-on just wouldn't work. It continually crashed with the dreaded "goatgame-win32-shipping.exe" error.
None of the suggested fixes I could find (verify cache; update graphics drivers; reinstall; tweak some shader settings) had any effect. Although Vista is still officially supported, it may have been something to do with that. Since the machine was dual-booting with Linux, and Goat Sim now has a Linux target, I just installed it on there.
Ran fine using the current AMD fglrx drivers in Xubuntu 14.04. Bit of glitchiness in the water rendering, but nothing that couldn't be ignored.
Really the only issue was the mouse sensitivity in-game. I've found this with other games based on the Unreal Engine, the mouse speed goes way down. I fixed this by tweaking the mouse speed in the desktop using xset:
$ xset m 4/1 0
(Basically, followed the advice from this thread).