For a Mac, if you already have a monitor, keyboard, and mouse, the cheapest way is the Mac Mini (starts at $499), although it connects to monitors via HDMI. Other than that, the basic 21.5" iMac would do just fine as everything is built in, that starts at $1099.
My only suggestion is that if you need the computer more for games, then a cheap PC may be the better route to go. Macs have some games available for them, but not nearly as much as there are for PCs. If they are online games played through a browser, then no problem.