Rome's transportation system is crowded at times, a tiny bit scary at others, but you can get to a great many places around Rome on the Metro and Urban Rail system at a very reasonable cost.
In fact, if you're looking for an inexpensive Rome vacation, you can spend a week or two using Rome as a base (or staying in the Colle Albani and "commuting" to central Rome).
Places you can get to on the Urban Rail system include the spectacular ruins at Ostia Antica, Hadrian's Villa to the east of Rome, and the towns of the Colle Albani to the south, including the wine town of Frascati and the wonderfully scenic town of the Popes, Castel Gandolfo.
Click on the map markers to see more.
Ok, right now the map looks exceedingly crowded with markers to Rome's two metro lines that cross at Termini Station (Linea A in red and Linea B in blue) and the black markers for the light rail or urban rail system. Also there are gray markers for bus stations and parking.
The large black markers are interesting stops on the light rail system. From West to East we see Ostia Antica (near Fiumicino on a line that also gets you to Rome's close beaches at Ostia Lido), then the Castelli Romani towns of Castel Gandolfo, Marino, and Frascati. To the extreme east on the map is Tivoli, where you'll find Hadrian's Villa and the Villa d'Este.
The purple markers indicate a transfer station. The biggest of these is, of course, Termini Station, Rome's central train station, with a taxi stand and bus station in front. On the southwest side of Termini Station you'll find "Termini Laziali" - Regional Railways Terminal, which is the terminus of FR4 - FR 8 Urban Rail lines.
The yellowish markers indicate that the line continues to other towns outside of Rome. In the upper left, for example, clicking the marker for Cesano Station - FR3 tells you the train ends up in Viterbo, an interesting town to visit.
Zooming into the central part of the map brings you to central Rome, which is likely to be your primary focus as a tourist.
Public transportation tickets can be purchased in advance from tabacchis (tobacco stores), newsstands, some bars, or vending machines at metro stations and major bus stops.
At time of writing, the cost of a metro ticket was 1.50 euro (check current prices here). This ticket is good for 100 minutes of travel on the metro, bus, or tram within Rome (The Roma-Lido line is considered to be within Rome, so making a trip to the excavations at Ostia Antica a real Roman Bargain).
Be absolutely sure that you validate and keep your ticket. There is a machine on the bus and tram; your entrance to the metro automatically validates your ticket when you insert it to gain access to the station. There is a steep fine for not having a ticket or not having a validated ticket.
Rome Taxis - Read this before you listen to the touts at Termini Station.
Rome Discount Passes and Cards discusses some options that might make Rome attractions and public transportation cheaper.
Visiting Rome on a Budget by Katie Parla talks about the variety of transportation passes available and gives you a few budget lodging tips as well.