The cheapest means of traveling is by foot and, assuming you're in good health, you should avail yourself of this method most of the time.
Paris is a very compact city whose tourist attractions are, for the most part, in a fairly narrow band on either side of the Seine. If you plan correctly, on most days you will find it easier to simply walk from one sight to the next.
That being said, the public transportation system is excellant. It has more subway stations per square mile than any other such system in the world (245 stations within the 16 square miles that are Paris proper and another 55 in the close suburbs). The bus system adds to this density.
Metro, RER and bus systems are all integrated and part of the RATP. they use the same tickets and a pass good on one is good on all.
You should note however that there is a zone system that affects the price of travel on the RER and the Bus lines. All of Paris proper is contained within zones 1 and 2 but certain common visitor destinations are outside these zones. CDG airport, for example, is in zone 5.
Taxis in Paris charge according to the day of the week and the time of day with fares varying from .86 to 1.35 euros per kilometer. There is a starting charge of 2.20 euros and a minimum fare of 5.60 euros.
A taxi ride from the Louvre to La Defense (a distance of 7.5 km) would cost something on the order of 10 euros or more if there were no traffic (there is a "waiting charge" that also kicks in when you are stopped in traffic).
I suspect you mistake the geography of Paris and the ease of public transport when you ask for shopping spots "in and around the 16e." There is no practical reason why anyone would confine their shopping excursions to a single arrondissment when all the others are only a few minutes Metro/RER ride away.
A good place to shop is Tati which is a fairly low price store favored by young Parisians. http://www.tati.fr/www/
The "Flea market" at Saint Ouen (Metro: Porte de Clingnacourt) consists of thousands of stalls selling everything from antiques to vintage clothing.
As to food, cheap is a relative term (as is "good" for that matter). The classic area to find inexpensive three course meals is along the rue Mouffetard in the 5th arrondissement. You'll find dozens of places selling three course meals for prices ranging from 8-15 euros at linch and only slightly higher in the evening.
Some personal suggestions of relatively inexpensive places to eat would be:
Auberge Le Pot de Terre at 22, rue du Pot de Fer (5th arr.)http://www.lepotdeterre.com/index2.htm
Crepes a Gogo at 12 rue Soufflot (5th). [trust me...its much better than it sounds] http://www.gogo.fr/crepes/index.htm
Sans Gene 122 rue Oberkamph (11th) http://www.lifeinparis.com/msg/sg-menu.php