Before you get to a city, make a plan. List all of the places you want to see and things you want to do, and group them by location. If you're planning on visiting the Louvre one Paris morning, then you can easily walk to the Tuileries Garden and Place de la Concorde afterward. Don't avoid the icons either: visiting Hawaiian-shirt hotspots like Munich's Hofbrauhaus is part of the fun. Pick the cheesiest tourist trap in the city and embrace the schlock.
Still, don't be afraid to get lost. Target a safe, interesting area where out-of-towners generally don't venture, and make some time to wander through the streets in the daylight hours. My greatest experiences (and meals) in Venice, Beijing, and countless other cities have happened this way.