Best spots downtown are East 4th Street and the Warehouse District, which is West 6th/West 9th streets. East 4th is closer, just north of the Q. Actually E 4th dead-ends at The Q. This has a couple bars and some nice restaurants and a comedy club and a bowling alley. Iron Chef Michael Symon has an upscale restaurant called Lola on this street. Also the House of Blues concert club is there.
Warehouse District, which is closer to the Browns stadium, isn't too much of a walk, probably about 15 minutes. That has more of a club scene, though there are some nice restaurants (a couple of steakhouses included). While E4 usually shuts down around midnight, the Warehouse District is open til the state says the bars have to close, which is 2 a.m. Basically, the Flats moved up the hill from the river to this area. It is much less rowdy though.
The West Bank of the Flats still has a couple things open, including a popular gentleman's club. But you'd have to drive there unless you really want to walk.
There's also some good, solid bars right across the street south and east from Progressive Field. Most people don't venture too far from these areas downtown. There are other places to go, but these are where most people congregate downtown.