I was spoiled, grew up in New Brunswick NJ, where punk rock first started showing its face here in New Jersey, so the stores all seemed pretty hip to it. You could get more cool import stuff in New York or Philly but the stores here were not slouches.
Cheap Thrills was the first spot I began buying punk records. That chubby dude from That Metal Show used to work the counter there on occasion. Cheap Thrills had a heavy selection of all the newest punk records coming out (US and UK stuff), and the people who worked there had no problems telling you what was good.
New Brunswick had other shops that catered to punk rock as well. You had All Ears, Flamin' Groovies (which became Captain Video and later Music In A Different Kitchen), and in neighboring East Brunswick you had Crazy Eddie's which had a little punk and a lot of new wave, and Prob Records which also catered to some punk stuff.
Cheap Thrills changed me from saving my lunch money for AC/DC and Queen records to all sorts of punk records.
I lived in New Brunswick from '95-'00. I think I remember Cheap Thrills... if it's the place I'm thinking of I saw some of the early Humpers albums on LP (that another friend bought first!) so wound up getting the CDs. Got a bunch of Shadowy Men on a Shadowy Planet 7"s. I can't believe I remember that!
I liked Sound on Sound (in New Brunswick area) and bought the first 5 Ramones LPs for $5 each in pretty good condition. This would have been maybe 1996? Don't think SoS lasted more than a year or so. Got some Oblivians "Soul Food" and Teengenerate "Get Action" used even though they weren't that old.
Curmudgeon and Vintage Vinyl had their finds (last time I was at VV was maybe 8 or so years ago and it seemed mostly metal). VV had a lot of Crypt, Sympathy and Rip Off Records. I bought the Mummies "Tales from the Crypt" boot from VV (they had 3 copies at one time!) in maybe '97? I remember VV selling a Rip Offs t-shirt (of the "Got a Record" cover) but I refused to buy it cuz it was a shitty print on a shitty Hanes undershirt. Little did I know!
I'd say Flipside Records in Pompton Lakes, NJ broke my cherry. It's still open and although I only make it there once a year (when I'm back home visiting family) I still see some records that I know have been in the bins since the mid-'90s. Dan still runs the joint and sometimes he looks at me like he recognizes me, although truthfully I haven't been a regular since the early '00s.