Yep. Paint removal is not that big a deal. Ordinary razor scraper, as for scraping silicone,works fine, or you can get one of the brush- on or wipe-on paint removers. Paint remover won't hurt glass and so long as you use a water soluble one, not hard to wash clean afterward.
I'd stick to water washable types, way easier to clean off afterward.
If the tank has a plastic top and bottom frame, there's a chance that paint removers might do some damage to the plastic or possibly the glue that's used to stick them on. If it were me I'd remove the frames before using paint remover. If you can't or don't want to do that, mask them well with plenty of masking tape.
Removing frames isn't difficult, a putty knife or similar tool and maybe a rubber mallet or a hammer used gently is all that's needed to pry a frame off. They're usually only glued in a few places, and I think it's usually just hot glue, which isn't all that strong.
It's very easy to put them back on using silicone. Fill the frame channel with silicone and gently push onto the glass. Shouldn't be much, if any squeeze out, but if there is some, you can cut or scrape it off when dry. It makes a better seal for the frame than the glue does, because it fills the entire channel, so wicking of water under the frame can't happen if the tank is overfilled.
I've had a couple of tanks that had wicking issues, due to the fact I usually fill them past the point the manufacturer had in mind. So I take the top frames off, scrape out the few blobs of glue and stick 'em back on with silicone. Can do the same with the bottom frame too.