Yes, getting a PSP is a great idea if you're into SRPGs. If you ever happen to get one I could make a great list for you.
Now then, I'm playing La Pucelle Tactics on PS2 right now and loving it. I usually play for one or two hours at a time, once a week, and it's been lasting quite a while. Battles are fairly short so you won't have to worry about saving. The role playing (or stat building, as it were) is completely freeform and you've got to put some thought into everything you do. Or just grind. Great music and an entertaining story so far. It's also pretty cheap. I think it's a very good first choice.
There's also Front Mission 4, which seems to be about giant robots, and Front Mission 5, which is Japanese only but received a very good fan translation. You'll need a modded PS2 or a PCSX2-capable PC to play it. Front Mission 3 on PS1, or Front Mission on DS, are also good options.
I was looking for Kartia: World of Fate on PS1 for a while. I don't remember why, but check it out.
Valkyrie Profile: Covenant of the Plume. Unlike other Valkyrie Profile games which are absolutely top-tier aRPGs, this one is a mediocre sRPG. I recommend it for the difference in tone from any other SRPG. The hero cares nothing for justice at all, just gathering (sin) power for revenge on the valkyrie from the first game. There's a branching story, and each playthrough should be pretty quick. It's not the most well-made, but it still manages to be the most interesting. If you haven't played Valkyrie Profile 2 on PS2, get to it. Even if it's not a traditional grid-based tactics game, you'll need to plan carefully to win. Whole game gets harder each time you beat it too!
Knights in the Nightmare: Best game of the 7th gen if you ask me. This is an experimental game for advanced players - it takes an hour just to figure out how battle works. Uncompromisingly creative in its bullet hell/SRPG design, beautiful art, interesting story, some of the best synth music around. Just be warned it will ruin 50% of other sRPGs for you
Also on DS Front Mission, Jagged Alliance, Pokemon Conquest, Summon Night. I haven't payed Final Fantasy Tactics A2, but it's there.
On GBA, try Ygddra Union and the Fire Emblem games. They're pretty hard, but very rewarding. PSP version of Yggdra Union has way better music though.
These should all be cheap except the GBA fire emblem games. Enjoy!