Progear - I had heard of this game many times, of course, but this morning was the first time I actually played it. A Cave hori always seems a little weird, and this is no exception - but my initial instincts are that I like the setting/mechanics a little bit more than Deathsmiles. Fun game that looks great.
Avengers: Galactic Storm - An odd graphical style and, as far as I can tell, a less than intuitive control scheme. I like the story mode, as it makes it feel like there's a point to each fight and it seems well done.
Capcom Sports Club - I was surprised how fun this is. It reminds me a bit of the Mario Sports titles, but with classic Capcom sprite work. This is a good go to game for a sports fix.
Dead Connection - I really liked this one and played through the first 4 levels before deciding not to credit feed anymore. It certainly tries to do with one stick what games like Smash TV do with two, and mostly succeeds. Love the prohibition-era setting.
Golden Axe: The Revenge of Death Adder - while the game looks ok, it runs noticably slow. I am hoping my impending RAM upgrade (from 1GB to 2GB) will help.

I'm Sorry - this is a weird one, agreed. Interesting sense of humor and fun maze-based gameplay.
Metamorphic Force - this is the (surprise) star of the show this morning - the only recommended game I felt the need to play through to completion on my first go. The story is ridiculously bad, but something about the artwork and characters reminded me of Thundercats. Playing as a guy who transforms into a purple panther and kicks lizard-men's asses is a lot of fun, and the game has decent spritework throughout. Most importantly, it just FEELS like an excellent brawler, and has very crisp controls. This stomps all over Altered Beast, which is the closest comparison that comes to mind.
I'll continue to look into others in time...but thanks for the tips so far!