Thanks for clarifying. I think you just aren't going to find a really good narrative in these games. They get long winded like Front Mission or FF Tactics and full of cliches and unfinished like Xenogears. My opinion is that companies won't break out of the cliche Jap/D&D mold any time soon for numerous reasons.
1. They don't think that games where the story is the most important element will sell well
2. It would take too long to develop such games to be profitable
3. The game buying Japanese public seem to crave only a few different versions of RPG stories and especially the turn based mechanics (see reactions to the changes with DQ9 and FF12)
4. Game development management is not often structured to reward the risk takers and visionaries. (Capcom's Clover studio and Namco's Katamari broke this trend)
5. The writers/designers for a games story/world are not usually lauded. Producers like Miyamoto, Will Wright, Sid Meier and Kojima get credit.
6. We keep buying them so they keep making them
7. Maybe you should be looking at the adventure games genre.
I agree that the cliches are getting a bit old. Its interesting to note that Valkyrie Profile is a remake of an old PSX game so it can't count as new. Personally, I recently found Breath of Fire 5 and some of the Nippon Ichi games to have a fresher story. Aren't there any games that you've played lately (RPG or not) that had a satisfying story?
I actually knew that about VP before I bought it, but I never played it because I didn't want to pay +$100 USD for a copy or anything like that. I agree with your points but about point 1, I've always thought of RPGs as being a very story driven genre of games, I mean like I said earlier in here "RPGs are Role Playing Games, if the role you're playing is boring what is the point of playing it?" Because I think if thats the case, all you're left with is a bunch of game mechanics and thats it.
I really enjoyed Clover Studios it's a shame they were shut down
But I should also say RPGs were never really my strong genre, I've played a lot of them but usually not too many end up on my favorite list. I am actually more inclined to Adventure games anyway, it's actually my strongest genre probably, that and Fighting, Survival Horror and Action. I've also always been a big fan of Arcade like games, thanks to Sega and Street Fighter, and I guess this explains why I was never too big on RPGs anyway.
As for the last newest game I played that I thought had a good story...it would have to be God of War. I have Psychonauts, Okami and the newest Splinter Cell and a few other newer games I've gotten recently but I've not gone through them yet.