Nocturne is the opposite of Persona 4. Persona 4 is this semi-hip social simulator where the objective is to establish relationships and understand characters. There's a dungeon you go to frequently, but it's secondary to the game's social component. Nocturne is a giant dungeon with almost nobody to talk to. The world is more important than any individual character in it, but it has such a compelling world and battle system that the lack of characters isn't even something you consider.
So Persona 4 is an exploration of characters and Nocturne is an exploration of a world in change. Play them both. Nocturne's better though.
Also I have no idea about Persona 3. I've never played it!!!!! I've heard Persona 3 is not nearly as good from people whose PERSONA OPINIONS I respect, but that 4 is also a similar concept executed much better.
Just My Professional Opinion As A Gamer.
FYI both Persona 3 and Persona 4's normal storyline battles tend towards being easy, but if you set the game's difficulty to Hard, it gets a lot more challenging. Also, both of them have optional dungeons at the end with ridiculously difficult enemies and bosses. Persona 4's optional dungeon/boss in particular are definitely worth your time.
The modern Persona games are definitely not SHINING EXAMPLES IN THE RPG GENRE or anything but they are very enjoyable games if you're in the mood for them. They weren't what I expected after Persona 2 but I liked them.
Positives:
Outstanding art direction (especially 4)
Good music, as long as you don't mind the style
Excellent writing
Unorthodox plot and structure (this applies more to 4 than 3, though)
Good battle system
Negatives:
Rather awful pacing
Lots of 'filler' content to pad out the game calendar
A lot of the interactions between the main character and female characters are ridiculous stereotype bullshit. 4 improves on this but still suffers from it pretty badly.
Lack of direct party control in P3 is frustrating
In my opinion P3 and P4 are just the result of taking one modern trend in JRPGs (story-driven games) to its natural conclusion. Instead of filling the game with prerendered cinematics and using them to tell a linear story, though, they just fill it to the brim with well-written dialogue and intricate plotting. Naturally, some of the other things that might make JRPGs appealing for you take a hit - the dungeons and combat are not quite as compelling as in other SMT games. I think they're also notable for how much control you have over the progression of the characters and the story. Both P3 and P4 offer some particularly interesting choices to the player that have a significant impact on the game world, even if the "ending" is set in stone.
P4 in particular is also notable if you're one of those literature wanks who cares about gender roles and stereotypes.
This article and
this article provide a pretty good overview of the notable things about P4 even if you don't agree with the articles' conclusions.