Final Fantasy 4 was the first FF I ever played and it's one of my favorites. The storyline is great, the music is phenomenal, and the battle system is simple but robust. And Rydia is a very memorable character.

Now having said that, I'm inclined to agree with Clint here; Final Fantasy 5 has excellent gameplay. The class system is stellar. The story is good, not the best the series has to offer, but it certainly has it's moments. Lots of replayablity and you'll have a ton of fun customizing everything to your liking. Want a Warrior that can use Summons? You can have it. Want a Thief that can cast Flare? You can have that too.

Go with FF5 first, and then if you get the chance, go back to 4. That's what I would do.