Oblivion
Its undoubtedly the easiest Elder Scrolls. I beat the game severely under leveled. I ran through most of the oblivion portals. It was personally the shortest main line, and side quests for me as well.

I have never found any of the Final Fantasy's to be especially hard. Nothing a little grinding couldn't take care of anyways. Same goes for Pokemon.