Don't mess with Leves until you have run out of Quests imo.

Even then just do Fates to level up.

I found Leves helpful for lvl1-3