It's too much like Oblivion for me if I lower it any more than that. I haven't encountered one-hit kill bosses, but I have encountered some bosses that were too tough for me at my current level. So I leave those till later.