Obsidian Entertainment - Chapman on Difficulty
Obsidian's Nathaniel Chapman opens his blog account with an article on game difficulty settings:
Anyways, in my opinion games tend to offer the most interesting difficulty options when they rely on tweaking or even adding new core challenges without invalidating the core gameplay. A great example of this is Thief. Thief's difficulty options added new challenges to their already existing stealth gameplay. They didn't choose to increase enemy health (at least, as far as I remember) because that runs at cross purposes to their core stealth gameplay. Instead, they force you to not kill anyone. This makes the game's environment navigation and perception/awareness challenges much more complex, but doesn't really alter the core balance of the weapons and tools.
The reason why more blunt instruments, like just increasing health and damage, tend to fail IMO is that they don't actually make the game more challenging, they just mess up the pacing. I played an ARPG recently that scaled damage and enemy HP and rather than really being more challenging at higher difficulty levels, it just turned into a massive slog. That's something you really want to avoid at all costs... pacing is key to the game being fun, and hard doesn't mean frustrating or boring, it should mean challenging.