One of the most significant challenges that I've faced in running Mage: the Awakening is working out how to structure a satisfying challenge. For all that the many books of new Mage include chapter after chapter of advice to GMs on different kinds of campaigns one could run, ideas for threats, and advice on pacing, along with reams of setting detail. What it doesn't ever do (that I've seen) is offer suggestions on how to create a viable challenge: something from which the PCs will probably emerge, battered and drained, and having had to show some degree of creative thought, but that will not frustrate or overwhelm them to the point of ruining the game.
Mage is particularly significant for this, as compared to other nWoD lines or most editions of D&D (after first level, anyway) because of the remarkable fragility of characters. A reasonably tough mage has three dots in Stamina, and therefore eight Health boxes. Mages rely very heavily on their armoring spells to keep them alive in a bad situation. In nWoD, a mage's armoring spell is a surprisingly absolute defense, because reducing the attacker's dice pool is a pretty big deal. You might still take some hits, but empirical evidence to date suggests that if you can diminish an attacker's dice pool to 3-4 dice while still having a decent attack pool of your own, that's going to be about all she wrote. (Of course, if you're also a Thyrsus, you can probably heal through, as long as you have a decent mana pool. But I digress.)
I've only addressed Mage so far, but it's a point that generalizes to many other games. D&D 3.0 was the first game I'd played that had clear and definite rules for "okay, PCs of this power level should be able to handle this much of a threat." Challenge Rating is something of a wedge issue for the D&D blogosphere, of course, and one that I could write about for hours; I'll short-circuit that digression by saying that I like the existence of Challenge Ratings and Encounter Levels, but there are flaws in the system that can only really be corrected with very pointed advice to DMs: discussions of economy of actions, for example (cf., "why a wizard by himself is not a great way to challenge players").
Then there's non-combat challenge, such as 4e D&D's skill challenges. I and the rest of the 4e-friendly blogs have written reams of commentary about the good and bad of 4e's skill challenges. The 4e DMG and DMG2 do their best to give cogent advice on writing a discrete scene that challenges the players in a non-combat fashion. It takes an above-average DM to make skill challenges feel like a risk that the players can engage. It's entirely too easy for a skill challenge to come across as "okay, all attacks do one point of damage. Do 4, 6, or 8 points of damage before you (collectively) take 3 damage." It took a lot of development time before WotC and blogging writers started to improve on the systemic side of this model, generally by creating situations where an attack might deal more than one "damage," or a character's action could remove one "damage" from the party. No discussion of skill challenges is complete without nods to Stands-in-Fire and Ben, two excellent DMs who can make the system feel like something more than the sum of its parts. Even if the 4e skill challenge system is flawed, though, it's a better engine for creating non-combat challenges than the vast majority of games offer. Only Technoir and SIFRP match or exceed it, in my experience, and SIFRP only offers social challenges.
Maybe I can get to the meat of what I'm talking about with a few case studies from my own gaming experience.
Mage: How to Banish a Spirit from a Crystal in Thirty-five Dice Rolls or Less
A few months ago in my Mage chronicle, the players received a pouch with five quartz crystals inside it. After a good bit of esoteric investigation and consideration (the juicy stuff in Mage, as far as I'm concerned), they determined that there were spirits occupying those crystals. They determined also that those spirits were Bad News, and they would be better off evicting said spirits with the aid of the Thyrsus's banishing spell. Okay, good so far. Even better, the party works out a way for two other mages in the party to help out.
But... oh. This is actually really hard, in the low-chance-of-success sense. Or, more accurately, low chance to accumulate enough successes to win. On the other hand, the cost for failure is... nil... except that the way the other two mages assist is costing them mana. There were no immediately-apparent threats for me to use as a timer on their actions.
So there are a couple of ways I could have gone with this:
Allow it to succeed outright - treating it as a no-pressure situation that they will eventually succeed. Without pressure or conflict, there's no reason for them to stop trying, so I should fast-forward to the point where they succeed. The compelling problem with this is that any expenditure of resources (such as mana) is made arbitrary.
Have the players just keep rolling until they either succeed or give up. (This is what I did.) This keeps the situation "honest" with regard to player resources, but it was boring for the players involved because it was repetitive and had no particular tension, and even more boring for the players not involved. In the end, the players were glad to be done with it, but I don't think they felt a particular sense of accomplishment, since they'd just kind of brute-forced the situation and the process itself had been boring.
Interpolate extensive further spell-description content, describing how the mage casts the banishing spell and how the spirit resists. Introducing new weaknesses to a PC's power just so I can make a scene more interesting is not really kosher, to me. The player's expectations are built on that spell's description; if I wanted to change it, I should have done so when we were distant from the situation and the player could adjust his planning. The other problem with this is that I don't think my players are interested in engaging with the magic system on that level and would find it at odds with the rest of the chronicle's approach to magic.
Give the spirit an additional way to resist, perhaps a custom-created Numen that lets it retaliate against any who attempt to banish it (turning the situation into a single-player combat: not great), or against any who fail to banish it (not a combat situation, but creating tension nonetheless). The primary reason I didn't do this at the time is that I didn't think of it until today at 3:10 p.m. The compelling problem with this answer is that some GMs look at adding powers to an antagonist just to make a more exciting scene as dirty pool. In general, this kind of GM behavior also runs the risk of creating continuity errors. This idea also feeds back into my post on Multiple Damage Tracks.
Cause a new threat, external to the situation, to erupt. This would have simply caused the character to stop working on banishing the spirit, resolve the new threat, and return to the banishment effort. One of the regular conceits of Mage is that the PCs do have a Sanctum in which they are more-or-less safe most of the time. (Not that the Sanctum is off-limits to enemy attack, but enemies can't be knocking down your door all the time, because that leaves the chronicle without low-tension periods.)
My point, then, is that Mage could really use advice on when and how to create tension. Many systems and writers point out that there's no reason to even roll dice if there's no credible downside to failure, and on the whole I agree with this advice but feel that it's best to introduce threat of failure if you don't really want to just let the player have it for some reason. (Because, well, anything you didn't overcome challenge of some kind to acquire is not a victory.) Most WoD games expect characters to have a home base where they can relax and work on personal projects - Mage (both Ascension and Awakening) take this further and give characters a huge capacity to change the world around them from the comfort of their living-rooms.
D&D: How to Make a Set-Piece Encounter Actually Interesting
One of the best improvements that 4e D&D made on its predecessors is its emphasis on making the physical surroundings of a fight an important part of the fight. Whether this is a terrain power or an area of fantastic terrain that has a passive effect, this emphasis revolutionized my appreciation for how to build a challenge that got the players thinking. It's hard to do so consistently - honestly, if this were easy for me, I would move from systems design to level design here at work - but it's worth the work.
At the other extreme, I've run and played in far too many fights that have nothing interesting going on in the setting. It's a 30' x 30' room with a door in the far wall and a bunch of monsters in the middle - good luck! Obviously I'm not saying such fights can't be fun; we did keep playing, after all. But they could have been something more with the addition of a few more objects or effects to interact with. Certainly there have been DMs who figured this out on their own years before the release of 4e, but 4e was the first edition to give this advice to DMs in a clear and approachable way.
Spirit of the Century does this kind of thing very well, as it encourages GMs to give every location several significant Aspects that PCs and NPCs can exploit over the course of the scene. Because it's all an economy of Aspects, tags, and Fate points, though, it's easy for it to all become too samey, or for players to feel like they're being lame if they can't come up with the next new and different way to exploit some Aspects.
What 4e DMs need even more than the advice to include terrain features is some kind of help in coming up with ideas for terrain features off-the-cuff. Writing a set-piece encounter is just one of those tasks you can make sure you've done before the session. Much more challenging (really, it separates good DMs from great DMs) is improvising a scene with interesting features once the Plan has (inevitably) failed to survive contact with the Enemy. I think a lot of DMs have set pen to battle mat thinking, "Well, it's another random encounter in the wilderness. It's... more of the same wilderness they saw in the last random encounter." Because the players are sitting there waiting for things to get started, the DM doesn't really have time to do any better than some trees, maybe some underbrush, and possibly a river. Sometimes this is even pretty good - the underbrush might be difficult terrain, maybe some of those trees are ready to fall if given a firm push.
My point here is that I want to see books of GMing advice that train people to make not just well-paced plotlines that appeal to multiple player types, but compelling encounters along the way, even when the pressure is on. If the book is part of a specific game's product line (as opposed to being the single most famous treatise on GMing ever written), I want the book to go a step further and organize the information in such a way that I can find it, determine that it's the right tool for the job, and start implementing it in two minutes or less. For that second requirement, I recommend attaching to each specific feature keywords that indicate the right kind of locale, other features that it synergizes with well, and creature types that are aided or hindered by that terrain.
And now, as always, I open the floor to my dear readers: what kinds of advice do you think are missing from DMGs and other books of GMing advice?