In a recent post on rogues, I discussed some of the issues surrounding 4e's implementation of skill challenges. I've had a few days to give it more thought, and the idea has expanded in my imagination into something I might try to publish as a PDF, someday when I have free time to write. For today's post, though, I want to talk about the strengths and weaknesses of 4e's skill challenge system, as well as changes that would address the weaknesses.
Strengths of 4e's Skill Challenges
Skill challenges represented something new and unusual when 4e first published them: a way to hang a whole conflict on a series of skill checks, with clear victory and failure conditions, and with a huge variety of means to that end. The system has a considerable variety of levers with which the DM can manipulate difficulty. With some creative interpretation, many different skills can apply to a situation, which means that just about every character will have an applicable trained skill (in theory, anyway; in practice this is still a problem).
I am sure that there are comparable systems in other games - FATE seems pretty well designed for something like it - but I can't recall such systems offhand. FATE is awfully good about making just about every skill useful in combat at some point, though - so even if it's a bit off-topic, they get high marks for that! SIFRP also deserves a note for its complicated, but deep and robust, Intrigue system; it is hands down the best social-conflict rules system I've seen, and I've seen a lot of them. (This post is not the place for arguments about whether one should have rules for social conflict at all.)
Weaknesses of 4e's Skill Challenges
Let me start by pointing out that analyzing weaknesses will take more text than listing strengths. This doesn't mean it was a hopeless system; far from it. A skilled DM such as Stands-in-Fire or ShaggySpellsword can make the system sing; some of the best sessions of 4e I played were largely or entirely skill challenges, such as a staged debate between the PCs and wonderfully blustering NPC in Wombat Warlord's 4e campaign. Now that I've gotten those points out of the way...
The exact form of the success or failure of a skill challenge is the first and greatest problem. Every skill challenge that WotC published, as far as I know, used the format of X successes before Y failures, where Y was originally 2, 3, or 4, and was later revised to always be 3. The value of X was the central difficulty lever; obviously, requiring more successes created a lot more chances for things to go wrong, and if PCs are (perhaps unwisely) shooting for Hard DCs and/or using skills that are anything less than the absolute best they could be at that level, just a few average or poor rolls wraps things up pretty quickly.
I am hardly the first person to comment on this, but the really strange thing going on here (and traditionally going on with... basically all skill rolls in every game, ever) is that here, the system chooses to punish failure. Compare this to combat: in D&D, unless there's a system for critical fumbles in place, the worst possible outcome of a normal attack roll is that you make no progress in killing the enemy. With a critical fumble system, the player suffers something worse than a lost turn only a roll of a 1. From this perspective, it looks like a strange choice that many skills in a great many skill systems punish failure, unless the stakes for failure are incredibly low. (It was for these situations where failure isn't interesting that 3e introduced take-10 and take-20 rules, and just about every game printed since 3e tells GMs not to bother calling for a roll if failure isn't interesting.)
Opposed rolls are also... complicated. I don't know of any skill challenges that ever relied on opposed rolls, a point to which I will soon be returning. It's interesting, though, that the closest thing 3e and many other games have to a skill challenge structure is just a sequence of opposed rolls, often Diplomacy or Athletics.
All of this is to say, I think tallying failed skill checks as the failure condition for the whole skill challenge causes a bunch of the other problems in the skill challenge mechanic. Specifically, I've seen groups of players come to feel that they should maximize their odds by performing as many Aid Another actions as the rules allow (or just passing their turn), and let me tell you, that stops being narratively interesting in a hurry. A long while back, I wrote about the problem of skill distribution by class in 4e, and that problem is trivial in comparison to the problem of skill distribution by class in 3e. Skill challenges that don't go out of their way to include all classes are quite likely to leave the fighter, among others, with no apparent way to contribute, while skill challenges that do often come across as contrived (because, well, they are).
Another significant problem with skill challenges is that there's no clear room in the course of a skill challenge for an increase in tension, the intervention of an antagonist, or a change in circumstances, except for the tension provided by the failure track - and given the overall success rate of skills, the players might wind up feeling like they're held hostage by the dice here. Many kinds of conflicts would logically include active opposing characters, rather than simply passive characters that the PCs act upon, but as far as I know the skill challenge rules don't incorporate such characters; the tension remains focused on the players checking skills against fixed DCs, rather than competing in a mechanically direct way with NPC adversaries.
I haven't learned enough of the Cortex Plus system used in Smallville and Marvel Heroic Basic Roleplaying Game to comment on it in detail, having played only a few sessions of the former and none of the latter, but I recall an increasing tension/doom pool under the GM's control, providing the GM with a clear way to increase the threat even in non-combat situations. My thinking is at least related to those lines, though I'm phrasing it a bit more in the "attack vs. defense, damage on hit" style that is D&D's basic engine.
One of my major problems with Mage: the Awakening (well, the whole Storyteller system) was that extended rolls often had no sense of opposition or conflict other than time spent. To make the scene interesting, there needs to be a defined and credible threat of failure. It's fine to have simple checks in a scent that don't have any threat of failure beyond the result of the dice, but if the players can keep rolling and working at it until they accumulate the necessary successes, there's no tension riding on each roll. (A secondary important lesson from this: the failure condition of every skill challenge should feel significant. Not crippling, necessarily, but there should be some cost.)
The final problem with skill challenges - the one that really drives this post - is that they require a lot of preparation and thought to do them right. The 4e rules here tried to be simple enough that DMs could throw together a decent skill challenge on the fly; in my personal experience this was too complicated to handle without stopping the session for 15-20 minutes, which I certainly don't prefer to do.
What I Am Therefore Proposing
I think the idea of a challenge resolved through skill use is absolutely worth saving. I want to get rid of the current form of the success and failure tracks, and instead create hit-point-like progress tracks. This is only better in that it is more granular, and effect-dice rolls fit the rest of D&D's flow a bit better for establishing in the players' minds both the fact of success and the degree of success. To that end, each challenge would have at least two and probably three or more pools of "hit points" as separate tracks: one that Fate deals damage to (the players' situational resilience to failure), one for the players' progress toward their core victory condition, and if necessary additional pools for miscellaneous victory or failure conditions, such as a Time Passing pool that advances by 1d6 per round - perhaps the duration of the challenge is limited, but the DM doesn't have a fixed value of that duration in mind. For physical challenges, the failure track may sometimes just be the PCs' hit point pools.
If this sounds a lot like Fallen London to you so far, you'd be correct. It's the next step that puts a bit more of a wrinkle into things. Fallen London isn't for everyone, but I think it does a great job of framing a wide variety of challenges for the player, beyond violent conflict. Still, the game has a basic predictability that needs some variation if this system is to entertain a party of players.
In addition to the players taking turns, there should also be a place in the turn order for the DM, in which the antagonistic forces of the challenge act. This might be one or more NPCs or NPC groups, or this might be Fate itself - whatever it is that causes climbers, thieves-in-the-night, masters of arcane lore, and the like to err despite their great skill, as a way to introduce narrative tension or complications in mid-scene. Fate is usually trying to thwart the protagonists, but Fate does so without planning or direction. That is, I actually don't think the DM should plan Fate's actions; instead, I think Fate's action(s) on any given turn should be determined by a roll on a table at the start of Fate's turn.
As a side note, I am not typically a strong proponent of the randomized roll to govern the flow of a game. With Fate taking a variable number of turns in a round and offering a wide variety of outcomes, however, the players' perception of fairness in the game is at stake, and something that absolves the DM of the choice may be desirable. Naturally, any DM who approached the system that I'm proposing and felt that only one or two outcomes on the table made sense in context could freely change things up.
The results that come up on Fate's table take the general format of a description of a threat, an attack bonus against a target defense (or saving throw DC), an effect value, and any special occurrence that the action includes. What I'm imagining here is something like: "As you perform the delicate ceremonial gestures to open a teleportation circle, the supporting ritualist's mind goes suddenly blank. Roll +5 vs Will (4e approach) or force an Intelligence saving throw against DC 14 (D&DN approach). On a hit/failed saving throw: 1d10+1 damage to the Undesirable Cosmic Attention pool." I would hope to create at least 10-15 possible Fate actions on a table customized to that particular skill challenge. Some of these actions are "Fate attacks twice this round. Roll two more times, applying both results, but ignore additional rolls of this result," while others are things that can only happen once in the entire course of the skill challenge, and are really very bad (or unexpectedly good).
The problem I haven't yet satisfactorily resolved is the scaling of damage dice for the PCs' skill usage. I've assumed that 1d6 + something (likely an attribute bonus) is the baseline for actions. I don't really want the use of trained skills to derive additional benefit; 4e's bonus of +5 and 5e's bonus of +3 to +7 are enough to make it so the untrained just about shouldn't try; I don't want to exacerbate this with giving them better effect dice as well. Some other possibilities:
The skill challenge has pre-defined a suggested list of actions that are more appropriate to the task, and players receive die-step increases from working out what those are. (This might create replay-value problems.)
The player might attempt harder skill DCs to increase the die step. This creates a lot of risk calculation, which I typically find to be intrusive, and it's not at all a natural thing to fold into the narrative.
Rogues get expertise dice in the current rules. Maybe they can spend expertise dice for additional damage? (But this has the problem of making rogues by far the best at a particular area of gameplay, again so much so that other players need not apply.)
Check results substantially higher than the DC might grant die-step increases. This favors trained skills and rogues with Skill Mastery quite significantly.
Since I don't really like any of the above... something else TBD, or deciding that there is no damage-die scaling.
Difficulty scaling for the challenge I hope to handle with modifiers applied directly to Fate's action roll (shifting the overall result toward the better end or the worse end of the table) and the various hit point pools. The values of those hit point pools are another sticky area of design that I haven't sorted out yet; I'd like player stats to factor in somehow, but without eliminating the threat of failure. I imagine that this might broaden the niche for spells that interact with skill challenges in an interesting way.
My plan, if I can resolve the above to my satisfaction, is to release a PDF of such skill challenges, in groups of 3-5 challenges for the same situation. These challenges are distinguished by differing Fate tables and internal circumstances. This should make it easy for a DM to flip through the book, pick something appropriate, and go with it. If I have a hard time resolving automatic difficulty scaling, I'll just have each challenge of the same type represent a distinct grade of difficulty. (This is like saying that the Monster Manual includes both goblins and hill giants, because goblins are no longer a challenge to higher-level characters. This surprises no one, and in fact we would be disappointed were it otherwise.)
A few types of skill challenges that I anticipate including:
Persuasion of individuals or groups
Investigation and evidence collection
Rituals of every kind more involved than the basics - naturally I look forward to writing these
Many different forms of Man vs. Nature: group climbing, whitewater rafting, crossing rickety bridges...
Trap-disarming sequences (though I need to work out ways for non-rogues to be involved)
Infiltration (extended Stealth or Bluff)
Crafting - it's my hope that with multiple rolls and opposition from Fate, the system will become interesting and nuanced
Research - creating tension during research is always a trick; this is why DtD research tends more toward paying a resource than presenting a challenge, though we've done minigame-like skill challenges as well
Domain rulership - after all, Pendragon's manorial system was a significant inspiration here, for good and ill; also, Birthright's domain action rounds include all kinds of chaos