On Memorial Day, I ran a session of the D&D Next playtest with Kainenchen, Stands-in-Fire, Samhaine, and two more friends. If, like my players and me, you have never before played through the Caves of Chaos, then either turn back or be comfortable with some minor spoilers. This will be your only warning. In a seven-hour session with a rule set no one had played before, the party cleared the kobold area of the Caves of Chaos (avoiding the encounter with 40+ kobolds), and fought two groups outside of the caves: one group of orcs and one group of human cultists. The kobold cave represented four distinct combats, one of which took less than a round thanks to my unwillingness to roll 18 saving throws against sleep.
Also, Mearls has posted about feedback so far here. I'll just say that the things he clarifies - hill dwarf using d12 rather than d10 for HD, cleric using d10 rather than d8 for warhammer, presumably the same for halflings and rogue weapons - these are things that looked like typos at the time, and the whole playtest would have been a lot clearer if some guidance on the baked-in bonuses had been included somewhere. Also, while they're pretty much okay balance-wise, I don't really like this benefit to picking a class that my race favors. Actually, I'd be fine with absolutely no favored-class style of rules, because it's such a short step from there to that race/class pairing being the only one that feels valid.
Our first observation, then, is that it's a pretty speedy system at first level. Players found it reasonably easy to understand what their characters could do. This is a strength it shares, more or less, with 4e, and I would venture to state that it may do this better than 4e did. There were a few specific abilities that weren't all that clear, but I think we puzzled our way through them - digging up how many reactions a character got in a single round, for example (answer: 1).
The much-discussed advantage/disadvantage system really couldn't be much faster to resolve. It is, however, quite swingy. It wasn't absolutely true, but it felt very much like you'd never miss with advantage, and never hit with disadvantage. The rogue's sneak attack ability, which is pretty good at first level and gets a lot better thereafter, relies on advantage, and it feels balanced only because advantage is not something a player can currently get every round (since there are no flanking rules). The Guardian theme's "tanking" mechanic - using a shield to protect one adjacent target per round - was hugely effective and felt very natural to us, less gamist than the marking mechanics of 4e (which bothered some of the players, and didn't bother others).
Our group strongly prefers having a gridded map with minis, rather than the theater-of-the-mind style that the WotC devs are encouraging people to try out. It's how we've played for, um, twelve years or more in my case, and it's what we're all comfortable with. Similarly, we'd like to see a bit more of a shift toward tactical play and focus on movement. The lack of any kind of opportunity attacks meant that when the party was in something other than a 10'-wide hallway, the bad guys could maneuver to attack the squishier members of the party with impunity - not great given the very low defenses of the wizard, and the very little that the wizard can do about it. Also, every character can split up movement before and after attacking - what we called "every character has Spring Attack." This is something else we might well do away with in seeking a more tactically-driven experience.
We liked the loose definitions of the skills, to the point that some players expressed concern that that would be reduced in future rules releases. One character has Survival, one has Wilderness Lore, and one has Nature Lore. Without the rules ever spelling out what these do and just relying on interpretation, it meant that we had three characters using their skills for basically the same tasks. I didn't regard this as a problem.
The four pillars of character creation - class, race, background, and theme - are great for making more rounded-out characters. The fighter has some amount of training in something more than swinging his axe. This particular fighter is good at two things that 3.x fighters would have had an awfully hard time doing well: Perception and Survival. I'm not sure, but it looks like fighters could take just about any background, and thus have a skill list completely unrelated to "fighter stuff." In 3.x, backgrounds gave a one-time skill bonus of a few points, but because they didn't turn those skills into class skills, the bonus quickly became irrelevant as DCs scaled up. Since skill values barely scale at all in this rule set, the +3-or-so from backgrounds looks like it should stay relevant for a long time, and I like that.
Adding 4e-style at-will spells to Vancian spellcasting made those characters much more fun for my players. The wizard has two good options for attack with at-will spells, while the human cleric has one. I liked that there were two different 1st-level healing spells, one that took up the character's turn (but was much stronger) and one that didn't. I would hope that that pattern continued later on in the spell progression.
The down side is that both clerics felt like they had to save all of their spellcasting for healing effects at this level. The one healing potion they started with and the one Hit Die of "second wind" healing that each character had weren't enough to keep them going once they'd taken a few hits - of course, it didn't help that several of those healing effects rolled a 1 on their respective dice. The 25 gold to make a new healing potion was good for keeping the game gritty, but given how tough it was to gather much in the way of treasure, the players felt like they needed to collect the chainmail off of the elite Dragonshield kobolds to sell back in town. While I feel that there may be things that are overpriced in the economy, I liked the feeling that the players primarily wanted to amass treasure so that they could purchase supplies rather than upgrades, particularly since the rules explicitly rule out purchasing magic items other than healing potions.
I'm concerned, though, that the ongoing game will be too heavily driven by using healing potions for most of the game's out-of-combat healing, since there's no equivalent of healing surge value to stop players from sucking back a basically endless number of healing potions - just a translation of what I saw as the wand of cure light wounds problem in 3.x. I might try allowing players to add up to of their HD to the value of a healing potion, but any healing potion that isn't improved with spending an HD heals even less than 1d8 - really maybe 1d2. The 1d2 I'd leave in just because the logic of the potion not working at all bothers me, and they're still useful for stabilizing the dying if a healer's kit isn't an option.
The oddities of the armor chart didn't take long to present my players with other options. If you had just captured chainmail off of an enemy cultist, and you really wanted to upgrade your scale mail or your studded leather to a chain shirt, what would you do? Well, my players decided that they'd ask the blacksmith to cut the full suits of chainmail down by clipping some of the length off of the hauberk and not wearing the leggings. I could think of no reason in the world that I should stop them from doing this, so several players netted mundane gear upgrades toward the end of the session.
I'd like to see every character have two or maybe even three different at-will attacks they can make. These should have different situational uses, and shouldn't ever be better than what the character can accomplish with a stunt (especially a difficult-to-repeat stunt). There are just plenty of times when no more interesting stunt presents itself, and I think it would be cooler if the character had a few basic options to choose from rather than a plain ol' basic attack. I realize this concept is anathema to OSR folks, and possibly even 3.x/Pathfinder adherents, but I felt that having access to multiple at-will attacks did a lot to make even a "normal" round be a more interesting choice.
I could offer direct commentary on the Caves of Chaos module, but since I never run published modules, I don't feel like that has much bearing on how theoretical future D&DN games of mine would go. It was, at least, an interesting look back at a style of play that wasn't what came naturally to me when I was teaching myself to DM, and wasn't what anyone else ran for me on the rare occasions that I PCed early in my gaming career.
In summary, I like what I see in D&DN, and my objections are things I'd be comfortable houseruling because they don't feel like they'd have a lot of unanticipated consequences elsewhere in the rules. I'm excited about the next playtest release, and the next after that. If we play a second time in this playtest release to get a little deeper into the Caves of Chaos, then I'll certainly post about it.