I have, from time to time, done some freelance design and writing in the general area of D&D (starting in 3.0 and continuing through d20 Modern and 4e), so while I am far from an authority on matters of the Open Gaming License and its hollow look-alike, the Game System License of 4e, these are things that have affected me directly. Now that 5e has been announced, there might be room for hope for a shift back toward the robustness of the OGL; I'll explain the lessons that I think can be learned from the history of this topic. If you've been deeply invested in D&D for the past twelve years or more, this will be little more than a rehash of things you already know - and if I'm just flat wrong about this history, you're welcome to tell me.
It's like this. Back in days of yore, a guy named Ryan Dancey convinced the necessary set of people that the D&D Third Edition rules could be the foundation of a much broader base of gaming if the use of the rule set were opened up to absolutely anyone who wanted to work with it. In theory, this would serve WotC's interests by getting more people than ever before to buy the core rulebooks and any later rulebooks that WotC added to the SRD. Much like other Open Licenses, innovation benefits even the originator of the product, since they might be able to lift ideas from other products to improve or inspire their own work.
This caused a staggering boom in third-party publishing. I would guess, though I couldn't begin to prove it, that there were more people actively publishing game material in the first several years of the OGL than at any other time in the history of tabletop gaming. There were really very few rules on what you could or couldn't do in your work during that time. Just judging from the way things developed, though, I'd guess that (in WotC's eyes) a lot of that freedom led to undesirable consequences. First of all, there were companies that republished the core rulebooks in new formats, such as smaller perfect-bound books; this meant that WotC wasn't earning money for their core rules the way they'd expected. Though I could be wrong, I don't think they worried quite as much about companies that were actually doing their own thing in creating new classes. The other big thing was that not everyone operates on the same standards of taste - the Book of Erotic Fantasy caused a shitstorm, and I would guess that it was the direct cause for the 4e GSL to impose strict controls on what kinds of content people could create.
Why did WotC even care about this product? Well, part of the whole deal with the OGL was that if you followed their rules, which wasn't all that hard (though there were some irritating minor cases that I'll get to in a minute), you could put a "d20 System" trademark on your book. This gave fans useful information about the book, and confidence that it played within the sandbox that they understood. But WotC didn't like the idea of this product, labeled "For Mature Audiences," showing their trademark - let's skate calmly by the irony of that same company publishing the Book of Vile Darkness. They expressed that displeasure in the short term by requiring the BEF to remove the d20 System symbol and present only the OGL symbol. In the longer term, they put much stricter rules in place when they created the GSL for 4e.
The other thing you couldn't do with the OGL was publishing products that used character-specific names, such as Mordenkainen or Bigby. On the surface, this seems fine: just change the names, right? I mean, as a third-party publisher, I definitely don't plan to write adventures about Leomund, Otiluke, or Rary. Except that 3.0 and 3.5 were published with these names attached to numerous spells in the spell list, and adventure writers needed to be able to refer to those spells in stat blocks, treasure hoards, and so on. For a long time, third-party writers fumbled with a standardized way to refer to those spells, since just cutting out the forbidden name sometimes creates names that don't have enough information to clue readers in to what they came from, such as the sword spell (formerly Mordenkainen's sword). When publishing class spell lists, it was even worse, because changing the initial word changed the spell's position in alphabetical order.
Still, times were good for a lot of companies, and times were enough-to-survive-on for a much larger number of publishers. A lot of that third-party material wasn't great, in keeping with Sturgeon's Law, but that's not really a problem in itself. Cream rose to the top; people made money, lost money, learned through mistakes, and so on. I don't really buy arguments that shoddy third-party products damaged WotC in any way.
After the cut: the GSL, DDI, and what I'd like to see in the future.
WotC announced that 4e, and shortly after announced the GSL. This much-maligned license was, as I've suggested above, much stricter in what third-party publishers could and couldn't use. Initially, it also cost $5,000 to use, sort of like devkits for game consoles. This fee was roundly lambasted, and rightly so; such a fee would be an insurmountable barrier to entry for a great many third-party publishers, who were writing more for the love of doing so than because of the great profit margins. Charging that much money and offering a much more restrictive license than the OGL might have been reasonable from WotC's perspective, somehow, but a lot of big fish in the small third-party-publishing pond balked at this. Their expectations had been set in the era of the free-for-all OGL. The horse, as they say, was out of that stable.
Before too long, WotC reversed course and removed the price tag from the GSL, but the other restrictions on use were severe. A relatively small portion of the Monster Manual was open to third-party use. Any question of cosmology that was answered in the core books couldn't be contradicted - a kind of boundary on the imaginative sandbox that hadn't previously existed. These and other restrictions convinced publishers (sooner or later) to look elsewhere - such as sticking with the OGL and Paizo's Pathfinder, or going still further back and publishing retroclones.
There was one more thing that WotC did to make it clear that third-party publishers weren't welcome. They released DDI, with its Character Builder and Adventure Builder. These were designed in a way that made homebrewed material difficult to implement and clumsy once it was entered, and third-party material was not available. DDI was the solution to the complex choices of character creation and character advancement, because it compiled all WotC-issued choices in a single list. At the same time, though, it was a painfully efficient means of stifling third-party and fan innovation. Once my players were relying on DDI for their characters, that was that - but I've noted my problems with DDI in this blog before.
I do think it reasonable for WotC to want to make money off of every hard-copy and electronic copy of their core rulebooks. There has to be a workable legal middle ground between "you can print your own Player's Handbook if you want" and "you can't change any significant part of the system." Obviously, though, a company has to be able to reprint some rules if they can change any rules, if only to be clear about what they're not changing (without the rulebook reading like a change log). One of the best ways to stop third-party publishers from even wanting to create their own reprintings of the Player's Handbook is to beat them to the punch - that is, for WotC to offer alternate bindings of the same document. It's kind of like fighting IP piracy: if you make it easy for people to buy what they want from you, most of them will do that rather than going hunting for that item through alternate channels.
Getting some of the big names in third-party publishing on-board with 5e needs to be a top priority for WotC. Regardless of sales numbers, the D&D community listens to people whose names they've read on the covers of their favorite gaming books, especially when they don't think that person is a shill. Make it easy for these people to publish, and they'll at least give it a shot - they may have been burned by 4e and found a welcoming home in Pathfinder, but if 5e looks at all promising, they'll at least publish a few things just because they don't want to risk getting left out. (Obviously, none of this matters if 5e flops from other factors, but WotC got a huge amount of negative press from the GSL, and that gave a lot of ammunition to people who were looking for reasons not to buy in.)
WotC desperately needs to find a better economic model for DDI. Its initial release was something you downloaded and installed on your computer, so that you could store your character on your computer, copy it to a flash drive, whatever. The problem here is that every time they released a new book full of crunch, they automatically gave you that content the next time you updated DDI. Why would I buy the book, though, when I can have the content for free? Well, but it wasn't free; there was a monthly subscription. A logical solution, except that you could cancel your subscription, wait six months or a year, pay for a single month of subscription, and get all of the new content that they'd released since the last time you patched the program. The steady access to Dragon and Dungeon content just wasn't a good enough lure to get people to stay subscribed - especially when the crunch offered there was likewise part of DDI content updates.
WotC didn't like not getting paid. This is not surprising. They attempted to resolve this by moving the whole thing to the web; you have to be a subscriber to use it at all, which certainly satisfies WotC's goal. To sell this as a benefit to users, they focused on making your character available from any computer - it's just that it's more difficult to use, and I don't actually need to access my character on other computers, unless I've forgotten to bring my character sheet for the night. Homebrewed material went from difficult to basically impossible.
It didn't have to be this way, and they could do a lot better in 5e. This is something Kainenchen has talked about at length, and she's absolutely right. Charge for content, not usage. Let the basic 5e DDI include only the PH, DMG, and MM content, with check-boxes to handle the player's and DM's choices of modular rules for that campaign. Through some means that would admittedly be fraught with issues on the return-policy front, give those who purchase hardcover books a way to redeem a code from that book to get that content added to their DDI access for free. Those who don't buy the hardcover books pay the WotC website a much reduced price to get that content. (The aforementioned return-policy issues in bookstores would probably make it easier to just charge everyone for the electronic version of the content.)
Moving on beyond those issues, create a content-management template that is equally open to fans and third-party publishers, and allow those fans and third-party publishers to sell their content patches in your store for a small percentage, or on their own websites for no fee. The soul of D&D is that everyone's game is different, and fans now go online to exchange ideas about those differences - monetize that. The model here, for those who don't recognize it, is The Sims, where fans hugely expanded the content available and sold it in EA's store. This didn't stop those same fans from paying EA for The Sims 2 and The Sims 3 and all the content expansions along the way - and I honestly believe that D&D fans are enough like Sims fans to do the same.
When writing the 5e OGL, GSL, or whatever, give absolutely as much of the game as you can stand to give over to free usage. Design the game on the foreknowledge that you're going to be giving content over to those third-party guys, and don't mix product identity with rules terminology, because they won't reinvent your content for their own usage if you don't force them. Let competitors make their own clones of DDI if they want - if they can do it better than you can, start copying the features that are improvements on yours. This is certainly how WoW handled their modding community.
Oh, and - a freely-accessible System Reference Document (as shown here) is such an incredibly useful tool that you could put a little advertising on that page and call it a day. This is just another case of the real message of this post: let other people play with your toys. Focus on knowing the most interesting ways to use those toys and staying one step ahead in developing new additions to the toybox. You might actually get away with making everything modular enough that you never again have to publish a new edition.
But hey, these are just my opinions, many of which I got from Kainenchen. What the hell do I know?