The failed economics of our software commons, and what you can about it right now

[   econ   tech   ]            

Progress in the software industry has been crushed under the weight of technical debt accumulated over the last 40 years. The tools, languages, libraries, and platforms we all rely on suffer from massive incidental complexities responsible for billions of dollars in wasted productivity. Anyone who spends any amount of time as a developer and loses days, weeks, or months implementing some library that “should” exist or working around some shitty software tech to implement a task that “should be simple” knows, deep down, that this can’t be the best humans are capable of. It’s not that we don’t know any better. Given more time and resources that could be devoted to tasks other than shipping the next release, most programmers at least have ideas for how to do things better. But we never seem to get these resources because of the failed economics of our software commons. I’ll explain the problem and tell you can what you can do right now to start changing the game. (Spoiler: check out snowdrift.coop, and after you’re convinced what a great idea it is go fund their launch.)

What’s the problem? In software, everyone is solving similar problems, and software makes it trivial to share solutions to these problems (unlike physical goods), in the form of common libraries, tools, etc. This ease of sharing means it makes perfect sense for actors to cooperate on the development of solutions to common problems. To take a couple examples, a large number of companies need web serving software, and a large number of companies need an implementation of fundamental encryption protocols like SSL/TLS. Given the massive number of companies and individuals that rely on this software tech, obviously, getting those who benefit to pitch in to appropriately fund and/or staff development of these software commons projects makes sense. Obviously, it would be crazy to staff such critical projects largely with a handful of unpaid volunteers working in their spare time. Er, right?? Yet that is what projects like OpenSSL do. A huge number of people and businesses ostensibly benefit from these projects, and the vast majority are freeriders that contribute nothing to their development. This problem of freeriders is something that has plagued open source software for a very long time. As we’ll see later, carefully considerating the underlying game theory and doing a bit of mechanism design leads us to much better equilibria that can result in important common software projects getting the resources they finally deserve.

As an aside, it’s actually quite amazing what a distributed group of volunteers can accomplish while working in their spare time. That open source projects are as good and as successful as they are is a testament to what passionate, talented people can accomplish in little time when working on something that inspires and motivates them. It also inspires me to think about what our civilization could accomplish if we actually devoted real resources to building out our software commons—the collection of languages, libraries, tools, and platforms that are freely available and which large numbers of people rely on.

To give some sense of just how backwards our civilization’s resource allocation is, here are a few sobering thoughts:

• In the past year, Google has likely dedicated more developer time to optimizing font sizes, colors, icons, and other trivial visual details of their ads and apps, than the OpenSSL developers have spent over the lifetime of their project.
• The website for Arby’s, a restaurant which sells horrible roast-beef sandwiches, is developed and maintained by a team of well-paid professionals, who do this as their day job. In a week, the development of the Arby’s website likely receives more developer resources than OpenSSL does in a year.
• Various hedge funds and investment banks likely devote more resources to getting trade times down by a few milliseconds (including building a straighter fiber-optic cable between Chicago and NY) than the sum total value of all developer time devoted to to every open source project ever, since the dawn of software.

Why does our civilization devote so much resources to the above tasks while largely neglecting investment in fundamental software tech? Unlike building our software commons, devoting resources to the above tasks is financially well-compensated. Google optimizing font choices and the like helps them sell more ads, which make them more money. Arby’s having a nice website helps them sell more terrible roast beef sandwiches (or at least they think so). And building a straighter cable between Chicago and NY helps traders profit from minute arbitrage opportunities, before others have a chance to do so.

In the case of building our software commons, though, we have a genuine market failure. This isn’t merely a situation where some people dislike the results of a market working as intended to reflect people’s revealed preferences (as is the case for my deep disappointment that civilization spends far more resources on Arby’s roast beef sandwiches than on B.Good’s tasty, locally-sourced burgers). The failure to adequately fund our software commons is due in large part to a collective action problem called the snowdrift dilemma (also sometimes called chicken).

You might be familiar with the prisoner’s dilemma. The snowdrift dilemma (or ‘chicken’) is similar in that both parties may choose not to cooperate even though it appears to be in their best interest. The details are a bit different and it’s a better model for funding of open source software. From the wikipedia description:

The snowdrift game imagines two drivers who are stuck on opposite sides of a snowdrift, each of whom is given the option of shoveling snow to clear a path, or remaining in their car. A player’s highest payoff comes from leaving the opponent to clear all the snow by themselves, but the opponent is still nominally rewarded for their work.

Aside: What distinguishes this from a prisoner’s dilemma is that players are still somewhat rewarded for cooperation, regardless of whether the other party chooses to cooperate as well.

In the snowdrift dilemma, there’s an incentive to freeride and wait for the other person to do the work. There are snowdrift dilemmas playing out all over the economy. The real world is a bit messier and people aren’t classically rational actors, but consider:

• Huge numbers of businesses all rely on bug-free, usable implementations of cryptographic protocols like TLS/SSL. A bug-free, usable TLS implementation is the snowdrift. Businesses have the option of supporting development of TLS libraries, or “waiting in their car” for others to do so. The vast majority freeride.
• Even if each of these businesses contributed 1% of a single developer’s yearly salary to a project like OpenSSL or LibreSSL, these projects would have more than enough money to hire a sufficient number of full-time developers and maintain a much higher level of quality. Perhaps they could rewrite TLS in a safe language and expose a C ABI for widespread use in a multitude of languages! In the under-resourced open source world, this would be insanity and people are always springing up shouting worse is better to rationalize sticking with the status quo. In a world where projects like a common TLS implementation might (very appropriately, given the benefit) receive several million dollars in funding, ground-up rewrites using better technology can start making a whole lot of sense! Isn’t that pretty inspiring to think about?
• Had businesses chosen to cooperate in this way, they likely would have saved themselves money. The direct costs of fixing systems after the heartbleed bug were significant (probably more than OpenSSL gets in developer resources in a whole year), and the indirect costs are huge too (losing yet more user trust by telling them you’ve probably leaked their confidential information again).

But I don’t want to single out TLS. Any time we have a software project which (a) needs to be freely licensed and available for practical reasons and (b) is solving a common software problem, we have a kind of snowdrift dilemma. One of my personal pet causes is developing a better alternative to HTML/CSS. This is a case where the metaphorical snowdrift is R&D on new platforms (which could at least initially compile to HTML/CSS). Like getting hundreds of self-interested companies to collaborate on funding TLS library development, moving away from HTML/CSS is a huge coordination problem. But there’s no need to adopt a fatalist perspective and resign ourselves to whatever awful equilibria occur in these naturally occurring real-world games. By engaging in a little mechanism design, we can change the game for the better.

The solution proposed by Snowdrift.coop is a very simple mechanism of matching pledges. Let’s return to our hypothetical snowdrift. You are sitting in your car deciding whether to help out clearing the road, or wait in your car. What if you could make a matching pledge—that is, you pledge that you’ll help clearing the road so long as the other driver does their share. If the other driver makes a similar pledge, and we have a way to enforce that people will follow through on their pledges when they align, then all parties can announce their intent to cooperate in a risk free way. Such a mechanism lets the game find a better equilibrium, in which willingness to cooperate is properly reflected in outcomes, rather than being actively discouraged by the payoff matrix of the game.

In the case of funding our software commons, Snowdrift.coop lets patrons make matching pledges of a form like: Each month, for every 10 people who pledge 1 ‘project share’, I agree to donate $0.01. If you end up being the only one pledging, your share is worth nothing and no money is deducted from your account. To the extent that others join you in pledging support for a project, your contribution grows quadratically, up to the amount you have in your Snowdrift.coop account which you of course control. You can pledge multiple shares to a project—read all about it here and follow some of the links there for additional info (they’ve done their research). Besides changing the game to better incentivize cooperation, this mechanism has the beneficial effect of consolidating resource allocation. That is, it leads to a smaller number of well-funded projects rather than a large number of projects all hobbling along with insufficient resources and funding split randomly between them.

Aside: I have some ideas for how to further improve Snowdrift.coop’s mechanisms, which I’ll discuss in a later post.

If you’re convinced, consider funding their launch. In fact, you might have noticed that funding the launch of a common fundraising platform is itself a snowdrift dilemma! Once launched, Snowdrift.coop will become self-hosting and will fund itself as a regular project on the site, rather than taking a percentage of everyone’s transactions as do sites like Kickstarter. Come on, how awesome is that?

Note that the Snowdrift.coop developers are modestly asking for only $3k to cover some legal fees. However, I’d love to see them raise several hundred thousand dollars or more, enough to not have to worry about funding for the forseable future while they build out their platform. If something is worth doing, it’s worth paying people well to do it, to ensure the developers can give it their full attention and live a comfortable life while building something that can benefit us all.

So please, dig deep into your hearts, and this holiday season, fund the launch of Snowdrift.coop. If we all pitch in, maybe we can reallocate some of those developer resources away from optimizing font colors at google, building websites for bad roast beef fast food chains, and shaving another few milliseconds off HFT trade times!