The best advance in my life as a software developer recently is having to spend less time building (and re-building) plumbing. Much has rightly been made of the move to cloud computing and the virtualization of computing/storage, but in the last year the trend has continued up the stack to include all sorts of services. The benefit to developers is spending more time focused on your product, and less on the myriad of necessary-but-ancillary aspects of building a webapp.
In my thinking about this trend, I’m making two assumptions. The first is that the scarce resource is developer time, not money. If you have zero dollars to spend then you have to scrounge. The second is that the goal is to optimize for iteration speed, instead of other goals like education or maintaining complete control. In the past I used to (perversely) enjoy learning about sendmail, ipchains and makefiles; now having to wade into those type of issues is a distraction from improving our site.
A few examples of what I’m thinking about:
- SendGrid - I hate the Yahoo! spam filter after how many emails my apps have sent that it has incorrectly flagged as spam. I just want emails to get into someone's inbox, not become an expert in SPF & DKIM. The day after switching to SendGrid from AuthSMTP this wasn't a problem anymore. Love it.
- Chargify / Recurly - Billing and subscription billing in particular is a huge pain. You just want to get paid but have to worry about merchant accounts, vaults, payment gateways, PCI compliance, etc. Oh, joy. Several companies are trying to simplify this process for developers (which Sachin covers well in his Intro to Online Payments post).
- Mogotest - the bane of web developer's lives is testing all the permutations of browsers & platforms that exist today. Firing up VMWare and manually testing pages is painful answer to know if your site works everywhere. We've been using the Mogotest beta and it has saved us hours of development time, and found things we would have otherwise missed. There is a direct correlation between fewer hours spent testing Internet Explorer and developer happiness. While this might not seem like plumbing exactly, having to ensure browser compatibility is a painful tax on creativity.
- Apigee - the web has seen an explosion of RESTful APIs in the last ~5 years. Many are simple to consume with tools like ActiveResource but anyone who built on other's APIs knows you can spend a lot of time dealing with error conditions, latency, analytics, downtime, etc. Apigee is building a way to consume APIs by using their smart pipe between you and the the 3rd party.
- Many more than I can cover: SimpleGeo for geodata storage & discovery, twilio for voice/sms-enabled apps, Zencoder for video encoding, Hunch for a recommendation service or BigDoor for adding game mechanics.
And it is easy to see why: developer’s time is expensive and limited. The biggest danger for developers choosing to rely on other’s platforms is that abstractions leak. Suddenly what had been a simple blackbox requires understanding the inner workings of these domains if something goes wrong or you outgrow the service. Still, for startups it is a smart move to trade some money to buy more time to focus on your core business and customers.
I think there are a number of areas ripe for significant improvements in infrastructure and how developers spend their time:
- SEO - Having developers spend much time attempting to deeply understand Google is a net waste. And yet, Google is how normal people find things on the web, so it is too important to ignore. And the mantra of just "build good content" can be simplistic in practice. It is a difficult problem to solve but there is some recent activity in this space with companies like BrightEdge.
- Understanding user behavior - Collecting actionable data on your users is challenging. Who are your most engaged and what motivates them? What was missing for those that visited and never returned? Tools like Google Analytics aren't auditable, and feedback services like UserVoice may miss apathetic visitors. There are several startups attacking various aspects of the problem such as KISSinsights gathering better surveys (Hiten is the man in this area), Performable A/B testing behavior on landing pages or UserMojo trying to analyze user emotion.
- Debugging and responding to issues - One of the most difficult aspects to web programming is the number of moving parts. While it is a good goal to get to a state of zero errors there is often a steady mix of real issues (ones affecting real people), and those due to strange circumstances (badly formed requests, bots). We use Hoptoad and New Relic that are both valuable but neither completely solve the issue of discerning signal from noise. It can challenging to quickly know if a real problem that requires attention immediately, or if it is just background noise. Services like these could monitor constantly but only notify when levels have changed significantly. In addition to diagnosis based on data trends, it would be great to react to future exceptional conditions automatically (perhaps notifying subsequent users).
- Hiring - While not plumbing per se the recruiting & hiring process can be a huge time sink for small development teams. While it is enormously important to build a great team it can be a serious distraction from customer learning and building the product. The existing models of recruiting and typical job boards both fail to provide enough screening for employers nor sufficient information for job seekers. Especially when you consider that most of the people that your company wants to hire are currently employed elsewhere. </ul>