Big thanks to my beta testers. Your feedback was extremely valuable.
The Web vs. App Stores
There’s no doubt that developing apps is a rewarding experience. Something happens when you see *your *app running on *your *phone. It’s kind of magical. I come from a web development background and when I started writing web apps, I had a similar experience. But writing apps for the phone is different.
One of the biggest differences between writing apps for the web and writing apps for the phone is that the web is more forgiving. On the web, I have control of the server where my code runs. If I find a bug, I can fix it, push a new build to the server and hope that my users don’t notice. If I upset my users in some way (like breaking a login form) the consequences are negligible.
With apps, I have less control. I’m beholden to Microsoft and their approval process. If I find a bug, I have to re-submit my app and wait, all the while my users are experiencing a buggy app. In the app marketplace, users have a lot more influence on my work in the form of ratings, which can have a profound effect the visibility and popularity of my app.
But, for me, this app store model has forced more discipline both with the code I write and how much attention I pay to detail. And no matter how much I test the app myself (in emulators and on multiple devices), I will never find all the bugs. I will also never realize the full potential of the app either because I have only my perspective to rely on as I build it.
Thankfully, Microsoft makes it easy to submit an app as a beta. This process has been extremely valuable for me as I embark upon my first publicly available app, down to zero. I ended up releasing 4 beta builds. Unlike the non-beta Windows Phone app approval process, beta apps go live in a matter of a few hours, which is convenient.
Benefiting from the Beta
So, how did my app benefit from a beta version? The biggest benefit was getting feedback about the user experience. Most of my beta users liked the way the app worked initially, but provided me with some great ideas on how to make it better. One of my beta users (@y2bd) went the extra mile by providing me with a list of enhancements that took my app from good to great.
Another benefit was…you guessed it. They found lot’s of bugs. Some of the bugs I missed where so obvious it was embarrassing. And over the course of 4 releases, those bugs where slowly worked out and now I have a high degree of confidence that the app is stable. I’m sure once it gets out to a wider audience, more bugs will avail themselves, but this is software and that’s expected.
Now, had I not beta tested, my users would have checked out the app, maybe they’d think it’s nice, hit a few bugs and just moved on, or worse, raked me over the coals in the reviews. You do not want the primary consumers of your app to be your beta testers. Expectations are high when it comes to apps and patience and loyalty are hard to come by. Because I ran a beta, my app is so much more refined and less buggy. There will be a lot more work to be done once it’s released into the wild, but I’m coming out of the gate in much better condition.
How I Did It
I conducted my beta by using Mail Chimp (it’s free) along with this website. I set up a list in Mail Chimp for the beta and added the signup form to a blog post. Once that was complete, I tweeted about my beta using the hashtag #wpdev, which is a pretty popular hashtag for the Windows Phone developer community. I also headed over to Reddit and added a short post with a link to my blog post to the /r/windowsphone sub-reddit. With Twitter, Reddit and a few Windows Phone-using co-workers, I assembled a list of 30 beta testers.
The important part of assembling your list is to make sure your users provide you with the email address that they have linked to their phone. Those email addresses are then added to your beta list in the Dev Center. I fired off my first email to the list once my app was available. From there, I received feedback and responded accordingly with more builds and more emails until the app was dialed in. That’s it. Easy and worth it. I’m really impressed with the Windows Phone community and very grateful for all the help.
If you haven’t already, check out down to zero at the Windows Phone Store.