Sachin Rekhi's Blog

Google App Engine Task Queues, Push vs. Pull Paradigm, and Web Hooks

Sachin Rekhi — Mon, 24 Aug 2009 18:16:27 +0000

Despite my post last week on the Shortcomings of Google App Engine and my decision to move away from it as a viable platform for upcoming projects, I have been impressed with the overall architecture and design of their experimental Task Queue API.

Google throughout its years has been a leader in interface design and that has been reflected not only in the UI of the products they have built, but the countless API interfaces they have published. Google has made available some of the most easy to use yet powerful API interfaces. A clear focus on leveraging open standards where possible has helped them along the way. Google App Engine is probably the strongest testament to this, allowing developers to quickly build web applications that scale to millions of users on an easy to use Python or Java runtime environment. Their latest experimental design for the Task Queue API in Google App Engine is no exception.

Definition
Before I discuss its advantages, I should provide a definition of a task queue:

Task Queue is defined as a mechanism to synchronously distribute a sequence of tasks among parallel threads of execution. The global problem is broken down into tasks and the tasks are enqueued onto the queue. Parallel threads of execution pull tasks from the queue and perform computations on the tasks. The runtime system is responsible for managing thread accesses to the tasks in the queue as well as ensuring proper queue usage (i.e. dequeuing from an empty queue should not be allowed).

Source: Task Queue Implementation Pattern: Ekaterina Gonina (Author), Jike Chong (Shepherd), UC Berkeley ParLab

Task queues have all sorts of uses for offline processing, including periodically pulling data from third party sources, computing aggregate statistics, delivery of emails to users, etc.

Simple Interface
One of the most straightforward advantages of Google's Task Queue API is its very simple interface. While you can define a set of configuration options, they are all optional. Enqueuing a task for execution is as simple as the following:

#python

from google.appengine.api.labs import taskqueue

#Add the task to the default queue.

taskqueue.add(url='/worker', params={'key': key})

A default queue is provided, though you can easily define additional queues with their own execution options. After being enqueued, the task is run as soon as possible (according to the queue's scheduling options). Optional configuration options are specified in a queue.yaml file, including queue names, rates of processing, and bucket sizes.

Push vs. Pull
While a simple interface is nice, the push vs. pull model of the GAE Task Queue is what makes it really shine. To understand this advantage, let's compare it to another popular cloud based queue solution, Amazon Simple Queue Service (SQS). With SQS, you define a queue and it becomes a central repository for unprocessed tasks. Then you create a set of worker processes (on, say, Amazon EC2 servers) that regularly poll the qeueue to see if there are available tasks for processing. If a worker process finds an available task, the task becomes locked, allowing that worker to process it without other workers having access to it. Once the work is complete, it is removed from the SQS queue.

While this approach provides a lot of flexibility, it requires constantly running worker processes that are polling for available work. In addition, if there is a spike in tasks in the queue, you must also manage the scale up and eventual scale down of worker processes.

In contrast to this mechanism, GAE Task Queue provides a push model. Instead of having an arbitrary number of worker processes constantly polling for available tasks, GAE Task Queue instead pushes work to workers when tasks are available. This work is then processed by the existing auto-scaling GAE infrastructure, allowing you to not have to worry about scaling up and down workers. You simply define the maximum rates of processing and GAE takes care of farming out the tasks to workers appropriately.

Web Hooks
What is also compelling about GAE Task Queue is its use of web hooks as the description of a task unit. When you break it down, an individual task consists of the code to execute for that task as well as the individual data input for that specific task.

The web already provides a great mechanism for this through HTTP requests, their GET and POST input, and the resulting status response. Since in GAE you already define code to execute on an HTTP request, you can leverage the same mechanism for defining the execution code for tasks. As far as the data input, the GET querystring params or HTTP POST body provide suitable mechanisms for providing any kind of input. In this way, a task description is simply a URL that handles the request and a set of input parameters to that request.

This allows you to leverage everything you have already learned in building web request handlers in GAE for user-initiated requests. And more importantly, leverages the fact that GAE has already invested heavily in auto-scaling web request handling. It can simply re-use this infrastructure for tasks queues without having to invent a separate scaling architecture.

Shortcomings
While the overall design of GAE Task Queues is compelling, it suffers from the same shortcomings I mentioned in my previous post. Namely, a given task has a 30 second deadline. That means any individual task cannot perform more than 30s of computation, including getting data from the data store, calling third party APIs, computing aggregations, etc. In many cases, this is fine, since you can simply enqueue many small tasks and make tasks granular enough to always complete in 30s. However, this often does introduce needless complexity in task division and some tasks simply cannot be divided into less than 30 seconds of processing.

Overall, I find the design of the GAE Task Queues compelling and think its a great pattern for modeling queue infrastructure, whether its on or off Google App Engine.

If you enjoyed this post, feel free to subscribe by RSS or Email or follow me on Twitter.

Related Posts

Shortcomings of Google App Engine

Sachin Rekhi — Mon, 17 Aug 2009 18:24:33 +0000

As many of you know, I have been a huge fan of Google App Engine. I love the vision and truly believe its the first real platform-as-a-service as opposed to the other dominant cloud platform Amazon AWS. While AWS has significantly moved the industry forward with on-demand virtualized instances and cloud storage, it has not developed a fully scalable runtime environment comparable to Google App Engine. Sure Google App Engine only supports a very restricted use case and set of technologies, but constraints can be liberating. If the scenario fits for your web app, the freedom to focus on your app and not on infrastructure and scaling is very compelling.

Thus far I've created a variety of small production apps on app engine, including this blog, TuneChimp, and MonkeySort. I am now in the process of embarking on a large project and have been planning on using Google App Engine for it. However, I have run into a variety of shortcomings in GAE that currently and for the foreseeable future seem insurmountable. It has led me to have to reconsider my platform choice for this project and at this point relying on Amazon AWS (or an alternative cloud platform) seems like the ideal option.

For those also considering building applications on top of Google App Engine, I wanted to discuss these shortcomings so that you can make an informed decision when making your own platform choice.

Urlfetch Requests Can't Take More Than 10 Seconds
Google App Engine in Python requires you to proxy all your third party HTTP requests through their urlfetch library. They have created a lightweight wrapper around it to allow python developers to use their typical urllib and urllib2 interfaces. However, the urlfetch library still has a hard restriction that enforces a deadline on any outgoing HTTP request to a maximum of 10 seconds. While in many scenarios this restriction is fine, when building a mashup application that leverages third party APIs (whether its Twitter, Facebook, YouTube, or others), there are many scenarios where you realistically run into this urlfetch deadline restriction. Some APIs allow you to break up long requests by paging, which allows you to get below the 10 second limit, but this often needlessly complicates your code. In addition, some third party APIs are simply poorly written, don't allow paging, or have high latencies that make it impossible to get meaningful results within the 10s limit.

While Google has already increased this deadline once to the higher 10 second limit, they have provided no roadmap or expectation of increasing this deadline further.

Requests Can't Run for More Than 30 Seconds
In addition to the urlfetch restriction, any web request cannot take longer than 30 seconds. This is the entire time allotted to responding to a request. The scenarios for longer requests typically are not user web requests, but instead offline tasks that periodically call to third party APIs to get the latest data and cache it locally or any other kind of offline computation. While Google App Engine has provided the scheduled tasks and task queue APIs to create a nice facility for this, the fact that any individual task can still only take 30 seconds severely limits the possibilities. While in many cases you can smartly divide the tasks to less than 30 second increments, this again requires significant management of task breakdown which may create needless complexity in your application. Or there many even be scenarios which just simply can't be modeled as tasks that must return within 30 seconds.

Since Google App Engine is designed primarily to respond to user web requests and not designed to be an engine for significant offline processing, there is no roadmap for significantly increasing the amount of time an individual task can take.

Can't Open Sockets To Arbitrary Ports
Given that Google App Engine is a constrained runtime environment, it has an understandable limitation of preventing you from opening sockets on arbitrary ports. This restriction is necessary for security and scalability and Google can only be expected to enable these scenarios by providing their own wrapper libraries for each desired scenario. However, this leads to restrictions on important scenarios. For example, if your application wants to incorporate email and connect to an IMAP server, then you have no ability to do this on GAE.

While Google does plan to eventually add additional services to their capabilities, there is no plan for providing a general capability for opening sockets.

Can't Support HTTPS on Own Domain
Google App Engine allows you to launch applications on either their appspot.com domain at a custom subdomain or to host your application on your own domain through Google Apps. However, if you want to handle HTTPS requests, it has to be done on the appspot.com domain. You cannot support HTTPS on your custom domain. This is a significant restriction since it prevents you from providing a fully secured experience for a user on your own domain.

Given technical limitations, Google has not provided a roadmap for when this issue will be solved, leaving people who require this functionality to simply find another cloud platform.

While I love the promise of Google App Engine, each of these technical limitations in the current platform with no clear roadmap to enable these scenarios has led me to abandon Google App Engine for my next project. I plan to continue to monitor Google App Engine developments and see where it goes as I still believe it is a great platform for certain constrained scenarios. But I plan to spend my time now investigating Amazon AWS and Rackspace Cloud.

If you enjoyed this post, feel free to subscribe by RSS or email or follow me on twitter.

Related Posts

Clara Shih, The Facebook Era, and Business Opportunities on Facebook

Sachin Rekhi — Mon, 22 Jun 2009 18:56:49 +0000

Several months ago I had the opportunity to sit in on a guest lecture Clara Shih gave at the Stanford Seminar on People, Computers, and Design. Clara has spent the last several years at Salesforce leading their social networking product strategy as well as developed Faceconnector, the first business app on Facebook that made it easy to integrate Facebook profile data into Salesforce CRM tools. With this insight, Clara recently authored The Facebook Era, a look at how social networks have changed people's behaviors, expectations, and relationships, and the resulting business opportunities it has created.

After attending the seminar, I decided to read the book and wanted to share some of the key trends discussed and the business opportunities that arise from them.

Opportunity #1: Transparent transitive trust opens up social advertising possibilities
Clara speaks at length about transitive trust: the notion that if I have mutual friends with you that I trust, I am by extension more likely to trust you. In addition, if a friend of mine endorses a product or service, I am more likely to respond positively to the brand. Facebook creates complete transparency in both of these scenarios, allowing you to quickly see who are your mutual friends with someone as well as to see a variety of brand endorsements through fan pages, groups, status messages, and a variety of notifications of engagement with various brand applications.

Facebook's enabling of passive word of mouth of brand recommendations creates many opportunities around social campaigns that leverage the higher conversion rate associated with word of mouth referrals. Brands can capitalize on this new channel and user's willingness and desire to associate themselves with the brands they care about to supercharge their previously offline and unscaleable user referral programs.

SocialMedia has been one of the early innovators on social ads, with the Word of Mouth Impression being their latest ad creation. This new ad product attempts to incorporate your friends sentiment around the advertised brand to increase conversion rates of ad campaigns. Appirio has also built a Referral Management for Viral Marketing product to enable brand advocates to easily share their brand preferences with their friends on Facebook.

Opportunity #2: Explicit self expression makes hyper targeting a reality
Compared to the social networks that came before it, Facebook has encouraged authentic online identities and personalities. This has led to users willingly and explicitly expressing themselves on their profiles and streams. With this has come never before available deep data about user's demographics, behaviors, and interests. Of course this creates exciting new opportunities for advertisers to hyper target a set of users based on their interests.

The Facebook Ads platform has provided an opportunity to get at some of this hyper targeted data. Advertisers can now target their campaigns on specific demographics and interest keywords found on user profiles. However, this platform is still in its infancy, only scratching the surface on the targeting vectors that are possible. Hopefully with time Facebook will continue to innovate on their ad platform as well as open up an advertising API to allow third party developers to help with the innovation around targeting. To date third parties have been severely limited by Facebook in their ability to use the data available on user profiles for the purposes of targeting.

Opportunity #3: New forms of casual interactions enable maintaining and growing weak ties
Clara also emphasizes the importance of weak ties on social networks. These are the friends you have on social networks that aren't your closest real life friends, but instead the people you casually or occasionally keep up with. What's important about this class of individuals is that research has shown these are the connections that will be most important to you in terms of business relationships, since over the life of your career these are the people that you are likely to benefit from in one way or another. Facebook enables you to easily maintain and grow connections with these weak ties through casual interactions, including reading their status updates, posting wall posts, sending messages, and more.

Many applications have been developed on Facebook to further encourage casual interactions to grow these weak ties. MyCalendar and Birthday Cards enable you to easily remember your friends birthdays and events and send them online greeting cards. Even many of the social games on Facebook have the affect of allowing you to casually interact with your friends and remind them of you.

Facebook has truly enabled new scenarios and behaviors for people across the world. With these new interactions come many new business opportunities to leverage the social graph to create value. Clara Shih has done a great job of researching and documenting this trend. So check out her book!

Respect for the Criminal Trial Process

Sachin Rekhi — Thu, 14 May 2009 01:27:56 +0000

In a departure from my usual focus on startups, I thought I would take a moment to reflect on my most recent experience in jury duty.

I was selected as one of twelve jurors for a murder trial against a defendant who was accused of beating up his girlfriend and throwing her out of her apartment window in San Francisco in 2005. After an intense 2 week trial and jury deliberation, we today found the defendant guilty. His punishment is still to be determined.

This was my first experience with jury duty and the criminal trial process and I must say I was impressed with the court proceedings and the overall fairness of the trial. I've included my specific thoughts on the jury selection process, the burden of proof standard, and the trial timeline.

Jury Selection
Probably the longest aspect of the entire trial was jury selection. Judge Kevin McCarthy mentioned that for this trial they tried to put together a jury four times prior to finally assembling one. The issue was that this case was projected to take up to four weeks and many potential jurors filed for hardship which allowed them to postpone their service and potentially serve on another case.

After a large enough pool of potential jurors was found that did not file for hardship, the Voir dire process began, which allowed both the prosecution and defense to examine the potential jurors and object to any that may have clear biases or additionally reject any juror for any reason at all (up to 20 times). I was impressed with how they systematically removed anyone with any potential bias that might effect this case. Since this case involved an African American, a victim who suffered from bipolar disease, a substance abuser, and occurred in the Tenderloin, anyone who was found to be biased against African Americans, had personal experience or a close relationship with someone who had bipolar disease, had personal experience with substance abusers, or currently lived in the Tenderloin were all ruled out.

Despite the motley crew that initially showed up to jury duty, I was happy to see that the final jury selected appeared educated, fair, unbiased, and very reasonable.

Burden of Proof
In a criminal trial, the burden of proof standard requires that the jury find the defendant innocent until proven guilty beyond a reasonable doubt. The judge carefully defined the "beyond a reasonable doubt" standard on multiple occasions to ensure we understood that it was not beyond all doubt, since nothing can ever be beyond all doubt, but instead the evidence left us with an "abiding conviction" that the defendant committed the crimes he was accused of. We had lots of discussions during the case and during jury deliberation specifically about this burden of proof to ensure we all understood and stuck to this standard.

In the end we convicted the defendant not based on direct evidence (for which there was none), but instead based on circumstantial evidence that painted a very clear story of what happened. This evidence included witness testimony from neighbors, DNA evidence, autopsy expert testimony, victim's daughter's testimony, a proven pattern of domestic violence, etc.

I'm glad the burden of proof standard exists to ensure avoiding wrongful convictions and that the process was fully followed throughout the trial to ensure a very fair process.

Trial Timeline
The one aspect of the trial that was very frustrating was the trial timeline. The murder occurred Nov 4, 2005 and only today on May 13, 2009 was the defendant tried and convicted of murder. That is nearly 4 years after the crime was committed! And during the first year, the defendant was at large. While I understand they couldn't keep the defendant under arrest without sufficient evidence, it was unclear the police were expending significant resources to both put this evidence together or try him. At the same time, if he had been innocent, he spent a long time behind bars without being tried. Throughout the trial it appeared this case wasn't top priority for the police and mistakes were made throughout, which is extremely saddening.

Overall I definitely saw this as an interesting opportunity to get first hand experience into our criminal trial process and come out of it both respecting our system and better informed.

And now back to our regularly scheduled startup discussions :)

Designing and Testing an Ad Product: 5 Lessons Learned From imeem's Audio Ads

Sachin Rekhi — Tue, 14 Apr 2009 16:15:41 +0000

Andrew Chen asked me to write a guest post on his blog Futuristic Play about some of my experiences monetizing music at imeem. I wanted to link it here, as I imagine many of you will enjoy reading it as well.

Make sure to check out his other posts, as his blog is definitely full of very interesting insights and on my must read list for any internet entrepreneur:

Designing and Testing an Ad Product: 5 Lessons Learned From imeem's Audio Ads

My Muses for Brainstorming Startup Ideas

Sachin Rekhi — Mon, 30 Mar 2009 13:26:40 +0000

As today marks my first day as an Entrepreneur-in-Residence at Trinity Ventures, I'm spending a lot of time thinking about how to formalize my process for starting a new venture. Every startup goes through phases including brainstorming ideas, selecting evaluation criteria, performing due diligence on top ideas, picking a winner, deciding on a corporate structure, putting together the team, evaluating funding options, and more. While the popular press often glamorizes the startup story as an epiphany moment from an opportunistic individual that grows to a successful corporate behemoth, I prefer a much more systematic approach to entrepreneurship (and life in general, for that matter). As I begin this journey myself, I thought I'd document my new venture process along the way, share it with all of you, and hopefully hear from you on your own thoughts.

Often one of the earliest steps in forming a new venture is coming up with the idea. I won't argue that this should necessarily be the first step though, since in my previous venture Anywhere.FM we picked the team before the idea and that was the right choice for us. I also won't argue that it is necessarily the most important step, because while a startup needs a great idea, it's not without great execution that an idea can become a successful endeavor.

I'm often asked where I come up with all my ideas, as I'm typically chock-full of potential ideas percolating in my head. I have a variety of sources of inspiration that help me put together laundry lists of potential startup opportunities. Before I dive into my muses though, I think what's important to realize is that the best idea brainstorming happens outside of any specific brainstorming phase. Good idea generation is like a muscle that you are constantly flexing. It's a mindset and attitude towards the problems you face everyday and your ability to effect change. While some people are innately curious and problem solvers, I still think like any muscle, this skill is something that can exercised and improved.

I'm personally innately curious. But what has been a huge boon for me was learning to program on my first computer, an Apple IIc, when I was in middle school. I quickly learned BASIC and fell in love with the idea that I could make things. In the early days it was often games to amuse myself (which is what typically attracts many young boys to programming), but as time passed I started making more useful applications. One of the issues I had early on was quizzing myself on vocabulary words. I quickly wrote a program I called Vocabulary Master to provide me random word definitions, give me three attempts to guess the word correctly, and tell me the word if I failed to remember it. It was a very simple application, but was the foundation for my strong belief that I can solve real world problems through software. Ever since then I've constantly had a list of potential ideas. Oftentimes when something bothers me or I think that something can be improved, I add it to a list of potential projects (and I have to say that I love the fact that now I have an iPhone always in my pocket to record my ideas). It's this critical lens of looking for ways to improve what you see every day and having the attitude that you can do something about it that I think makes for a great base of idea generation.

Logo for Vocabulary Master

Beyond that, there are specific muses I've used to help jump start my creative juices:

Improving Your Work Life
I think often the best source of ideas comes from your area of expertise and the problems you face on a day-to-day basis in your career. You probably find yourself saying "Why is this so hard? There has got to be an easier way" every so often. It may be a product extension to your existing business's core products, or may be unrelated to the business, but a pain point you feel trying to get your job done. From my experience at imeem focusing on monetization, I've come up with countless pain points in trying to monetize social media and large publisher sites that could be fruitful startup opportunities.

Analyzing Macro Trends
I think another very important exercise is analyzing macroeconomic trends and thinking about the opportunities it creates. What obviously comes to mind first these days is the recession, and I think a lot of startup opportunities will come out of that. Anywhere from helping people save money, find jobs, or tapping into the funds made available through the stimulus bill. But beyond the recession, I think there are a lot of other exciting market trends that are creating countless opportunities. Anywhere from the proliferation of APIs, to the rise of cloud platforms, to the opening up of mobile devices.

Bringing Existing Innovation to New Applications
I believe there remains significant inefficiency in applying new technologies across multiple vertical applications. It behooves a startup to focus at least initially on a very specific application to prove out the model and to provide the best experience. But this creates significant opportunity for other startups to bring that existing technology innovation to new verticals and thus becomes a fruitful source of startup ideas. I often think about how trends in certain spaces can be applied to new segments. For example, can we bring the viral spreading of applications native to Facebook apps to iPhone applications? Or can we bring the successful virtual goods monetization model from social games to non-gaming sites?

Latest News in Your Area of Interest
If you have a specific area of interest or vertical segment that excites you, I think its hugely valuable to constantly keep up with the latest news in that area. These days its so easy to use Google Alerts to subscribe to daily feeds of the latest news, blogs, and search results of your topic of choice. I have these setup for various areas of interest, including open platforms and personal productivity.

Exposure to Startup Ideas
I think it's also very important to constantly be exposed to new startup ideas. When you hear people's ideas and their take on solving them, it definitely helps to get your own mind thinking about related opportunities in the space or at least helps to think critically about how they may have come up with such a solution. I daily skim TechCrunch as a great source of startup ideas. I think Jessica Livingston's Founders at Work was also a great case study of many different startup ideas.

Talk to Existing Entrepreneurs
As I said, great entrepreneurs have this ability to constantly come up with ideas and potential solutions. I've been spending the last month catching up with a lot of my fellow entrepreneurs to not only chat about their business, but more importantly, interesting things that they have seen outside of their own startup. When an entrepreneur is busy executing against a specific idea, he is knee deep in one specific space. That doesn't mean though that he doesn't think about other ideas or have other ideas percolating in his head. I think its often great to tap into this creative thinking to spur some of your own ideas.

If you enjoyed this post, feel free to subscribe by RSS or email or follow me on twitter.

Related Posts

Interesting Metrics From Flash Gaming Summit

Sachin Rekhi — Tue, 24 Mar 2009 16:13:16 +0000

Winners from the Mochis Award Show @ Flash Gaming Summit

On Sunday I had the opportunity to attend Flash Gaming Summit, the first annual conference dedicated to flash game development organized by my fiancee Ada Chen from Mochi Media.

What's often most exciting for me about events like these is hearing different metrics tidbits from the speakers who are knee deep in the space. This conference was no exception, with a variety of different stats shared throughout the day. I've summarized some of the highlights below.

Flash Game Distribution & Advertising Market Size
Mochi Media is by far the largest flash games distribution and monetization network. They reported that they are now seeing over 100 million unique users who play games across their network each month, as well as over 1 billion monthly game plays. In addition, Mochi offers MochiAds, an advertising product allowing flash game developers to embed ads into their flash games. In 2008, MochiAds paid out over $1 million to flash game developers. While Mochi is definitely not the only distribution and monetization player in the flash gaming space, it definitely commands a healthy share of the market and thus these numbers can give you a rough sense of magnitude in the space. While the game play stats are impressive, the advertising revenue generated for developers is still modest and definitely suggests the need for alternative revenue sources like micro-transactions and virtual goods.

Ad Revenue From Portal Versus In Game
John Cooney from Armor Games provided some rough stats on the ad revenue difference from in-game ads versus ads around the game on a game portal page. John stated that the ad revenue from portal ads was at least 4x greater than the revenue from in-game ads. John attributed this large difference to a variety of drivers, including the ability to have higher quality multimedia ad units on a portal page, having 3-4 ads per page, as well as simply getting the user to play more games on your portal.

The message was heard loud and clear by flash game developers that it is going to be very difficult for them to make serious money from their games without building their own portal or partnering with existing portals (through revenue share or sponsorship agreements).

Conversion Rate of Active to Paid Players
Both Paul Preece from Casual Collective and Daniel James from Three Rings shared some interesting stats from their own games on conversion rates associated with paid players (players who purchase levels, virtual goods, etc through micro-transactions). Paul said that for their single player games, they see that 2% of their active players have converted to paid players. Keep in mind this stat is for "active players" who come back to the site regularly, not overall unique visitors. What was interesting was that for his multi-player games, 3% of active players converted to paid players, suggesting their may be some additional conversion lift from multi-player games. Daniel James echoed Paul Preece's numbers, suggesting 3-4% of Three Rings users pay as well.

The key thing they stressed was that only a very small percentage of your users end up converting, because first they need to become active users that come back and are retained by your game as well as be eager enough to pay through one of a variety of ways.

Per User Transaction Value & Lifetime Value
Daniel James went on to share that his company's Puzzle Pirates game was seeing an average transaction size of about $20, with a per user lifetime value averaging $125. This suggests that while its difficult to get people to pay, those that do pay may pay significantly.

Its thus important to ensure for that small number of users, you provide enough reasons and the ability for each user to plow a significant amount of money into the game if desired.

Leaderboards Drive More Retention
While everyone would agree that leaderboards drive additional retention of users who go on to compete with strangers or friends, Jameson Hsu from Mochi Media shared some details on exactly how much additional retention can be expected. Jameson said that they have seen leaderboards increasing retention by up to 30%.

This suggests that adding your own leaderboards or integrating with existing leaderboard platforms is often a worthwhile exercise to drive additional game play and retained users. Given that monetizing through micro-transactions typically relies on highly active users, this becomes even more important for driving lucrative monetization.

Social Games are Where the Money is at
Bret Terrill of Zynga gave a fantastic presentation enlightening flash developers on the large opportunity in social games, both in terms of traffic and monetization.

Bret mentioned that one of their social games, Mafia Wars, was getting 1.7 million daily players on Facebook. Texas Hold-em, in addition, gets over 1.5 million daily players. Bret also mentioned there were social games out there making $20-30K/day in revenue. These large numbers trump what most classic flash game developers would expect in terms of traffic and monetization.

Adam Caplan from Super Rewards, which provides payment and ad offers to fuel micro-transactions, said that they are now paying out over $10 million/month to social game developers, an equally significant figure. And Super Rewards is just one of many providers in the space.

While Bret encouraged flash developers to get into this space, he cautioned them that they won't win by simply porting their game over. They need to design their game with the right mechanics targeted at this audience, including making the game playable in 5 min chunks, appealing to user's desire for status, and focusing on reciprocity loops of gifting and social grooming.

For those of you who missed the event, you can check out the recorded sessions, event photos, tweets, or Mochis award show video.

Optimizing Offer Providers with Sometrics Virtual Currency Manager

Sachin Rekhi — Sat, 14 Mar 2009 01:06:41 +0000

As more and more offer providers enter the incentivized CPA and direct payments space, there is a clear need for a way to easily test different offer providers and optimize between them.

At imeem, I was responsible for evaluating, signing up, testing, and optimizing the various offer and direct payment providers that were leveraged as part of the imeem points virtual economy. I learned valuable techniques and lessons that I thought I would share with all of you as every day I see more interesting startups jumping on the virtual currency bandwagon.

Sometrics Virtual Currency Manager

In order to effectively test and optimize, you need tools to reduce the cost of performing the tests. If its painful to implement the test, record stats, and review analytics, then you'll never get around to doing it.

For optimizing offer providers, I decided to leverage Sometrics Virtual Currency Manager. At the time it was really a decision of whether to build my own custom tools in-house or to leverage their new tools. If I built the tools myself, I knew I would be writing custom hadoop jobs to get the analytics that I wanted. I knew that I wouldn't waste the time to create pretty graphs and would ask myself every time I wanted to track a new metric whether the cost of implementation was worth the effort. And I knew if I built my own A/B testing and reporting tools, the shelf life of the tools would be limited since I was going to optimize for a couple of months and then likely shift my valuable time to other projects. So my decision to use Sometrics was easy in that it came built in with the most common metrics I wanted to track, pretty graphs, and easy to use data viewing\manipulation tools that we have all come to expect from every analytics packages. And an equally important plus was the tools were free to use!

In many ways, Sometrics Virtual Currency Manager functions like your typical ad server (Google Ad Manager, OpenX, DART, etc). It allows you to drop a single Sometrics iframe tag on the page you want to render the offers. Then from within the Sometrics dashboard, you can easily add providers. To add a provider, simply make sure you have signed up and signed the terms and conditions of the offer provider, created your app or campaign within the offer providers interface, grab the iframe code that they provide, and drop it into the Sometrics dashboard. Where this gets more sophisticated than an ad server is in the implementation of the callbacks. After an offer provider calls you back to report an offer completion, you simply send a request to Sometrics servers so they can record the completion as well. You thus serve as a proxy for Sometrics collecting the data it need to provide a full picture of offer impressions to completion to revenue. Once you do get the hang of this, adding a new provider shouldn't take more than a couple of hours to have up and running.

Next you can specify the rotation frequencies of your offer providers. It makes sense to initially give them equal percentage rotation ratios or if you have been using one for awhile, give that one a higher rotation ratio as you test the others. By default you specify global rotation ratios.

After that its times to track performance. Sometrics automatically updates the dashboard every couple of hours to show you the latest impressions, offer completion, and revenue stats on a per provider and total basis. This allows you to easily see how each offer provider is performing over time and adjust frequencies as necessary. One thing to keep in mind when you are running these tests is that its best to run the tests in parallel (as opposed to one after another). You also have to run the test long enough because you will typically see a high eCPM rate the first couple of days a provider is added because this brings new fresh offers to your audience that they will quickly take advantage of. However, if you run it for at least a week, you should see that spike fade and get to a more realistic longer term performance. And that's pretty much all there is to it! Sometrics provides a quick and easy way to perform your testing.

The Sometrics Virtual Currency Manager, however, is not without its issues. It's still clearly a beta product on an early rev. For example, the dashboard doesn't automatically show you the eCPM calculation, leaving you to calculate it yourself. However, given than the target optimization metric is eCPM, this is something that should definitely be built in. In addition, Sometrics has a poor solution for allowing you to specify per country rotation ratios. You may, for example, find that one provider is working better than another provider in a certain country and wish to give the better provider a higher rotation frequency in that country. While Sometrics does provide a facility to do that, its needlessly complex to setup and requires too much configuration.

All in all though I would still recommend it as the Sometrics team assures me these issues will be addressed over time, its the best solution on the market, and in many cases will win in a cost\benefit analysis of building it yourself.

Drivers of eCPM Lift
After spending several months testing and optimizing offers, I started to formulate thoughts on the drivers of eCPM lift. I've cataloged these drivers in prioritized order in terms of greatest resulting effect on eCPMs.

Negotiated Rev Share. The most important driver of eCPM lift turned out to be not product related at all. It was simply your ability to negotiate a better revenue split between you and the offer provider. I've seen the split be anywhere from 50 - 90% net to publisher. By running a performance test across multiple providers, it puts you in a much better position to negotiate higher payouts from each of the providers who are looking to win your business.

Direct Payment Options. While the percentage of completed offers coming from direct payments may not be that high, the total revenue coming from direct payments tends to be fairly meaningful. Thus the number and quality of available options has a material effect on eCPMs. It's important to look at both credit card providers (Paypal, direct credit cards, etc) as well as mobile providers (Zong, Mobillcash, Paymo).

Front Page Offer Optimization. While it turns out that a lot of the offer providers in the end have a lot of the same offers, most users will never see the bulk of offers. Most users only see the first page offers or the first page of each categorized section of offers (free, popular, mobile, etc). Because of this, its very important that the offers shown on the front page are appropriately targeted and fresh. Some offer providers are much better and much more sophisticated at this than others.

Offer Wall Latency. We've had issues with some offer providers that had latency issues with loading their iframe. Users on social networks are jumpy and will quickly bounce if the offers don't show up immediately. This resulted in significant loss in eCPMs for certain providers. Just as Amazon and Google have found, latency costs real dollars.

International Offers. The mix of offers available for the countries that your users are in is important. This is an obvious one. The thing that is somewhat surprising is how low this is on the list of drivers. There are definitely noticeable differences in the number of available offers on a per country basis between offer providers. But it turns out that all international monetization is often a fraction of US monetization. Thus even significant percentage increases in international monetization ends up being much less meaningful on a total revenue basis. This obviously depends on your mix of international countries and thus your mileage may vary.

Offer Landing Page. There are ways of enhancing the completion rates of offers (which tend to be very low). One enhancement that some offer providers did is put the resulting landing page for a specific offer in an iframe and place a small toolbar at the top explaining exactly what needs to be done to complete the offer. I saw a meaningful increase in offer completion rates from offer providers that did this.

I hope that provides some valuable techniques for optimizing your own virtual currency offer and direct payment providers. Check out my related posts below for more thoughts on the space.

Related Posts

Protect Yourself with the Corporate Veil

Sachin Rekhi — Tue, 10 Mar 2009 02:18:22 +0000

While I am a big believer that entrepreneurs should spend the majority of their time focusing on getting a quality product to market, one piece of overhead that should never be overlooked is incorporating or forming an LLC prior to product launch. To some this is obvious. Of course you setup your corporate structure before anything else. But to hackers and hobby programmers this may not be their first instinct. Sometimes you have a hobby website that just starts taking off and you never planned on it being a real business. But then it starts to become one and you may not have the protections incorporation affords.

Take a look at this recent case, EMI vs. Seeqpod. This case has received a lot of press lately not only for the fact that EMI has joined Warner in suing Seeqpod, but EMI went so far as to sue Ryan Sit, creator of Favtape, for simply using the Seeqpod API. This case has huge implications for mashups across the web because it then potentially holds all mashup developers liable for potential infringement due to their use of third party APIs.

Many saw this case as particularly nasty of EMI because they sued Ryan Sit, the developer of Favtape, himself. Since Ryan Sit was named directly as a defendant, all his personal assets are now potentially at stake in this law suit. It's an unfortunate turn of events given that Ryan built Favtape simply as a side project outside of his day job. But what isn't evident from the press is whether Ryan ever incorporated or formed an LLC around Favtape in the first place. The site nor the parent company Freestyle Labs show any indication on their sites of being incorporated, which could suggest the company may likely be simply a Sole Proprietorship.

Regardless of what the specifics of the Favtape case end up being, the point is that it is important for every entrepreneur to protect himself by forming either a corporation or an LLC, which affords limited liability protection to the entrepreneur. While sole proprietorships and partnerships are the easiest corporate structures to form, I would highly discourage them since they provide no such protections.

Even among the corporate structures that do provide limited liability protection, one must still decide between forming an LLC, C-Corp, or S-Corp. Out of the three, an LLC is the easiest and cheapest to form. It simply requires an $80 filing fee, a few corporate documents, and paying the annual $800 minimum tax. You can even use services like MyCorporation to come up with and file the documents for you for a nominal fee. So I would suggest that at the very least, you do this.

However, there are many cases where a C-Corp or S-Corp is a much better fit. If you intend to take any outside money at all, you will very likely be required to form a corporation and thus its better to directly incorporate (though you can convert an LLC to a corporation when required). Incorporation is more involved, requires more organizational maintenance, and is more expensive. It can run anywhere from hundreds to upwards of ten thousand dollars to setup everything correctly and maintain it. It is best to work with an experienced corporate lawyer to ensure it is set up appropriately.

I am by no means a lawyer and am not qualified to recommend a specific corporate structure. All I am suggesting though is that you please protect yourself and make sure to incorporate or form an LLC prior to launching your product to ensure that you are not one of the victims in this highly litigious country of ours!

Lessons Learned from imeem

Sachin Rekhi — Fri, 06 Mar 2009 17:02:09 +0000

Before moving on to a new phase in my career, I always like to reflect on the previous experience and put together key takeaways that I can leverage in the next opportunity.

It's that time again as this past Wednesday was my last day at imeem. As some of you know, imeem acquired Anywhere.FM at the end of 2007. Since then I've helped to migrate Anywhere.FM, develop the imeem Media Platform, and contribute to a variety of monetization projects. But now I'm eager to move on to the next adventure :)

Since I have a blog this time around, I thought I would share my lessons learned from imeem with all of you.

Content Matters. Having interesting or exclusive content is a great source of traffic. imeem's decision to allow user uploaded content has definitely helped it obtain valuable SEO for hard to find tracks. People who really want access to a specific song will go wherever they need to in order to find it. imeem's ability to get exclusive content like the Britney Spears' Circus pre-release album definitely resulted in a nice bump in traffic as well.

Creating a Community. While many people would say developing a web 2.0 UGC site is easy because users contribute significant content, it's actually a lot of work to harvest the desired community. It requires staff to police user-contributed content, answer questions\moderate forums, contribute\manage editorial content, encourage appropriate behavior, and so on. In order to scale, it's important to automate as much as possible and allow users to help manage the community themselves.

Conversion Funnel Optimization. Don't underestimate drop-off rates resulting from adding an extra click in a flow. There are lots of flows that can be optimized simply by removing extraneous pages or reducing non-essential exit paths. It's worth re-looking at all your important conversion funnels to see if you can further optimize (sign up, contributing UGC, sharing, purchase events, premium account sign up, etc).

A/B Testing. Everyone knows that A/B testing is a good idea and they should do it. Yet still so few people do. And why is that? It's because A/B testing is hard and the tools often used to perform it are limited. However, if you take the time to either build or use an existing great A/B testing framework, the cost of A/B testing goes down significantly and becomes easier to do on a regular basis. Investing in A/B testing tools is hugely useful, especially for optimizing monetization for sites where small gains have huge effects due to volume. (There's likely a startup opportunity to provide better A/B testing tools that let you look at the full effect of variations over time).

Widgets. While allowing widgets to be embedded on third party sites significantly extends your reach and can be a huge opportunity for building your brand, the amount of traffic that converts to destination site users is often minimal and the ability to monetize widget traffic is still dismal. When developing widgets, one needs to think very carefully about the actual benefits for the site and exactly how much functionality to expose in the widgets versus reserving for only the destination site.

Using an API Internally. Building an external API is a great discipline for even improving the quality of internal API methods. It forces you to think through good design, re-usability, and creating common usage metaphors. All things you should be thinking about for internal APIs. Speed to market for imeem's own apps has significantly increased with the creation of our external APIs. The audio and video flash players, the imeem Uploader, the VIP player, MySpace\Hi5 apps, and the mobile app were all built on top of these APIs.

Evangelizing a Platform. While getting large well-known companies to use your developer platform provides great case studies that will help you convince other developers to jump on board, you have to trade this off with the fact that large companies take a long time to decide whether to engage as well as a long time to build. For each large integration, you could probably get 3-5 small integrations up and running.

Focus on Monetization. Very few music startups have focused on monetization. There is still a lot of novel business models that should be tried in the music space. Instead startups have focused on building compelling products without much care to the business model. There is room for innovation in the music space if people are willing to tinker with music business models as well.

Competing with Free. It's very difficult to compete with free. Users have come to expect free music streaming from the days of Napster and BitTorrent. And now there are plenty of free music streaming services (either illegal or legally ad-supported) and continue to propel user's expectation to pay nothing for music consumption. It's very difficult to aggressively advertise or charge users without fear of user's flocking to the competition, which just gives it all away. The one nice thing about the recession is that it has forced imeem's competitors to more aggressively monetize in order to stay afloat\get funding and therefore allows imeem to follow suit without fear of losing traffic.

Media and Entertainment Monetization. It's tough to monetize media and entertainment properties through advertising, lead gen, or affiliate revenue. This is because users are there for socializing and consuming content and have very little purchase intent. While they do share a vertical interest in music, associated music commerce opportunities are limited either because of the small margin the publisher gets working with partners for digital downloads and ringtones or because the providers for concert tickets and merchandise are still not aggregated well or lack established affiliate programs. (There's likely another startup opportunity in an aggregated music merchandise storefront and affiliate program).

Direct Sales. Having a direct sales team is expensive. Not only do you need sales reps, but you need sales planning support, post sales production support, and trafficking support. It may make more sense to outsource direct sales to rep firms in the early days of a startup's life.

International Monetization. It's very difficult to monetize traffic outside of the US and a few key markets (UK, Canada, etc). Ad spend in most other countries is still very low since their online ad markets are still nascent. Oftentimes it is probably a better use of time trying to improve US monetization or trying to attract additional US traffic as opposed to trying to optimize international monetization. (I smell a startup opportunity for anyone who can crack international monetization).

Users Willingness to Pay. I was surprised that users are actually willing to pay for online services. Obviously conversion rates are very very low. But it was surprising to me to learn that people were willing to pay at all - anywhere from $3/mo - $100/year for imeem's VIP subscription service. The features were really around convenience. Not even access to content. People will pay for quality products.

Online Audio Ads. Online audio ads are a promising area for innovation and monetization. There is still $21B being spent on offline audio ads and there is clearly an opportunity to move some of those dollars online. No one is aggressively innovating with the right ad unit. Most are simply re-purposing offline audio ads online.

Incentivized CPA Offers. Incentivized CPA offers can be used to monetize a variety of digital goods even outside of the social gaming space. However, the highest eCPMs seen thus far are still in their use in social gaming.

Music Licensing. It's very difficult to get on-demand streaming deals done with all four major labels. And this doesn't even include indie content. Even if you get the deals, you are looking at a large upfront payment to each label, giving up equity, plus a rev share or per-stream fee. The labels have not been looking to give the deals to everyone either, instead focusing on making some large bets.

Value of Data. Every site of any interesting size has a wealth of data. It's important to know exactly what data is tracked and available and to mine it wherever appropriate. All too often this valuable data goes unleveraged. On the other extreme, many believe that the ultimate business model lies in selling data. For those who believe this though, I think it's tougher than one thinks to monetize data itself. But there are many valuable insights that can be gained by mining it for product improvements and getting a better insight into your audience.

Don't Believe Everything You Read. It's interesting being on the inside of a large web property with many eyes watching it. Since I had the inside scoop, I knew that many times imeem was written about, the reporters simply got it wrong. Either because of misinformation, not really understanding the service, or some rumor that someone else started. Because of this, I've become much more critical of what I read online in the tech press and look much more closely at their stated sources of information.

The Value of the Y Combinator Experience

Sachin Rekhi — Thu, 26 Feb 2009 21:57:43 +0000

I'm often asked about my Y Combinator experience so I thought I would take the time to blog about it. I did Y Combinator the Summer of 2007 in Boston with two awesome co-founders. We built Anywhere.FM, a web music player that brought an iTunes-like experience to the web, and eventually sold it to imeem.

So what is Y Combinator? Y Combinator is a new kind of seed stage venture firm. While they provide financing and advice like all venture investors, their model for doing so is very different. They give small amounts of cash (<$20,000), take small amounts of equity (<10%), and fund startups in batches twice a year. These summer and winter batches bring all the entrepreneurs in a given batch together in Silicon Valley for 3 months to have each startup build a demo-able product to show off to investors at the culminating Investor Day in hopes or raising a follow-on angel or VC round.

I would break down the value of the Y Combinator experience into four main benefits: jump starting the startup process, access to a fraternity of entrepreneurs, investor day, and funding.

Jump Start the Startup Process
Starting a company is a lot of work and a lot of very important decisions need to be made early on. Who do you pick as co-founders? How should you set up vesting schedules? What corporate structure is best? How do you assign board seats? How much money should you raise? What do you focus on early on?

Y Combinator not only provides a solid battle-tested structure for setting up your startup, but also gives tons of advice on how to succeed at your startup. The advice comes from a variety of speakers that Y Combinator bring ins. This included people like Langley Steinart, founder of TripAdvisor, who shared incredible advice on getting deals with partners and having a strong hand in investor meetings. Greg McAdoo, venture capitalist at Sequoia, also came in to provide us a complete inside look into the VC funding process. And Paul Buchheit, creator of Gmail, provided a ton of valuable lessons learned from the early days at Google and how to build a product to tens of millions of users. It's these kinds of high quality speakers that YC is able to attract that would be difficult to have access to on your own.

Fraternity of Entrepreneurs
One of the most remarkable phenomenons is the bond that is created between fellow entrepreneurs who have gone through a shared experience of trying to make something people want. What results is a fraternity of startup founders. This fraternity extends beyond your own class of YC startups to the entire alumni network of YC startup graduates.

This strong network is accessible each and every day for founders to get advice ranging from who is the best hosting provider, to evaluating deal terms, to making intros, and more. I've never met a group of people more willing to help you out. For those new to the startup community, there is no easier way to get assimilated.

Investor Day
The culminating Investor Day is by far one of the most valuable aspects of the YC program. In one or two days, you get access to an unbelievable number of top notch investors that are eager to hear your pitch. Due to early YC success stories, including Reddit, Loopt, Xobni, and Zenter, YC has been able to attract an incredible line-up of VCs who come out every batch in search of the next big thing. These investors span the entire range of angels to VCs looking to make small to large investments in a variety of tech startups.

Funding
While the funding is an important benefit, I believe it is actually the least important of the above mentioned benefits. YC only invests a very small amount of money, which is really just enough to help you cover basic expenses during the three month project. It's not difficult to scrap this kind of money together on your own. So if you evaluate Y Combinator purely as a funding source, there are many other options out there. But the real value of YC comes from the funding combined with all the other more important benefits.

My overall assessment is that Y Combinator is an amazing program for a young first time entrepreneur who is serious about jump starting the entrepreneurial experience with the advice and guidance of a strong network of fellow entrepreneurs and successful investors.

A whole class of Y Combinator style seed stage venture firms have cropped up due to Y Combinator's success. They typically offer a similar program in a different city. These range anywhere from TechStars in Boulder\Boston, LaunchBox Digital in Washington DC, DreamIt Ventures in Philadelphia, Shotput Ventures in Atlanta, or Seedcamp in Europe. My advice though is if you are serious about starting a company, I would highly encourage you to get to Silicon Valley, as its a night and day difference between starting your company in the heart of technology innovation as compared to the outskirts.

So what are you waiting for? Go ahead and apply. The deadline is March 18 :)

Monetize Online Music with Audio Ads

Sachin Rekhi — Fri, 20 Feb 2009 00:01:16 +0000

Despite the over 200 music startups that launched in 2008, I am disappointed with the lack of startup innovation in the space. While many of them definitely nailed building something people want, most failed to make something people will buy. When developing a startup, figuring out a viable business model is as important as producing a compelling product. And unfortunately there was little in the way of innovation in business models in the music space.

This is an even more pronounced issue in music, where content licenses are owned by an oligopoly of four highly litigious record labels. The labels have been eager to shut down or sue a variety of music startups, including Muxtape, Songbeat, Seeqpod, Project Playlist, and others. Yet what the labels are struggling most with is recouping lost revenue from the 45% drop in CD sales since their peak in 2000. What they are most desperate for is not a large settlement, but a new sustainable revenue stream. What they need are startup partners seeking to help them generate new commercial value with their content and through relationships between artists and fans.

There are a few notable exceptions who did focus on business model innovation. LaLa, for one, introduced a new a la carte on-demand streaming pricing model, charging users 10 cents per track for unlimited on-demand streaming. Sure one could argue that there are already too many free alternatives for on-demand streaming and thus there is a significant uphill battle for LaLa to gain traction. But LaLa is focusing on trying to build a compelling enough user experience with a clean playback experience that would be worth paying for. And at the same time, it's still to be seen whether the existing fully ad-supported music streaming services are going to be profitable businesses. To be sure, LaLa is far from a success, but at least it went out on a limb and tried something new.

However, the most interesting potential bright spot in online music monetization is monetizing time spent listening through online audio ads. While the online audio ad market is still nascent, the offline radio advertising market is still a $21B industry. The time is now for these dollars to move online in a meaningful away.

There have been various attempts at developing the online audio advertising space to date. Unfortunately early attempts were limited in success. Some have tried to bring the same offline radio ad spots from broadcast radio to online music streaming services. These are your typical 30-60 second national brand ads, like Geico car insurance ads. The attractiveness of this format is that it is much easier to convince ad agencies to take their existing audio creative and do a test buy online. For broadcast agencies, it is exciting to finally have access to actual impression data. To them it is a novel concept to actually know exactly how many people viewed their ad spots, since in the offline world much of the metrics are based on sampling. Yet this traditional radio ad format is completely inappropriate to the web world, where oftentimes users choose to move to online music services to get away from these long obnoxious ads.

What is needed in the online music space is an entirely new audio ad format that is appropriate for the medium. Just as how the online video space has developed its own unique format - the video ticker overlay ad unit - a similar new ad format needs to be developed to adjust to user behavior and expectations regarding online music.

And that's exactly what we are finally starting to see happen. imeem, for one, is serving a unique 5-8 second audio ad spot at a low frequency with a companion medium rectangle (300x250) banner that takes over the player during ad playback. Pandora just introduced 15 second audio ad spots with an initial frequency of one spot every 2 hours. TrueAnthem has developed an 8 second audio spot with a companion banner played at a frequency of once per session. Jingle Networks, which provides a free 411 voice service, has been serving 10-12 second audio spots from both national advertisers and geo-targeted local advertisers. They have had no trouble filling their inventory and have even launched a complete voice ad network to allow the insertion of in-call audio ads for other services.

There are several existing online audio ad networks already out there, including Ronning Lipset Radio and TargetSpot, which recently merged to create the largest online audio network. Unfortunately a lot of the units they carry tend to be the longer more traditional 30 second ads. This month Google announced that they would be shutting down their broadcast radio advertising business that Google has pursued since its acquisition of dMarc Broadcasting in 2006. While many picked up on that news, I think the most interesting piece is that Google also said that they are going to now actively pursue online streaming audio ads. I'm excited to see Google enter this space and I hope they bring with them advertisers willing to try shorter and more innovative formats, even if it means Google or someone else taking on the challenge of producing this new custom audio creative. This is exactly what VideoEgg had to do in the early days to get advertisers to embrace the video ticker overlay.

Early data suggests that in-stream audio ads have higher CTRs and greater brand recall than traditional online banner ads and can be minimally detrimental to site usage when done tastefully with shorter audio spots at low frequencies. Instead of seeing yet another slick AJAX interface for creating and sharing music playlists, we need more entrepreneurs to optimize the audio ad format to figure out the right set of parameters to make this work. Whoever does that may be able save the music industry and line their pockets all at once.

9 Startups to Help You Build Your Virtual Economy

Sachin Rekhi — Sun, 15 Feb 2009 17:49:59 +0000

So are you thinking about adding a virtual economy to your app or website? The great news is there are a variety of startups out there waiting to help you!

Incentivized CPA Offers and Direct Payments
Over the last two years more than a half dozen partners have emerged to supply publishers with offers and direct payment options to fuel cash flows into their virtual economy. Most partners in the space offer both a set of direct payment options as well as a large selection of offers.

When deciding on partners to integrate with, there are a variety of considerations to think about. Each partner offers a different revenue share, so its important to negotiate to ensure you are getting the lion share of the revenue. I have seen the rev split be anywhere from 50% - 90% net to the publisher. Keep in mind that it may make sense to separate the rev split for offers vs. direct payments, as its often very easy for a publisher to sign up direct payment providers like PayPal, Amazon, Zong, etc themselves and as such, the offer providers aren't justified in taking much of an additional cut on the transaction.

Another very important consideration is the breadth of offers that are available from the provider. Providers have a mix of both hard offers (requiring a credit card or purchase) and soft offers. You want to make sure that your partners have a diverse mix both in terms of types of offers but especially in terms of availability of offers per country. You'll find that some partners are stronger in certain international countries than others. In general though you should expect international monetization to be significantly lower than the US, as availability of offers are limited outside of the large ad markets (US, UK, Canada, etc).

When it comes to direct payments, the direct payment options and country coverage are both very important. Most partners integrate with PayPal or other credit card options as well as mobile payment options. The mobile payments space is still nascent and no one yet has coverage of mobile payments in all countries. So you'll want to ensure the partner you go with has mobile payment coverage in the countries that are most important to you.

Let's take a look at the players in this space:

Offerpal Media, founded in June 2007, is the most well known managed offer provider, supporting 2.5 million completed offers per month across hundreds of apps and websites. Offerpal is a full service provider, offering a variety of direct payment options (Paypal, Amazon, Mobillcash, and Ultimate Game Card) as well as a diverse selection of offers.

Offerpal makes it easy for any publisher to integrate, offering a complete self service platform as well as a dedicated business development and technical staff ready to assist you in your implementation. Offerpal has also sought to establish thought leadership in the space, putting together great articles for publishers as well as events to help publishers learn from each other's success.

Super Rewards is definitely Offerpal's fiercest competitor, launching their product publicly in December 2007. They are aggressive about winning your business and have obtained a lot of high profile Facebook and MySpace applications on their platform. They are definitely worth a look.

Gambit is a fairly new entrant in the space. Built by a group of developers who originally used the technology to monetize their own Facebook sports related applications, they have recently made the same technology available to any third party application.

Being small gives them the agility to innovate on a daily basis. I've made suggestions to them to improve their offering and have been amazed to see them implemented immediately and carefully tuned to optimize conversions. I've already seen their eCPMs start to surpass that of the more established players.

Most offer providers obtain their offers from large CPA ad networks. Several of these CPA ad networks realized that they themselves could get into the business of providing a managed offer product. PayBuyPartner is one such product brought to you by CPA Storm.

The attractiveness of working with them is that it cuts out a middleman which should theoretically result in higher returns. However, they are focused on both offer acquisition as well as offer optimization, which may spread their efforts thin.

SocialCash is another offer provider built by Gratis Internet, one of the largest CPA ad networks. Gratis Internet was even the CPA ad network that brought us the most notorious of incentivized cpa offers: free ipods!

However, they have recently launched a new ad product called Headliners, which has resulted in them de-emphasizing their virtual currency PointCash product. While SocialCash may make sense if you are interested in offers in international countries where they are strong, I would suggest avoiding them as a general solution as they will likely no longer be innovating on this product.

TrialPay, founded in 2006, has a long history of providing qualified offers from prominent merchants. With the growth of virtual currencies, TrialPay has adapted their system to work in the virtual currency environment. There solution isn't quite as plug-and-play as some of the others, but definitely worth looking at for their offer selection.

Peanut Labs originally built Xuqa, a social network that has taken off in Turkey. When ads weren't working, Xuqa introduced a successful virtual currency called peanuts. Peanut Labs is now offering this same monetization platform to third party developers.

Peanut Labs specializes in quality market research surveys, which are often the best type of soft offer. Peanut Labs also has built an impressive selection of direct payment options that differentiates them from other players in the space.

Offer Provider Optimization
Given that there are already so many different offer providers and more coming, a solution is needed to easily A/B test them to select which work best for your site in each country. This should be done for both offers and direct payment options. Since every offer provider will tell you that they are confident they are better than their competition, the only way to know for sure is to test and find out!

Sometrics is the first to build a virtual currency manager product that makes it easy to test multiple offer providers and specify rotation ratios for each provider. They have done a good job encouraging developers to get the most out of offer providers. This product is still in its infancy and not without its limitations, but its a great start at a product specifically designed to address this very real need.

I hope to see others enter this emerging space.

Virtual Economy Optimization
In addition to optimizing the offers and direct payment options, its important for publishers to carefully monitor and tune their virtual economy. For example, a publisher needs to look at available sources (ways of acquiring currency) and sinks (ways of spending currency) to ensure they are appropriately balanced. By graphing currency acquired and currency spent over time, a publisher can ensure that the graphs are aligned. If, for example, the currency spent dips compared to currency acquired, a publisher needs to introduce new valuable virtual items available for purchase or generally increase the price of their goods to keep their users hungry for currency.

SocialGold by Jambool is a great virtual currency analytics product to help you easily monitor your virtual economy, study price elasticity, and generally tune your virtual currency. Through simple instrumentation of your application, you get access to easily understandable graphs and tables overviewing the performance of your economy. In addition, SocialGold provides a wide selection of direct payment options across both credit card and mobile alternatives. Definitely worth checking out as well.

Additional Opportunities
Beyond these existing startups, there are still additional opportunities for startups to build tools to help support the virtual currency ecosystem. With the continued growth in both usage and revenue from social games and virtual economies it's a valuable startup opportunity.

One area ripe for innovation is merchandising of virtual items in social games. Every social game has some form of store where users can spend their virtual goods. This has effectively made every social game an etailer. However, social game developers are not well versed in the best ways to merchandise items in their storefront, on deciding how to feature the most valuable items, on suggesting related items, managing shopping costs, order histories, etc. And it's not necessarily something developers want to focus on when there is enough to deal with in developing the core game mechanics of their experience. It provides an opportunity for a partner to provide expertise in retail to the social games arena.

I expect in 2009 we will see continued growth throughout the virtual currency ecosystem. Got a new product in this space? Let me know!

Incorporating Virtual Currencies in Non-Gaming Sites

Sachin Rekhi — Thu, 05 Feb 2009 04:25:33 +0000

2008 saw an explosion in virtual currencies and their associated virtual goods all across the internet. With Facebook social game Mob Wars suspected of making a million dollars a month and Zynga rumored to have made $50 million in revenue in 2008 off of its social and iPhone games, companies are seeing real revenue from this burgeoning monetization model. Even other indie social game developers like Mobsters and Lil Green Patch who also monetize virtual currencies have seen huge success with 13M users on MySpace and 5.9M monthly active users on Facebook, respectively. To top it off, over $580 million was invested in virtual goods related startups in 2008.

While social games is the newest category of apps monetizing virtual currencies, MMOs like World of Warcraft and Second Life have had vibrant virtual economics for years. In addition, casual MMos like Gaia Online, Habbo, and IMVU have also built thriving currencies.

Yet the most recent trend that is fascinating to me is the incorporation of virtual currencies in non-gaming sites. As growth in the online advertising market continues to deteriorate, sites are seeking alternate forms of monetization to wane their dependence off ads. Many eye the robust gaming virtual currency revenue stream and are eager to expand its success into non-gaming environments. With Facebook itself estimated to have made $30-$40 million from its virtual gifts, many others hope to imitate and expand upon such early success.

Given this, I decided to put together a quick case study of five of the most interesting virtual currencies in non-gaming sites today.

myYearbook, a teen oriented social network founded by two high school students in 2005 with now over 4.4M US uniques, is a popular destination for socializing, meeting new people, and entertaining yourself.

myYearbook has incorporated their "lunch money" virtual currency into every aspect of their experience. Even when a new user first creates an account, they are awarded lunch money for simply uploading a profile picture and inviting their friends to join. Wherever possible, myYearbook attempts to offer you additional lunch money for additional activity across the site. By getting high scores in flash games on the site, you are awarded lunch money. For winning popularity battles, you are granted additional lunch money. myYearbook has even built their own rendition of Friends for Sale called Owned. Users acquire points through the game but also spend points to purchase their friends. You can also acquire points through various offers as well as direct payment options ranging from credit card, to mobile payments, to PayPal.

Once you have acquired lunch money, you have a variety of ways to spend your currency. In addition to buying your friends in the Owned game, you can purchase additional top friends slots to showcase your friends on your profile. You can also purchase a pimped out mp3 player and more songs for your playlist. Or battle super votes to ensure you win battles or support your friends. They have even built a Causes application, allowing you to contribute lunch money toward world causes like hunger, global warning, save the rain forest, and more. myYearbook then converts your contributed lunch money into hard cash that they contribute to the appropriate charities.

Heysan is a mobile web-based IM aggregator allowing users to easily connect to MSN, AIM, ICQ, Yahoo, and Gtalk on the go from their phone.

Similar to myYearbook, Heysan initially offers up "coins" for simply joining the service and inviting your friends. Heysan also provides additional coins for performing actions on the site that they want to encourage, including daily sign-in, validating your phone number for SMS notifications, or filling out your profile. You can also get additional coins through various offers and direct payments.

What's cool about Heysan is how you can spend your coins. They can be used to get exclusive emoticons, new buddy icons, wallpapers, photo backgrounds, background colors, profile font colors, friend gifts, feature profiles, and more.

Scrapblog is the easiest way to create stunning multimedia online and print scrap books. With over 2 million registered users, media partnerships with Disney, Discovery, and Photobucket, and a newly raised round of $4 million, Scrapblog is poised for success.

Scrapblog has traditionally monetized through printing and sponsored promotions. However, they have recently introduced the Scrapblog Marketplace, which allows you to purchase premium content for your scrap book with Scrapbook "credits" that are purchased with a credit card.

These credits are used to purchase anything from new backgrounds to specially made stickers for your scrapblog. A variety of premium designers have put together an expansive selection of exclusive content for your scrap booking needs.

Dogster is the premier social network for a man's best friend. Showcase your own animal, browse your friends dogs, and check out the half a million dogs already captured on the site.

Dogster has monetized through advertising sold through their own direct sales team targeting the $42B pet industry as well as a premium subscription service for $20/year that provides benefits including the ability to upload more pictures, photo captions, ad-free browsing, and more.

In addition, Dogster has monetized through its own virtual currency, known as zealies. Dogster provides a variety of free ways to acquire zealies. You get some when you join. If you give them more info about yourself, they'll award you more. Sometimes you get them from games and other times they give them away for holidays. They try to rotate a variety of free ways of acquiring zealies. You can also acquire zealies by purchasing them at a rate of 20 zealies for $5 through Paypal. Users can additionally get monthly zealies through the paid subscription service.

Zealies are then used to purchase a variety of gifts for dogs across the site. Gifts include rosettes, which are one-month lasting ribbons you can give to your favorite dogs to recognize them. Then there are a variety of holiday related gifts. They even have sponsored gifts, including a recent Febreeze branded collar that was given away for free and paid for by the advertiser.

Mahalo recently launched Mahalo Answers, a Q&A service similar to Yahoo and Google Answers.

The interesting twist for this service is that besides users answering questions for free, the question asker can also offers Mahalo Dollars to the best answerer. This Mahalo Dollars virtual currency can be funded through PayPal. The question asker then awards the tip to the best answer. The users who receive Mahalo Dollars can convert it back to real cash after they reach a minimum amount of tips and after Mahalo takes it 25% cut.

Mahalo has also created a points system, allowing people to acquire points for answering questions well and adding friends. As a user acquires points, they progress through multiple belt levels, with higher belts designating that you are more of an expert in the community.

Resources

Monetize the Twitter API

Sachin Rekhi — Sat, 31 Jan 2009 05:05:02 +0000

Silicon Alley Insider recently announced The Create a Twitter Revenue Model contest. I decided to throw together an entry and today SAI has picked it as one of the finalists. So I thought I would write a blog post summarizing my submission.

Thinking about business models for Twitter is always a fun exercise. I see Twitter as the epitome of web 2.0 - a sticky communication app with few differentiated user segments, minimal expressed interest, and no purchase intent. I always joke that if you can successfully monetize Twitter, then you can monetize anything :)

There are many different ways to think about monetizing Twitter. The ones that have been frequently discussed include a freemium subscription plan for power users or commercial users, inserting sponsored ads as tweets, or other forms of advertising. While these may eventually be successful strategies for Twitter, I decided to take a different approach.

Obviously today Twitter is not monetizing and therefore has no revenue. Yet the interesting thing is the Twitter ecosystem is definitely monetizing. We have desktop clients like Twitterriffic ($14.95) and Blogo ($25). We have a variety of iPhone clients available, including Tweetie ($2.99), Summizer ($2.99), Twittelator Pro ($4.99), and Tweetsville ($3.99). In addition, there are several companies already inserting ads as Tweets, including Twittad and Be-a-Magpie. The ecosystem has already seen StockTwits and TweetDeck raise funding and two acquisitions with Summize and Twhirl.

My approach to revenue then is for Twitter to monetize the channel that is already generating revenue for the ecosystem by monetizing the Twitter API that all these applications are built on top of. By introducing a premium Twitter Developer Program, Twitter can start to share in and encourage the profits that third party developers are making on top of the Twitter platform.

There have already been a variety of successfully monetized APIs, including:

Microsoft Parter Program & Certified for Windows
Microsoft has always had a long tradition of providing a partner program offering to developers who build on their various platforms. These programs provide developer tools, marketing support, and access to premium support. In addition, the Certified for Windows program charges every app for this badge that ensures customer trust.

PayPal Developer Central
PayPal's API for third party publishers to accept transactons and payments monetizes every transaction by taking a variable percentage.

iPhone Developer Program
Apple requires all developers that want to publish to the App Store to join the iPhone Developer Program. At $99-$299 per developer with over 15K applications, that's a several million dollar opportunity.

Facebook Platform's Application Verification Program
Facebook Platform’s Application Verification Program provides developers with a badge to encourage user trust of the app, increased distribution allocations for all viral channels, and an advertising credit. At $375/app/yr and over 48K apps, that’s tens of millions in annual revenue.

What would a Twitter Developer Program look like? Here are the suggested program benefits:

Benefit	Details
Commercial Use	Only applications that have registered for the developer program will be allowed to use the Twitter API for commercial use (ex: charging for your app, advertising in your app). Non-commercial use will always be free.
Twitter App Directory	As part of launching this program, Twitter will launch a full app directory that allows any app developer to add their Twitter mashup for easy app discovery by twitter users. Developer’s who are part of the developer program will be eligible to apply for the certified app program to get a badge to increase trust with users and will also receive premium placement in the app directory.
Increased Rate Limits	Developer program participants will have increased client rate limits (currently 100 requests/hr) and developer rate limits (currently 20,000 requests/hr) to improve user and developer experience.
Technical Support	Program participants will have priority access to developer technical support to support their application development.
Access to Firehose API	A premium developer program will be offered for those key partners that need access to the proverbial “firehose” that gives them access to all non-protected tweets. (This would only be available as part of a higher priced Twitter Developer Program, not in the standard plan).

One thing to be careful of in charging for access to a platform is to ensure that doing so does not stem innovation on the platform. That's why access to the Twitter API should always be available for free for anyone to tinker. This program simply provides value add for those building commercial businesses on top of the platform.

For more details on proposed pricing, size of opportunity, and FAQ, check out the full presentation embedded below:

Monetize The Twitter API

View more presentations from Sachin Rekhi. (tags: monetize twitter)

Make Something People Will Buy

Sachin Rekhi — Sun, 25 Jan 2009 08:59:49 +0000

Y Combinator is famous for its well known motto "Make Something People Want." This very simple statement serves as a guiding principle for a startup, helping to focus it on this core goal early on: to ensure that its product provides compelling value to its target audience. It also de-emphasizes many secondary issues that, while important, are not likely to be an early cause of failure for a startup. I'm a big fan of the motto and use it myself as a quick filter for all the startup ideas I evaluate.

Yet the economic recession that we are in the middle of poses new challenges. While it hit in the financial and real estate sectors first, it is undeniable that it will have a lasting effect on all sectors of our global economy.

For the startup community, the most immediate ramifications have been difficulty in raising funding. Many angel investors have significantly cut back their investing. VC dollars have dropped 33% in Q4 08 alone. And for those who can obtain funding, lower valuations, down rounds, and 3X liquidation preferences are back.

At the same time, consumer confidence is down. Unemployment is at all time highs and only increasing. This has resulted in consumer spending being down across the board. With the drop in spending comes the slashing of advertising budgets.

The days of eyeball companies are also gone. No longer can you hope to build a service, drive significant traffic, and be acquired by one of the tech giants. Google, Microsoft, and Yahoo have all announced layoffs and project closures. Everyone knows that acquisitions in 2009 with be far fewer than those in the previous years.

Despite painting this landscape of doom and gloom, I still believe now is a great time to start a company. It simply requires a tighter lens through which to evaluate startup opportunities. A lens that takes into account the challenges of raising funding and early acquisitions and instead focuses on achieving cash-flow positive, ramen profitability, and self sustainability.

And hence I come to my own corollary to the YC motto: "Make Something People Will Buy".

Let me first address how this relates to the original motto. If you think about what this means, it fully encompasses the original statement. In order to make something people will buy, you first have to make something people want. But now making something people want is a necessary, but insufficient condition. You have to take it to the next level and make something people find compelling enough to buy.

This most definitely makes the issue of business model and monetization a first order problem. Instead of taking the tack that this can be addressed later, it forces you to consider this with the initial problem definition. With this paradigm you'll find yourself, as I do, thinking as much about ARPU and TAM as UX and traction.

Many who first come across this will assume it implies freemium, e-commerce, and subscription business models. And it definitely emphasizes these direct monetization opportunities as well as the now growing virtual currency and virtual goods space. As we won't see much growth in online advertising in 2009, charging your users directly for your service will become a much more popular model. We have already seen Sprout, Jott, and others abandoning their free applications in favor of simply providing their paid products.

Yet I by no means intend for this mantra to rule out indirect methods of monetization. It simply requires that if you do plan on leveraging indirect monetization models, you think through the entire conversion funnel of how your app will eventually result in a purchase event. So instead of simply planning on slapping on text and banner ads on your site from your favorite ad network and monetizing page views, you have to think through exactly how your site aggregates a qualified audience, purchase intent, or vertical interest that can be leveraged for commercial value.

We've seen a lot of innovation in the last 5 years on product, but limited innovation on the monetization front. I hope with the current flight to revenue, quality, and sustainable businesses, we start to see real creativity in the business models and monetization methods that drive web businesses.

I find myself going through the mental exercise to challenge myself to think through how Facebook and Twitter, the stars of 2008, could develop a revolutionary monetization strategy as Google once did with its AdWords and AdSense products. And I'm sure you'll see me writing more on my blog about my thoughts on monetization.

So I challenge you to go forth, try this new lens, and make something people will buy.

Does Facebook Connect Deliver on its Promise?

Sachin Rekhi — Mon, 19 Jan 2009 15:12:01 +0000

The announcement of Facebook Connect in May 2008 brought the next major chapter in the Facebook platform story. After building the first and most successful social networking platform, Facebook decided to expand beyond its own destination to bring the power of the social graph to any third party site.

Facebook Connect promised to deliver on the five following tenants: trusted authentication, real identity, friend linking, dynamic privacy, and social distribution. In the half a year since the announcement, how has Facebook Connect done?

Let's take a closer look at how Facebook Connect has fared on each of these tenants.

Trusted Authentication
Facebook Connect has by far the best authentication and single sign-on solution to date. It wins due to its simple and clear user experience. While Windows Live ID, OpenID, Google Friend Connect, and others have in the past provided single sign-on solutions, none saw significant traction. The most important innovation this time around is a rather simple one: Facebook Connect provides a javascript-based light box sign-in screen on the same page without redirecting the user to a third party site for authentication. In addition, the user simply logs into Facebook from the light box if they aren't already (but who isn't always logged into Facebook these days), and then simply authorizes the app with one click.

It's very satisfying to go to a Connect-enabled site, hit the Connect button, select authorize, and immediately have a presence on a site. No longer is there the friction of deciding whether to go through the hassle of creating an account, setting up a password, and giving away additional personal info just to begin to experience a site's benefits.

Facebook has stated that many sites are already seeing significant success with trusted login:

Some sites that have chosen to include login have already told us that they have seen a two-time or more increase in registrations and 2/3 of users creating accounts via Facebook Connect.

Real Identity
In addition to the sign-on experience, Facebook Connect provides publisher's with rich access to authentic user profile data, including profile picture, real name, birthday, location, relationship status, work history, and much more.

For many small publisher's, users will be much more willing to supply this data to Facebook and not likely to take the time to do so on your own site. Therefore this is a great opportunity for publishers to take advantage of access to this info as well as further simplify their site's registration process.

One important caveat though is you do not get access to a user's email address, which is often one of the most important profile fields during a registration process. Due to privacy and spam concerns, Facebook prevents access to this info. A publisher then has two alternatives. On one hand, a publisher can prompt for an email address outside of the Facebook Connect registration process. While this provides the greatest control, additional profile fields will reduce some of the friction-free benefits of Connect. The alternate approach is to leverage the email methods Facebook provides. For each Facebook Connect authenticated user, you are provided a proxied email address which you can use to email the user. However, this approach does have several constraints. The total number of emails you can send the user is governed by Facebook email limits, thus forcing you to adhere to Facebook messaging constraints. In addition, keep in mind you'll have to prompt the Facebook user with a Facebook permission javascript window to have the user opt-in to email notifications from your application prior to leveraging theproxied email address.

Friend Linking
One of the strongest promises of Facebook Connect is to allow a user to take their social graph with them across the web. As users come to your site and connect through Facebook Connect, they are automatically able to see which of their friends are already on the site and see their activities.

Once you have a decent community of Facebook Connect users, this works quite well. Since you have access to all of a user's friends' Facebook uid's, you can now show activity of friend's already using your service.

The problem though comes initially when you launch a Facebook Connect implementation. If you have a large existing user base that has not yet connected via Facebook Connect, then no friends will show up for a new Facebook Connect user, even if their friends exist on the site using the previous authentication mechanism. To help solve this problem, Facebook provides a publisher the ability to submit hashes of email addresses for all of their existing users to Facebook, so that the publisher can prompt a new Facebook Connect user with an invitation dialog to invite existing users of the site to connect via Facebook Connect so they can share friend connections. While this is a useful feature, it requires double opt-in from both the user and their friends. Many who receive such invites may choose to ignore them, making it difficult to jump start the social graph process with an existing user base. Facebook decided to require the double opt-in to ensure privacy for all Facebook users and avoid some of the issues of the previous Beacon product.

Dynamic Privacy
One tenant Facebook strongly advocates as a real win for Facebook Connect is dynamic privacy. This is an important pillar for Facebook given its previous blunder with Beacon. Users are now in full control, allowing them to choose whether to connect to each site with Facebook Connect, whether to connect existing publisher user accounts with Connect, and the ability to share and un-share profile information with sites and friends.

From a publisher's stand point though, this really just creates implementation limitations. Since all data from Facebook Connect is subject to the Facebook Platform TOS, which limits data caching to 24 hours, a publisher needs to adhere to this restriction and always pull data dynamically from Facebook.

Social Distribution
Facebook Connect also allows users to share their activities back to Facebook in their profile Wall and News Feed. This allows users to share their experiences with your site with their friends and hopefully drive more awareness of your site through Facebook feed channels.

Facebook makes it easy to pop-up a javascript light-box window to allow a user to approve the feed story. It even provides simple options for a full, short, or one-line story, so the user can decide how much they wish to emphasize this activity. While this is an opt-in message (again, correcting their mistakes from Beacon), a user can select to save their preference for this specific activity so future stories can be posted without further approval.

While this is definitely a valuable feature for users who are seeking to make all their activities viewable from Facebook, I wouldn't expect this to generate much traffic to your destination site. In the good days of the Facebook Platform, much of the platform growth was due to viral feed stories, notifications, and invitations. As users began to find these channels considerably spammy , Facebook significantly locked down all of them. These days you rarely see many third party app news stories in the News Feed due such limitations. I expect that right now Facebook is allowing Facebook Connect stories to crop up to encourage publishers and user's to use Facebook Connect, but I suspect over time these will also be constrained. In addition, Facebook has little desire to encourage users to leave Facebook and thus is at odds with ramping up distribution to third party sites.

Conclusion
For small publishers, Facebook Connect is a definitive win for its trusted authentication, real identity, and friend linking capabilities. It makes it easy to "socialize" a website that has classically been a straight content site or niche community by significantly reducing friction.

While Facebook Connect does provide value for large publishers as well, there are key issues that publishers must carefully think through before adopting Connect, including access to an email address, merging their existing social graph with Facebook Connect, dealing with Facebook TOS caching restrictions, and more.

To date we have seen some quick and obvious wins from Facebook Connect, including blog comments and socializing content sites. However I'm still waiting for a truly innovative Facebook Connect implementation that goes beyond basic authentication and friend linking to really take advantage of all that Facebook integration has to offer. Is your site ready to take on my challenge?

Resources
For those getting started with Facebook Connect, check out these resources:

The Rise of Media APIs

Sachin Rekhi — Fri, 16 Jan 2009 17:16:22 +0000

Last year at the SocialMedia Business School I gave a presentation on Media APIs and how they could be leveraged to enhance existing experiences or build entirely new services around freely available media content. Since I have received many follow up questions, I thought I would take the opportunity to expand on the topic.

Media APIs
2008 definitely saw a rise in media APIs across the major media content types with ProgrammableWeb cataloging 38 music, 42 video, and 37 photo APIs.

Photos
In the photo space, Flickr continued its dominance and saw strong growth in the use of its APIs, reaching a record of 704 API calls per second and ranking as ProgrammableWeb's second most popular API (after only Google Maps).

Videos
YouTube finally launched its own video APIs in a big way in March 2008, quickly becoming the default video API of choice. YouTube did a great job with their offering, providing full programmatic access to their video player as well as a chromeless player to allow third party sites to brand the experience and customize the player controls as they saw fit.

Music
2008 also saw a slew of entrants in the music API space, with imeem launching its APIs in March (disclaimer: I manage the imeem APIs), followed by Last.FM API 2.0 in June, Yahoo Music in August, and iLike and MTV in September. Each came with its own strengths, ranging from unlimited on-demand full length streaming, to limitless music-related metadata, to high quality music videos.

In all 3 categories, API providers have offered very complete solutions to recreate the entire site's user experience as well as leverage a lot of the site's underlying infrastructure. Each provide upload APIs to offload the cost of uploading, storing, and serving media content. They each also provide advanced search APIs to programmatically find exactly the content you are looking for as well as full access to associated metadata for every media item.

Media API Mashups
We are already starting to see categories of resulting mashups emerging, including startups seeking to innovate on the browsing, personalization, and social aspects of experiencing media.

Browsing Experience
We saw a variety of startups in 2008 evolving the user experience of browsing and consuming media content. For example, uvLayer created a beautiful and fluid webtop experience for experiencing YouTube and Flickr content.

Personalization Experience
Given the continued media fragmentation and move away from prime time TV towards online content spread across dozens of popular destinations, it has become much more difficult for a consumer to constantly find interesting content to consume. To fill this void, a variety of media aggregation plays have emerged to bring the best of these content sites together and provide recommendations based on your personal tastes. ffwd is one such service that has aggregated videos from many APIs and organized them in personalized channels based on your expressed tastes.

Social Experience
Others have focused exclusively on the social experience of sharing and discovering media with your friends. Slide has spent a lot of its effort on this, with the integration of premium video into FunSpace and premium music into Top Friends.

API Cost
All the major API providers have decided to provide their APIs for free to developers and hope to monetize through advertising within the media playback experience or indirectly monetize through driving traffic to their destination sites. This is good news for entrepreneurs looking to take advantage of these APIs.

However, I suspect with the flight towards revenue for many technology companies in 2009, entrepreneurs leveraging the media content should expect significant advertising in all syndicated media.

Licensing Limitations
It's not all good news though when it comes to media APIs. Some have very serious licensing limitations that can have significant implications on a startup that is built completely on top of them.

The limitations range from caps on number of API calls, to explicit language preventing building products that compete in any way with the API provider, to the ability to force you into a rev share agreement at a later date. So read the terms of use of each of the APIs carefully before embarking on an integration.

Monetization
The most serious limitations that some API providers have are those around commercial use. This limits your ability to monetize the service that leverages the third party content. Some prevent it outright whereas others have severe limitations on how and when you can monetize the content. Keep in mind as well that the content provider typically reserves the right to advertise in the media itself (instream audio or video pre-rolls\overlays typically). This also limits your own ability to command high CPMs from brand advertisers on media pages as you cannot ensure you can command 100% share of voice for the advertisers.

I hope this provides a more detailed overview for anyone thinking about leveraging media APIs in their next application. Embedded below are the slides from the original presentation.

Intro To Media APIs

View SlideShare presentation or Upload your own. (tags: media api)

Palm Gets it Right With Mojo Developer Platform

Sachin Rekhi — Mon, 12 Jan 2009 07:35:57 +0000

The most exciting news out of CES 2009 was the Palm announcement of the Palm Pre, webOS, and Palm Mojo Application Framework.

Palm to my surprise has reinvented itself and gotten back into the smartphone game with the release of the sexy Palm Pre device and new webOS operating system. Early indications suggests it should be in the same consideration set as Apple iPhone, Android G1, and Blackberry Storm. Yet the most innovative news of this announcement was the new Palm Mojo Application Framework.

Only several weeks ago I was having a conversation with one of my colleagues about the double-edged sword of open native mobile platforms. While the opening up and associated app stores have created a lot of opportunities for developers, they have also required developers to learn many disparate development platforms for each and every device. Sure Android should make it simpler to port apps across supported devices, but I suspect it will go the ways of OpenSocial in that it won't bring the promise of write once\run everywhere, but instead the philosophy of learn once and then simply minimize cost of porting to anywhere.

In the desktop world we have just gone through a revolution where we have moved many apps away from native Mac, Windows, and Linux applications to a web world where we build cross-browser and cross-OS experiences. And we are continuing to encourage the revolution with even more powerful browsers like Firefox and now Google Chrome.

I am eager to skip the pain of native proprietary platform mobile client apps and jump right to a world of mobile web browser based applications with cross-device javascript libraries to provide hooks into the native operating systems. Of course the typical criticism of browser based mobile apps today is that they can't take advantage of many of the benefits of the native device, including location based services, address book, local cache and offline data access, and native UI components and gestures. Yet these are all solvable problems by simply having each of the popular platforms exposing javascript APIs for each of these components. Joe Hewitt's early work on iUI shows just how powerful the existing iPhone Safari browser already is in allowing you to recreate full fidelity iPhone native app experiences within the browser.

Palm is taking the first step in realizing this vision and releasing the Mojo Application Framework to allow developers to build application on the new webOS using the web technologies they already know: HTML5, CSS, and Javascript. This thus allows organizations to tap into their existing web assets and already vast experience in building scalable web applications. Developers can leverage the local storage capabilities of HTML5 to have offline access to data. They also have full access to gestures, transitions, and more and access to many of Palm's native device components. It even has full support for background running applications and user notifications, a common criticism of the iPhone platform.

While its too early to tell whether the Palm Pre, webOS and Mojo will take off, it is definitely a step in the right direction for mobile developer platforms. I see way too many examples of native applications on the iPhone that could be much more cheaply developed, more stable and robust, more easily maintained, and available across many more devices by simply making enhanced mobile web apps. I hope to see iPhone and Android opening up even more capabilities in their browsers through javascript APIs and making apps developed with web technologies feel like full fidelity applications on the device, just as Palm is promising to do.

Top Underhyped Open Platforms

Sachin Rekhi — Wed, 07 Jan 2009 16:29:56 +0000

2008 was definitely a year of open platforms with the continued growth of the Facebook and OpenSocial communities, the unveiling of the iPhone and Android app stores, and the countless Twitter clients and mashups.

Yet I believe there is still considerable untapped opportunity in several promising platforms that have yet to see significant traction in terms of hype, developers, and ultimately end users.

So here is my list of the top underhyped platforms that I hope to see many entrepreneurs build on in 2009.

Webmail Platforms: Yahoo! Mail, Gmail
There are so many interesting problems to solve with email and we as entrepreneurs are finally going to be able to innovate on them with the opening up of the popular webmail platforms.

Just think of the pain that is email today:

Constant overload of email volume and very few ways to sort through the clutter
A sophisticated and natural social graph locked in email with no easy way to leverage it
Endless files shared through email that are problematic to find, store, and share

While companies like Xobni have developed very innovative solutions to these problems on predominant desktop mail clients like Outlook, we can finally bring these and new innovations to the webmail services we all now live by.

Yahoo! Mail, with over 250M users, has announced its application platform, launched a few white listed applications (Xoopit, Wordpress), and plans on opening up more broadly in 2009. Gmail, with 100M users, already has launched several first party gadgets for Gmail (Google Docs, Google Calendar) and has a developer sandbox available for any developer to place Google gadgets in their Gmail sidebar. I expect in 2009 we will see the complete opening up of these platforms and maybe even Windows Live Hotmail, with its 250M users as well.

Professional Networking Platforms: LinkedIn, Xing
I'm glad much of the craze of social networking platforms has died down as I don't think I could take many more vampire bites, food fights, or fluff friend races. These days it looks like the social networking apps that are still driving acquisition and engagement are those of social gaming and while I occasionally dabble with them, I fail to experience any lasting value from them.

However I'm hopeful on professional networking apps as I think they will likely delve deeper and look to provide more real value than their social networking counterparts.

I'm relieved the LinkedIn Platform did finally launch, but so far I've seen too little too late. I do find the Reading List and Blog Link apps useful to see what my colleagues are reading and writing, but there is so much more I hope to see. LinkedIn was smart in their early thinking of the platform in that they were looking to open up not only first degree contacts but also second and third degree to allow innovation on introductions, new contact-related applications, and more. I want to see applications actually start to leverage what is truly unique about LinkedIn. I think LinkedIn app innovation though will continue to be significantly hampered by their closed platform approach that requires an approval for even getting access to the sandbox. I hope LinkedIn decides to open up the sandbox to allow anyone to build innovative applications but then holds a tight review process to ensure it stays professional and relevant.

Xing, a popular German and European professional networking site, has announced its own OpenSocial platform for 2009, so this should provide interesting opportunities in the international space as well.

Cloud Platforms: Amazon Web Services, Google App Engine, Windows Azure
Many startups have already built consumer and enterprise apps on top of AWS and are starting to dabble with Google's and Microsoft's own cloud platforms. This has already been a disruptive shift in reducing initial CapEx for startups and helping to bring down both the operational cost and effort for basic infrastructure.

But where I think the significant opportunity is in 2009 is building infrastructure applications on top of these cloud platforms to provide higher level services to other startups looking to more easily leverage these cloud solutions. RightScale, for example, is one such infrastructure play that sits on top of AWS but makes it easy for you to manage and auto-scale your EC2 instances. Heroku is another exciting example of a very high level Ruby on Rails development platform that allows developer to simply focus on their app code and Heroku takes care of the rest in terms of spawning EC2 instances, managing load, and more.

Yet there are still lots of much needed services to be built to support cloud platforms that I expect we'll see much more of as more and more startups move to leveraging the cloud.

Got your own thoughts on an underhyped platform? Leave it in the comments!

5 Music Startups to Watch in 2009

Sachin Rekhi — Mon, 05 Jan 2009 15:57:56 +0000

It's that time of year that everyone is putting together their predictions for 2009 and so I decided to do one of my own.

Over the past year I've had the opportunity to work with a variety of music startups. Not only with the two music startups I've been directly involved with (Anywhere.FM and imeem), but also countless music startups looking to leverage the imeem Media Platform APIs I manage at imeem. Based on what I've seen, there are three music verticals that I am bullish about and 5 exciting startups that have started to pursue them in 2008 and expect to see much more from them in the new year.

Music Gaming
Music gaming has benefited from renewed traction in recent years. The current explosion can be attributed to the tremendous success of both Guitar Hero (2005) and Rock Band (2007), but we can't forget the earlier successes of Dance Dance Revolution (1998) and even further back with Parappa the Rapper (1996). What's most exciting about it this time is the size of the opportunity. The Guitar Hero franchise alone has seen over 14M units sold generating $1B in sales in North America. On top of that they have seen over 5M song purchases. And for the first time ever Beatles music is actually available digitally through Rock Band (even though the Beatles have classically refused to make their music available on iTunes, music streaming services, etc).

In 2008 we started to see what the next generation of music gaming will look like.

JamLegend

JamLegend is best described as a web based version of Guitar Hero. Not only have they done a great job of recreating the fun of Guitar Hero on the web and your computer, but they've opened up the opportunity for very interesting web-based additions. One exciting opportunity for them is continuing to build out their massively multi-player capabilities, tournaments, and leaderboards. At the same time the ability to eventually crowd-source all the tracks and build the largest library of playable content will be key. JamLegend could become the newest form of music promotion for artists and both indie and major label artists will be looking to take advantage of this latest phenomenon.

Ocarina

Ocarina is by far the most clever music game I've seen to date. Ocarina is the first true musical instrument created for the iPhone. Simply blow into the microphone, press down on the four keys, and you're playing beautifully sounding music through your Ocarina. And the social experience is great as well, since you can click on the world map and see who else is currently playing their Ocarina across the globe and tune in to enjoy their music. I'm excited to see these guys expand to actually hosting concerts, recorded videos, and even more instruments.

Rock Legends!

Rock Legends is the latest social game from Serious Business, the startup that created the Friends for Sale social game on Facebook. This time around a user starts a rock band, picks what kind of rocker they want to be, and recruits others to join their band. It's a social experience complete with playing gigs, battle of the bands, and more. I would be excited to see Rock Legends integrate JamLegend style playback into their game to tie the best of music gaming together.

Live Events
Another space in music that continues to be attractive is live events. Unlike the ongoing decline in CD sales and the slim margins of digital downloads, concert tickets sales continue to enjoy high margins and make real money for the artist. Yet there continues to be unsold seats at the majority of events. There clearly continues to be an opportunity in 2009 to more efficiently bring fans to see their favorite artists at live events.

SongKick

While there are a lot of players trying to tackle this space, I'm most excited by the work SongKick has done. In a little over a year they have built a comprehensive concert listings engine and database. It's no easy task as very few clean feeds exist out there from venues and ticket providers. Most recently they've been doing a lot to build out their website experience to make it easy for fans to find when their favorite artists are coming to their area and even recommend artists you may enjoy. With a new round of funding, it will be interesting to watch how they expand in the new year.

Dis-intermediating the Major Labels
What I am most passionate about are those out there seeking to dis-intermediate the major record labels. The major labels have been unable to come up with any significant revenue stream to offset their losses in CDs and unfortunately it hasn't been due to lack of technology innovation. They have for the most part failed to support the most creative entrepreneurs out there by making it very difficult for them to get access to legally licensed content. As many have recognized, its time to move beyond them and seek to establish relationships directly with artists and their fans and either create the next generation record labels or remove the need for labels entirely.

Topspin Media

Topspin Media is the most promising startup in this space. Topspin focuses on providing tools to artists to make it easy for them to communicate directly with their fans through a web presence, email distribution, and more. Topspin has started by smartly focusing on super-fans who are willing to devote time and money to engaging with their favorite bands through premium purchases, subscriptions, etc. I hope to see Topspin unveil additional products in the coming year and expect both its artist and fan footprint to grow substantially.

Is Google App Engine Ready for Prime Time?

Sachin Rekhi — Fri, 02 Jan 2009 05:08:54 +0000

I recently took the time to build a web application on Google App Engine and wanted to share my thoughts on the experience and the pros and cons of Google App Engine as a web development platform.

TuneChimp

The app I developed is TuneChimp, a music mashup that was recently named ProgrammableWeb's Mashup of the Day and a finalist in Mashable's Y! BOSS Challenge. TuneChimp makes it easy for you to discover the very best music, videos, photos, and more for an artist by mashing up content from imeem, YouTube, Flickr, Yahoo, Last.FM, Google News, and more. TuneChimp takes advantage of the dozens of music-related APIs that are now available on the web to auto-generate an artist profile to quickly discover new artists or play music from your favorites. The most useful feature is that it takes the top tracks from an artist based on robust Last.FM audio-scrobbling data and creates a playable playlist using imeem's on-demand music streaming platform.

The core of the application calls 11 different APIs, appropriately caches the datasets, and cross-links the various dataset to put together a meaningful artist profile. The APIs are all accessible through REST endpoints in either JSON or XML formats.

Pros
The single greatest advantage of Google App Engine is speed to market of an application. TuneChimp was designed as a small weekend project to let me perform competitive analysis on the most popular music-related APIs (I manage the imeem Media Platform and wanted to see how we fared against other offerings). The beauty of GAE was that the same day I started coding I had a basic site up that pulled data from several APIs. Since GAE only supports a narrow web app scenario, it makes it extremely simple to setup, develop, and deploy an app. No need to install an OS, configure apache, nor optimize mysql.

While some complain the datastore APIs are limiting because you can't perform classic relational join operations, anyone who has been involved in a large web app built on an open source stack knows that those operations don't scale anyway. GAE forces you to design for scale from the beginning, but its an easy mental model to learn and super-fast to get up and running. The dbmodel objects are similar to any web framework, like RoR and others, so its also very easy to pick up. And the immediate scalability benefit makes it so you don't have to worry how you are going to handle extra load.

Currently GAE is limited to Python and that has put off a lot of people from trying it. I decided to bite the bullet and learn Python and I have been so happy that I have. I see Python as a great compromise between PHP and Ruby on Rails in that python is more explicit like PHP but cleaner in code and still has many of the productivity benefits of RoR with the GAE framework (or Django framework if you choose).

Cons
Unfortunately I encountered some serious bugs in GAE during my development. One bug prevented any web request in production from returning multiple cookies. Unfortunately many APIs use cookies for authentication and it was impossible to read from certain APIs without implementing hacks. I filed the bug, complained to my contacts at Google, and it still took months for this issue to be addressed. GAE is still clearly a work in progress and the bleeding edge developers who are willing to engage with it now will have to continuously invent hacks to get around these kinds of bugs for some time to come.

In addition the restrictive exceeding high CPU quotas and inflexibly short time-outs make it VERY difficult to reliably build on top of third party APIs with varying response times. I ended up having to build in retry logic and significant caching to try to work around these time-outs. At the same time, without the ability to run long run processes and cron jobs, a developer is forced to continue to host a server outside of the GAE environment to perform batch processing and more.

Probably the greatest detractor from Google App Engine though is the propriety stack that it is built on and the resulting lock-in. This creates significant technology risk for a startup to build on top of GAE since its going to be extremely costly to move to a different infrastructure if necessary. Hopefully some of the projects third parties are working on to port the GAE web framework and datastore will mitigate some of the issues associated with this lock-in.

Overall
While Google App Engine has become my web development platform of choice for all my weekend projects, I would not yet take the risk of running a production web business on top of GAE. The platform though is very promising and I hope to see my concerns addressed over time as well as large web app success stories built on top of this cloud platform.

Welcome

Sachin Rekhi — Thu, 01 Jan 2009 07:34:07 +0000

As a new year's resolution this year I decided to finally start blogging to share some of my thoughts on technology, entrepreneurship, and product management. I'll be writing mainly to help collect my own ideas, but I hope that some of you may find it interesting as well. I'm also often asked similar questions about my experiences from colleagues, so I plan on using this as a public forum to answer some of them.

Along the way I hope to hear from you on any topics you find interesting. So leave a comment, send an email, or connect with me on twitter, LinkedIn, etc.