The crunch time diary - when servers don't serve

Last week CloudFactory launched at the TechCrunch Disrupt and took the first public step towards its vision to fight poverty in developing nations through crowdsourcing. We have a lot of experience to share about our launch. 

As Mark and Tom were busy demoing our platform to the curious crowd at TechCrunch Disrupt, the CloudFactory Team huddled together in our Kathmandu Office.  We were aware that a lot of people will try to use CloudFactory after seeing our demo at Disrupt and we wanted to make sure that our server and applications were at their best behavior.

It was a little over midnight here in Kathmandu, of course, when Disrupt kicked off in San Francisco.

CloudFactory team has been pulling all nighters repeatedly for last one month chasing away every last piece of bug. It's a different story that bugs have a way of creeping back into your code no matter how meticulous you are in getting rid of them. Whenever man comes with a better mousetrap, nature immediately comes up with a better mouse, ya know?

CloudFactory runs on a set of Unicorn application servers which handles each incoming request from Nginx (our HTTP server) with a separate, newly created thread. At some point CloudFactory was processing a production run that required extracting contact information from the images of dozens of business cards being uploaded into our system by Disrupt visitors. And suddenly one of our Unicorn servers started misbehaving. We noticed that some of the HTTP requests were getting lost before they could complete leading to a few business cards not being processed at all.  We tried to find out if there was anything weird about the request itself that could be crashing the thread. Nothing!

Bikash, our system admin, jumped into action and it took him a while to figure out that the real culprit was a setting in Unicorn that set the maximum life of the threads at 30 seconds. Our Unicorn servers are configured and optimized for situations that require CloudFactory to handle large number of short, concurrent requests. Apparently, a few tasks were taking longer than usual and consequently being   shut down abruptly by Unicorn after 30 seconds. He swiftly set it to 60 second and problem was solved.

The team breathed a sigh of relief when we realized to our great surprise that we didn't have any major glitch after that. CloudFactory successfully digitized business cards for several dozens of Disrupt visitors on that day.  Our team was delighted to see our product being received well by many who heard pitches and watched demos from Mark and Tom. Mission Accomplished.

The team slept for 12 straight hours the next day.  

Bikash_and_his_coffee_mug

CloudFactory launches at Techcrunch Disrupt SF: a third world startup fighting poverty by connecting businesses to Cloud Labor

SAN FRANCISCO, Sept. 12, 2011 -- CloudFactory, a "third world startup" based in Kathmandu, Nepal launched an innovative Cloud Labor platform at TechCrunch Disrupt today on its way to creating work opportunities for people in developing nations. Businesses can use this platform to design virtual assembly lines for all types of digital work such as inputting data from handwritten forms, extracting data from images, flagging bad content and categorizing things. The assembly lines are then instantly staffed by 500,000 cloud workers from around the world.

"Similar to how Kiva connects people through micro-loans, CloudFactory is connecting people through micro-work," said Mark Sears, CEO of CloudFactory. "We see this market-based approach to poverty alleviation being a key to unlocking the massive human potential in developing countries."

CloudFactory is an example of innovation by the developing world, for the developing world with the platform itself built in Nepal by bright young engineers wanting to help create life-changing work for people in countries like their own. Cloud Labor is an exploding industry that will demand a lot more workers in the years to come and CloudFactory is committed to raising up a handpicked workforce across developing nations through on-the-ground training, testing and equipping of cloud workers.

Early adopters like tech startups are finding unlimited uses for their own CloudFactory assembly lines whether it be at the core of their application or support for internal sales or operations. Since an average developer can build and integrate an assembly line in as little as an hour, innovating is easy without the usual hurdles and costs associated with other outsourcing options.

About CloudFactory

CloudFactory (www.cloudfactory.com), a social enterprise with a staff of 40 passionate people across offices in Nepal and Hong Kong, is creating work for people in developing nations by offering its Cloud Labor API to businesses needing access to an on-demand, scalable workforce. Initially targeted at startups and developers, the pay-as-you-go platform offers a motivated workforce, self-moderation and advanced workflows appropriately called assembly lines. Like in a real factory, every assembly line is a series of task stations where humans and robots team up to get digital work done.

About TechCrunch Disrupt

TechCrunch Disrupt SF 2011(http://disrupt.techcrunch.com) is TechCrunch's second annual conference in San Francisco attracting over 2,500 leading technology innovators and investors and over 150 new startups.  The main conference is preceded by the popular Hackathon -- a 24-hour competition where over 500 top developers present new innovations. TechCrunch Disrupt follows with a format that combines top thought-leader discussions with new product and company launches. Morning executive discussions debate the most timely disruptions in the technology industry. Afternoons host the Startup Battlefield where 30 plus new companies will launch for the first time on stage, selected to present from more than 1000 applications received from around the world. Another 100 early-stage startups will exhibit in Startup Alley. TechCrunch will award a US$50,000 grand prize along with other award recognitions at the conclusion of the conference. The Hackathon is Sept 10-11; the main conference is Sept 12-14 2011 at San Francisco Design Center Concourse, 635 8th Street (at Brannan), San Francisco, CA 94107.

About TechCrunch

TechCrunch (www.techcrunch.com) is a leading technology media network dedicated to obsessively profiling and reviewing new Internet products and companies. Founded by Michael Arrington in 2005, TechCrunch and its network of websites reach over 13 million unique visitors and more than 30 million page views per month. TechCrunch was acquired by AOL in 2011 and operates a global network of websites including dedicated properties in Europe and Japan, as well as specialized industry websites including MobileCrunch, CrunchGear, GreenTech, TechCrunchIT, and CrunchBase. TechCrunch's CrunchBase is the leading open database about startup companies, people, and investors. In addition to TechCrunch Disrupt, TechCrunch hosts other conferences and events, including the Crunchies Awards (http://crunchies2010.techcrunch.com) and various meet-ups worldwide, serving as community platforms for industry conversation and collaboration.

Contact:

Tom Puskarich, VP Solutions
tom@cloudfactory.com
+1-434-907-7498
Twitter: @thecloudfactory

 

Sneak Preview of CloudFactory is Up!

The day has come. After several months of blood, sweat and tears, the fruit of our labor is finally ready and roaring to go live.  Coming Monday, on September 12, CEO Mark Sears and VP of Business Development Tom Puskarich will unveil CloudFactory at Techcrunch Disrupt in San Francisco amidst a 1700-strong crowd of entrepreneurs, business leaders,  hackers, journalists, and investors. Following the launch, CloudFactory will be open for the entire world to get their digital work done through its Cloud Labor API (right now you need an invitation code to set up your own factory in CloudFactory). 

In case this is the first time you are hearing about this new service, CloudFactory is the Cloud Labor API for businesses to build applications that are powered by real people. If you are interested in how it works, you should definitely have a look at our previous blog post. 

It's been quite a journey since this idea of creating an on-demand cloud-based workforce of hundreds of thousands of people was first conceived nineteen months ago.  After all these months of hectic days and sleepless nights we spent arguing, agreeing, and coding, we were able to launch CloudFactory in private beta 4 weeks ago. The plethora of feedback, bug reports, and suggestions we received from our beta users ever since had enabled us to further refine the product.

Here is a sneak preview of CloudFactory we did just over a month ago for Techcrunch Disrupt. Amazing how far we’ve come in even the last month. But take a look, enjoy and look for the full launch in one week.