Real World Web: Performance & Scalability.

NGINX + PHP-FPM + APC.

Gearman is interesting.

Well, he finally lost the appeal. (via)

The socket API uses a 6 byte header containing protocol_version/command_code/data_size. Let’s call this the cc-header.

protocol_version is there for keeping old code functioning when the protocol changes. Currently, version is still 1.

command_code is used for actual commands as well as error codes.

data_size tells you the size of data in bytes following the cc-header. Size often is 0.

If you are sending a picture or receiving picture text there’s a second header following the cc-header. Let’s call this the pic-header.

The pic-header is 20 bytes long and contains pic_timeout/pic_type/data_size/major_id/minor_id.

pic_timeout is used to tell Decaptcher how much time it has to get the captcha back to you.

pic_type serves as an affiliate application id, as far as I know.

data_size is the size in bytes that will follow the pic-header.

major_id and minor_id are sent to you when you get picture text, they are used when reporting picture text as bad.

You append your image binary after the pic-header, when you are sending.

Picture text comes after the pic-header, when you are receiving.

Logging in:

1. Open socket to Decaptcher’s server.
2. Send cc-header, command_code=1, followed by your username.
3. Receive 32 byte salt, with command_code=3.
4. Using sha256, hash the salt,md5 of your password and username (in this order).
5. Send the hash along with a cc-header and command_code=4.
6. Receive cc-header with command_code=7.

Sending a picture:

1. Must be logined.
2. Send cc-header with command_code=12, then a pic-header and then the picture binary.
3. Wait on the socket until you get a cc-header with command_code=14. A pic-header and picture text will follow. You must store the major_id and minor_id you get in the pic-header preceding the picture text (in order to report bad picture text).
4. If command_code is not 14 then it’s an error code.

Notifying of bad picture text:

1. Must be logined.
2. Send a cc-header with command_code=13, then a pic-header containing the major_id and minor_id that came back with the picture text.

Getting your API credits balance:

1. Must be logined.
2. Send a cc-header with command_code=10.
3. Receive a cc-header with command_code=10 and the balance follows as text.

Logging out:

1. Must be logined.
2. Send a cc-header with command_code=2.
3. Close the socket.

The command codes:

‘cmdCC_UNUSED’, 0
‘cmdCC_LOGIN’, 1 // login
‘cmdCC_BYE’, 2 // end of session
‘cmdCC_RAND’, 3 // random data for making hash with login+password
‘cmdCC_HASH’, 4 // hash data
‘cmdCC_PICTURE’, 5 // picture data, deprecated
‘cmdCC_TEXT’, 6 // text data, deprecated
‘cmdCC_OK’, 7 // ok
‘cmdCC_FAILED’, 8 // failed
‘cmdCC_OVERLOAD’, 9 // server overloaded
‘cmdCC_BALANCE’, 10 // zero balance
‘cmdCC_TIMEOUT’, 11 // time out occured
‘cmdCC_PICTURE2′, 12 // picture data
‘cmdCC_PICTUREFL’, 13 // picture failure
‘cmdCC_TEXT2′, 14 // text data

Picture timeout codes:

‘ptoDEFAULT’, 0 // default timeout, server-specific
‘ptoLONG’, 1 // long timeout for picture, server-specfic
‘pto30SEC’, 2 // 30 seconds timeout for picture
‘pto60SEC’, 3 // 60 seconds timeout for picture
‘pto90SEC’, 4 // 90 seconds timeout for picture

The default picture type:

‘ptUNSPECIFIED’, 0 // picture type unspecified

http://svn.blackhat-seo.com/stuff/php/Decaptcher/

Captcha farm services

The following is a list of commercial captcha farms. A few of those seem to be coupled with specific products, which probably means you can’t use them with your own code. Prices are per thousand captchas solved.

• Decaptcher – $2
• SocialMarketServices – ~$2
• Captcha-Bypass – $5
• BypassCaptcha – $7
• BeatCaptchas – $8
• WebEmulator – $10
• ImageToText – $15
• Ocommunity – $25
• ByByeCaptcha – $30
• CaptchaKing – $40

Decaptcher seems to be the best choice for now, with low prices and an assortment of APIs you can use. SocialMarketServices rivals Decaptcher’s prices, especially with their volume discounts, however I haven’t heard from anyone using them.

Advantages of commercial captcha farms

• 24/7 availability of captcha entry workers
• Quick deployment with off the shelf APIs
• Seamless scaling up to moderate volume

Disadvantages

• Higher cost than hiring your own workers
• Too slow when there is excess service load
• They can only handle so much volume

Running your own captcha farm

Having your own private captcha farm might be cheaper but it is a more involved affair. First of all, you need a backend and an interface for your workers. SocialMarketServices and SolveMyCaptcha offer solutions that let you hire your own captcha solvers. You could also surely find an assortment of ready-made captcha farm solutions by posting on one of the freelancer websites.

One important design decision is whether your workers initiate the captcha fetching scripts or poll a queue asynchronously. The first one means your scripts might be running too fast, which leads to either getting banned immediately or having big groups of accounts banned at the same time in the future. Fetching from a queue could frustrate your workers, unless you can provide consistent volume. If you pay your workers per captcha, they won’t be happy when you are not giving them more captchas than they can handle. Keeping your queue full and not overloading it, is a fine balance you have to keep, unless of course you like submitting timed out captchas.

Finding workers to solve captchas should be fairly easy. Here is an example project on a freelance site, there are 113 bids. There is a more than adequate supply of people that will solve captchas for $0.001 each.

You should assign your workers shifts that cover the whole 24 hours, that way you can spread your captcha fetching throughout the day. You might be inclined to divide the day in three eight-hour shifts. Typing captchas for eight hours straight is not an easy feat. I think workers would be more comfortable and efficient with two or four-hour shifts.

Advantages of running your own captcha farm

• Cheaper cost per captcha solved
• Designed for your use case, could add tasks other than captcha solving
• Can scale beyond what the commercial services can handle

Disadvantages

• Time wasted managing servers, software and workers
• Hard to keep workers occupied if you are doing low volume
• Takes time to deploy, with a service you can get started in a couple of hours

A better captcha farm design

If you are serious about setting up your own captcha farm, here are some ideas to consider. Please leave a comment if there’s something you would like to add.

Backend

• Ability to include simple tasks besides solving captchas
• Asynchronous queue for tasks
• When queue is empty, either try to increase the rate of captcha fetching or queue different tasks
• Monitor queue load, send warnings when things aren’t going smoothly
• Offload captchas to a commercial service when things are running hot

User management

• Configurable fixed-time as well as free-running shifts
• Collect statistics on workers, such as: completed tasks, failed tasks, timeouts, task completion speed
• Allow master accounts for managers of worker teams, slave accounts don’t receive payments and can’t choose their shifts

Worker interface

• “Take a break” button
• Tasks pulled from queue via AJAX
• Optional timeout counter attached to tasks, input disabled when time runs out
• Optional input restraints/validation, such as allowed characters
• Move to next captcha input field on
• Allow input field navigation with arrow keys
• Configurable number of tasks displayed simultaneously, vertical list of tasks is better
• Fetch new tasks before the previous ones are finished
• Support translations of the interface

A useful trick

I borrowed this trick from BlackhatWorld. The idea is that you can glue two captchas side by side and have workers treat it as a single captcha with two words, not unlike reCaptcha. Quite the brilliant idea, since I suspect this will not only lower your cost per captcha, but could also increase overall throughput. This is somewhat complicated to implement but I believe it would be time well spent.

Misc links

Here are some possibly defunct services as well as some worker’s login pages. On a sidenote, PixProfit is allegedly the other side of Decaptcher.

http://pixprofit.com/

http://captchasolver.biz/

http://www.captchakiller.com/

http://mrcaptcha.com/

http://captchasolve.com/purchase.php

http://captchahawk.com/

http://picworker.com

http://www.trueonlineworker.com/

http://www.truehardworker.com/

http://www.2dotnetguys.com/CaptchaWeb/CaptchaWebDecodingPage.aspx

http://www.captcha247.com/

http://www.captchawork.cn/

http://www.captchawork.cn/captchatype/

http://solvemycaptcha.com/WorkerLogin.aspx

Here is the source for DeferredChain.

The idea is that you extend DeferredChain and write a bunch of methods prefixed with ‘_’. These don’t have to return $this to enable you to chain them.


class MyChain extends DeferredChain {

// or all steps at once
$results = $m->doAll();


If you are still reading this post, here is a more interesting example.


$d = new DiggChain($user,$pass);

while(($requests = $d->doNext())) {
$c->addArr($requests);
$c->perform();
}


Here is the incomplete DiggChain.

This is how I like to write glue for loosely coupled cURL requests. For example, I wouldn’t use chains for just a series of HTTP requests that login at a website. I have been using the Curl Objects library, which provides a specialized class for tightly coupled HTTP requests. You should probably check that out if you use php-cURL.

Well that’s it for today. Stay tuned for more coding related posts.

If you were naive enough to turn your feed over to Feedburner, you’d be pleased it now is four times as accessible compared to your vanilla Wordpress feed.

Here is a demonstration:

• http://feeds2.feedburner.com/TechCrunch
• http://feeds.feedburner.com/TechCrunch
• http://feedproxy.google.com/TechCrunch
• http://feedproxy.feedburner.com/TechCrunch

Speaking of naiveness… do you know how Feedburner actually works? It’s real simple.

It seems that Google’s recommendation is to redirect anyone without “Feedburner” in their useragent.

So, if someone is using Feedburner and uses automatic excerpts or has spliced a ton of crap in his feed *cough*this blog*cough*, you just need to use “Feedburner” for your feed fetcher’s useragent and grab the full unmodified feed directly from the blog.