But sometimes you just have a simple UIWebView, and you want to cache the last version of the page. You’d think that NSURLCache would handle this for you, but it’s much more complicated than that. NSURLCache doesn’t cache everything you’d think it would. Sometimes this is because of Apple’s decisions in order to save space. Just as often, however, it’s because the HTTP caching rules explicitly prevent caching a particular resource.

What I wanted was a simple mechanism for the following case:

• You have a UIWebView that points to a website with embedded images
• When you’re online, you want the normal caching algorithms (nothing fancy)
• When you’re offline, you want to show the last version of the page

My test case was simple: a webview that loads cnn.com (a nice complicated webpage with lots of images). Run it once. Quit. Turn off the network. Run it again. CNN should display.

Exisiting solutions

The ever-brilliant Matt Gallagher has some interesting thoughts on how to subclass NSURLCache to handle this, but I find his solution fragile and unreliable, especially on iOS 5. The HTTP caching rules are complicated, and in many cases you need to connect to the server to re-validate your cache before you’re allowed to use your local copy. Unless everything works out perfectly, his solution may not work when you’re offline, or may force you to turn off cache validation (which could make your pages go stale).

AFCache is also promising, using essentially the same approach. I haven’t found the offline support to work very well, at least in my tests, for the same reasons as Matt’s solution. It’s designed to be an advanced HTTP-caching solution. The docs are limited and I couldn’t get it to pass my CNN test.

RNCachingURLProtocol

So, I present RNCachingURLProtocol. It isn’t a replacement for NSURLCache. It’s a simple shim for the HTTP protocol (that’s not nearly as scary as it sounds). Anytime a URL is download, the response is cached to disk. Anytime a URL is requested, if we’re online then things proceed normally. If we’re offline, then we retrieve the cached version. The current implementation is extremely simple. In particular, it doesn’t worry about cleaning up the cache. The assumption is that you’re caching just a few simple things, like your “Latest News” page (which was the problem I was solving). It caches all HTTP traffic, so without some modifications, it’s not appropriate for an app that has a lot of HTTP connections (see MKNetworkKit for that). But if you need to cache some URLs and not others, that is easy to implement.

First, a quick rundown of how to use it:

1. At some point early in the program (application:didFinishLaunchingWithOptions:), call the following:

   [NSURLProtocol registerClass:[RNCachingURLProtocol class]];

2. There is no step 2.

Since RNCachingURLProtocol doesn’t mess with the existing caching solution, it is compatible with other caches, like AFCache. In fact, the technique used by RNCachingURLProtocol could probably be integrated into AFCache pretty easily.

The cache itself is stored in the Library/Caches directory. In iOS 5, this directory can be purged whenever space is tight. Keep that in mind. You may want to store your caches elsewhere if offline access is critical.

Understanding NSURLProtocol

An NSURLProtocol is a handler for NSURLConnection. Each time a request is made, NSURLConnection walks through all the protocols and asks “Can you handle this request (canInitWithRequest:)?” The first protocol to return YES is used to handle the connection. Protocols are queried in the reverse order of their registration, so your custom handlers will get a crack at requests before the system handlers do.

Once your handler is selected, the connection will call initWithRequest:cachedResponse:client: and then startLoading. It is then your responsibility to call the connection back with URLProtocol:didReceiveResponse:cacheStoragePolicy:, some number of calls to URLProtocol:didLoadData:, and finally URLProtocolDidFinishLoading:. If these sound similar to the NSURLConnection delegate methods, that’s no accident.

While online, RNCachingURLProtocol just forwards requests to a new NSURLConnection, making copies of the results, and passing them along to the original connection. When offline, RNCachingURLProtocol loads the previous result from disk, and plays it back to the requesting connection. The whole thing is less than 200 lines of pretty simple code (not counting Reachability, which I include from Apple’s sample code to determine if we’re online).

There’s a subtle problem with the above solution. When RNCachingURLProtocol creates a new NSURLConnection, that new connection has to find a handler. If RNCachingURLProtocol says it can handle it, then you’ll have an infinite loop. So how do I know not to handle the second request? By adding a custom header (X-RNCache) to the HTTP request. If it’s there, then we’ve already seen this one, and the handler returns NO.

Again, this intercepts all HTTP traffic. That could intercept pages you don’t want. If so, you can modify canInitWithRequest: to select just things you want to cache (for instance, you could turn off caching for URLs that include parameters or POST requests).

Wrap-up

This technique isn’t a replacement for a full caching engine like AFCache or an offline REST engine like MKNetworkKit. It’s intended to solve a single, simple problem (though it can be extended to solve much more complicated problems). NSURLProtocol is extremely powerful, and I’ve used it extensively when I need to eavesdrop on network traffic (such as in PandoraBoy’s several ProxyURLProtocol classes). It’s well-worth adding to your toolkit.

The code is in the attached project. Look in RNCachingURLProtocol.m.

CachedWebView Example Project

Please send any comments or corrections to me at robnapier@gmail.com. In the meantime, you can take a look at the example code.

I see a lot of example code out there showing how to use CCCrypt(), and most of it is unfortunately wrong. Since I just got finished writing about 10 pages of explanation for my upcoming book, I thought I’d post a shortened form here and hopefully help clear things up a little. This is going to be a little bit of a whirlwind, focused on the simplest case. If you want the gory details including performance improvements for large amounts of data, well, the book will be out later this year. :D

First and foremost: the key. This is almost always done wrong in the examples you see floating around the Internet. A human-typed password is not an AES key. It has far too little entropy. Using it directly as an AES key opens you up to all kinds of attacks. In particular, lines like this are wrong:

// DO NOT DO THIS
NSString *password = @"P4ssW0rd!";
char keyPtr[kCCKeySizeAES128];
[password getCString:keyPtr maxLength:sizeof( keyPtr ) encoding:NSUTF8StringEncoding];
// DO NOT DO THIS

This key is susceptible to a variety of attacks. It is neither salted nor stretched. If password is longer than the key size, then the password will be truncated. This is not how you build a key.

First, you need to salt your key. That means adding random data to it so that if the same data is encrypted with the same password, the ciphertext will still be different. The key should then be hashed, so that the final result is the correct length. The correct way to do this is with PKCS #5 (PBKDF2). Unfortunately, prior to 10.7, there wasn’t an easy function to do that. I’m going to focus here on 10.7 code, but if you need a simple solution, just add about 8 random characters to the the string, and run it through -hash. Stringify that and run it through -hash again. Repeat 100,000 times. Take the lower “X” bits where “X” is the number of bits in your key. This isn’t perfect, but it’s easy to code and close enough. It works on all versions of OS X and iOS. 100,000 times is based on this taking about 100ms for a MacBookPro to calculate in my quick tests. On an iPhone 4, the same delay is around 10,000 iterations. The goal is to force the attacker to waste some time.

(If someone has a better quick-and-easy key generation algorithm, leave a comment.)

But like I said, don’t do that way if you can help it. We have PBKDF2 built into CommonCrypto now (well, in 10.7). Hand it your salt, your password, the number of iterations, and the length of your key and it spits out the answer for you. I’ll show how to do this in the code below.

Did I mention that CommonCrypto is all open source? So if you needed the PBKDF2 code for other platforms, you could probably get it to work.

OK, now you have a salt. What do you do with it? Save it with the cipertext. You’ll need it later to decrypt. The salt is considered public information so you don’t need to protect it.

And now the mystical initialization vector (IV) that confuses everyone. In CBC-mode, each 16-byte encryption influences the next 16-byte encryption. This is a good thing. It makes the encryption much stronger. It’s also the default. The problem is, what about block 0? The answer is you make up a random block -1. That’s the IV.

This is listed as “optional” in CCCrypt() which is confusing because it isn’t really optional in CBC mode. If you don’t provide one, it’ll automatically generate an all-0 IV for you. That throws away significant protection on the first block. There’s no reason to do that. IV is simple: it’s just 16 random bytes. “Save it with the cipertext. You’ll need it later to decrypt. The salt IV is considered public information so you don’t need to protect it.”

OK, now that I’ve gone on and on about theory, let’s see this in practice. First, here’s how you use it. The method returns the encrypted data (nil for error), and returns the IV, salt and error by reference. Slap the data, IV, and salt together in your file in whatever way is easy for you to retrieve them later. The IV has to be 16 bytes long for AES. The salt can be any length, but my code sets it to 8 bytes, which is the PKCS#5 minimum recommended length.

NSData *iv;
NSData *salt;
NSError *error;
NSData *encryptedData = [RNCryptManager encryptedDataForData:plaintextData
                                                    password:password
                                                          iv:&iv
                                                        salt:&salt
                                                       error:&error];

And here’s the code. I will leave the decrypt method as an exercise for the reader. It’s almost identical, and it’s a good idea to actually understand this code, not just copy it. Don’t forget to link Security.framework.

Go forth and encrypt stuff.

#import <CommonCrypto/CommonCryptor.h>
#import <CommonCrypto/CommonKeyDerivation.h>

NSString * const
kRNCryptManagerErrorDomain = @"net.robnapier.RNCryptManager";

const CCAlgorithm kAlgorithm = kCCAlgorithmAES128;
const NSUInteger kAlgorithmKeySize = kCCKeySizeAES128;
const NSUInteger kAlgorithmBlockSize = kCCBlockSizeAES128;
const NSUInteger kAlgorithmIVSize = kCCBlockSizeAES128;
const NSUInteger kPBKDFSaltSize = 8;
const NSUInteger kPBKDFRounds = 10000;  // ~80ms on an iPhone 4

// ===================

+ (NSData *)encryptedDataForData:(NSData *)data
                        password:(NSString *)password
                              iv:(NSData **)iv
                            salt:(NSData **)salt
                           error:(NSError **)error {
  NSAssert(iv, @"IV must not be NULL");
  NSAssert(salt, @"salt must not be NULL");

  *iv = [self randomDataOfLength:kAlgorithmIVSize];
  *salt = [self randomDataOfLength:kPBKDFSaltSize];

  NSData *key = [self AESKeyForPassword:password salt:*salt];

  size_t outLength;
  NSMutableData *
  cipherData = [NSMutableData dataWithLength:data.length +
                kAlgorithmBlockSize];

  CCCryptorStatus
  result = CCCrypt(kCCEncrypt, // operation
                   kAlgorithm, // Algorithm
                   kCCOptionPKCS7Padding, // options
                   key.bytes, // key
                   key.length, // keylength
                   (*iv).bytes,// iv
                   data.bytes, // dataIn
                   data.length, // dataInLength,
                   cipherData.mutableBytes, // dataOut
                   cipherData.length, // dataOutAvailable
                   &outLength); // dataOutMoved

  if (result == kCCSuccess) {
    cipherData.length = outLength;
  }
  else {
    if (error) {
      *error = [NSError errorWithDomain:kRNCryptManagerErrorDomain
                                   code:result
                               userInfo:nil];
    }
    return nil;
  }

  return cipherData;
}

// ===================

+ (NSData *)randomDataOfLength:(size_t)length {
  NSMutableData *data = [NSMutableData dataWithLength:length];

  int result = SecRandomCopyBytes(kSecRandomDefault, 
                                  length,
                                  data.mutableBytes);
  NSAssert(result == 0, @"Unable to generate random bytes: %d",
           errno);

  return data;
}

// ===================

// Replace this with a 10,000 hash calls if you don't have CCKeyDerivationPBKDF
+ (NSData *)AESKeyForPassword:(NSString *)password 
                         salt:(NSData *)salt {
  NSMutableData *
  derivedKey = [NSMutableData dataWithLength:kAlgorithmKeySize];

  int 
  result = CCKeyDerivationPBKDF(kCCPBKDF2,            // algorithm
                                password.UTF8String,  // password
                                password.length,  // passwordLength
                                salt.bytes,           // salt
                                salt.length,          // saltLen
                                kCCPRFHmacAlgSHA1,    // PRF
                                kPBKDFRounds,         // rounds
                                derivedKey.mutableBytes, // derivedKey
                                derivedKey.length); // derivedKeyLen

  // Do not log password here
  NSAssert(result == kCCSuccess,
           @"Unable to create AES key for password: %d", result);

  return derivedKey;
}

I’m actively looking for a new maintainer. It’s a good project, and the code base is in decent repair, though I don’t know if it builds with Xcode 4. The system basically works, but there’s a long list of things that could be improved, and some things that are now downright broken. If you’re interested, let me know and I’ll help hand it off. I took it over when I knew next to nothing about Macs. It’s a great place to learn.

Working at Dell KACE has been awesome. Yes, it’s strange being a “Mac developer at Dell,” but KACE is like this little startup inside of Dell. It’s like having the best parts of a startup, without having to worry about funding. And Mac is a big deal to a lot of our customers, so we take the cross-platform work very seriously. As a low-level guy, it’s interesting keeping everything working smoothly across Windows, Mac and Linux (and what seems an ever-growing number of Linux distros….)

I’ll try to bring back some Cocoa content soon. When not pushing product out the door at KACE, I’ve been continuing work on my upcoming book, which takes up most of my non-KACE time. The title is now official: iOS 5 Programming: Pushing the Limits with Mobile Apps for Apple iPhone, iPad, and iPod Touch. It should be out in a few months. I’m working with Mugunth Kumar, who has been an excellent coauthor, and I’m really pleased to have added Mithilesh Kumar (no relation) as technical editor. I like how the book is shaping up. We’re targeting those who want a more advanced iOS book, focused on the non-obvious things you can’t easily get by skimming the docs. Hope you all like it.

Today’s question is about laying out text without CTFramesetter. We’re going to take a whirlwind tour through some CoreText code to demonstrate this. It’s not quite what the OP was asking about, but it shows some techniques and I had it handy. I’ll be writing a whole chapter on Core Text soon.

The goal of this project was to make “pinch” view. It lays out text in a view, and where ever you touch, the text is pinched towards that point. It’s not meant to be really useful. Everything is done in drawRect:, which is ok in this case, since we only draw when we’re dirty, and when we’re dirty we have to redraw everything anyway. But in many cases, you’d want to do these calculations elsewhere, and only do final drawing in drawRect:.

We start with some basic view layout, and loop until we run out of text or run out of vertical space in the view.

- (void)drawRect:(CGRect)rect {      
  [... Basic view setup and drawing the border ...]

  // Work out the geometry
  CGRect insetBounds = CGRectInset([self bounds], 40.0, 40.0);
  CGPoint textPosition = CGPointMake(floor(CGRectGetMinX(insetBounds)),
                                     floor(CGRectGetMaxY(insetBounds)));
  CGFloat boundsWidth = CGRectGetWidth(insetBounds);

  // Calculate the lines
  CFIndex start = 0;
  NSUInteger length = CFAttributedStringGetLength(attributedString);
  while (start < length && textPosition.y > insetBounds.origin.y)
  {

Now we ask the typesetter to break off a line for us.

    CTTypesetterRef typesetter = self.typesetter;
    CFIndex count = CTTypesetterSuggestLineBreak(typesetter, start, boundsWidth);
    CTLineRef line = CTTypesetterCreateLine(typesetter, CFRangeMake(start, count));

And decide whether to full-justify it or not based on whether it’s at least 85% of a line:

    CGFloat ascent;
    CGFloat descent;
    CGFloat leading;
    double lineWidth = CTLineGetTypographicBounds(line, &ascent, &descent, &leading);

    // Full-justify if the text isn't too short.
    if ((lineWidth / boundsWidth) > 0.85)
    {
      CTLineRef justifiedLine = CTLineCreateJustifiedLine(line, 1.0, boundsWidth);
      CFRelease(line);
      line = justifiedLine;
    }

Now we start pulling off one CTRun at a time. A run is a series of glyphs within a line that share the same formatting. In our case, we should generally just have one run per line. This is a good point to explain the difference between a glyph and character. A character represents a unit of information. A glyph represents a unit of drawing. In the vast majority of cases in English, these two are identical, but there a few exceptions even in English called ligatures. The most famous is “fi” which in some fonts is drawn as a single glyph. Open TextEdit. Choose Lucida Grande 36 point font. Type “fi” and see for yourself how it’s drawn. Compare it to “ft” if you think it’s just drawing the “f” too wide. The joining is on purpose.

So the thing to keep in mind is that there can be a different number of glyphs than characters. High-level Core Text objects work in characters. Low-level objects work in glyphs. There are functions to convert character indexes into glyph indexes and vice versa. So, let’s back to the code. We’re going to move the Core Graphics text pointer and start looping through our CTRun objects:

    // Move us forward to the baseline
    textPosition.y -= ceil(ascent);
    CGContextSetTextPosition(context, textPosition.x, textPosition.y);

    // Get the CTRun list
    CFArrayRef glyphRuns = CTLineGetGlyphRuns(line);
    CFIndex runCount = CFArrayGetCount(glyphRuns);

    // Saving for later in case we need to use the actual transform. It's faster
    // to just add the translate (see below).
    //      CGAffineTransform textTransform = CGContextGetTextMatrix(context);
    //      CGAffineTransform inverseTextTransform = CGAffineTransformInvert(textTransform);

    for (CFIndex runIndex = 0; runIndex < runCount; ++runIndex)
    {

Now we have our run, and we’re going to work out the font so we can draw it. By definition, the entire run will have the same font and other attributes. Note that the code only handles font changes. It won’t handle decorations like underline (remember: bold is a font, underline is a decoration). You’d need to add more code if you wanted that.

      CTRunRef run = CFArrayGetValueAtIndex(glyphRuns, runIndex);
      CTFontRef runFont = CFDictionaryGetValue(CTRunGetAttributes(run),
                                               kCTFontAttributeName);

      // FIXME: We could optimize this by caching fonts we know we use.
      CGFontRef cgFont = CTFontCopyGraphicsFont(runFont, NULL);
      CGContextSetFont(context, cgFont);
      CGContextSetFontSize(context, CTFontGetSize(runFont));
      CFRelease(cgFont);

Now we’re going to pull out all the glyphs so we can lay them out one at a time. CTRun has one of those annoying Get...Ptr constructs that are common in Core frameworks. CTRunGetPositionsPtr() will very quickly return you the internal pointer to the glyphs locations. But it might fail if the CTRun hasn’t calculating them yet. If that happens, then you have to call CTRunGetPositions() and hand it a buffer to copy into. To handle this, I keep around a buffer that I realloc() to the largest size I need. This almost never comes up because CTRunGetPositionsPtr() almost always returns a result.

Note the comment about being “slightly dangerous.” I’m grabbing the internal location data structures and modifying them. This works out because we are the only user of this CTRun, but these are really immutable structures. If two CTRun objects are created from the same data, then Apple is free to return us two pointers to the same object. So it’s within the specs that we’re actually modifying data that some other part of the program is using for a different layout. That’s pretty unlikely, but it’s worth keeping in mind. My early tests of this on a first-generation iPad suggested that this optimization was noticeable in Instruments. On the other hand, I hadn’t applied some other optimizations yet (like reusing positionsBuffer), so it may be practical to get better safety and performance here. I’ll have to profile further.

      CFIndex glyphCount = CTRunGetGlyphCount(run);

      // This is slightly dangerous. We're getting a pointer to the internal
      // data, and yes, we're modifying it. But it avoids copying the memory
      // in most cases, which can get expensive.
      CGPoint *positions = (CGPoint*)CTRunGetPositionsPtr(run);
      if (positions == NULL)
      {
        size_t positionsBufferSize = sizeof(CGPoint) * glyphCount;
        if (malloc_size(positionsBuffer) < positionsBufferSize)
        {
          positionsBuffer = realloc(positionsBuffer, positionsBufferSize);
        }
        CTRunGetPositions(run, kRangeZero, positionsBuffer);
        positions = positionsBuffer;
      }

      // This one is less dangerous since we don't modify it, and we keep the const
      // to remind ourselves that it's not to be modified lightly.
      const CGGlyph *glyphs = CTRunGetGlyphsPtr(run);
      if (glyphs == NULL)
      {
        size_t glyphsBufferSize = sizeof(CGGlyph) * glyphCount;
        if (malloc_size(glyphsBuffer) < glyphsBufferSize)
        {
          glyphsBuffer = realloc(glyphsBuffer, glyphsBufferSize);
        }
        CTRunGetGlyphs(run, kRangeZero, (CGGlyph*)glyphs);
        glyphs = glyphsBuffer;
      }

Now we move around the characters with a little trig. I originally coded this using CGAffineTransforms, but doing the math by hand turned out to be much faster.

      // Squeeze the text towards the touch-point
      if (touchIsActive)
      {
        for (CFIndex glyphIndex = 0; glyphIndex < glyphCount; ++glyphIndex)
        {
          // Text space -> user space
          // Saving the transform in case we ever want it, but just applying
          // the translation by hand is faster.
          // CGPoint viewPosition = CGPointApplyAffineTransform(positions[glyphIndex], textTransform);
          CGPoint viewPosition = positions[glyphIndex];
          viewPosition.x += textPosition.x;
          viewPosition.y += textPosition.y;

          CGFloat r = sqrtf(hypotf(viewPosition.x - touchPoint.x,
                                   viewPosition.y - touchPoint.y)) / 4;
          CGFloat theta = atan2f(viewPosition.y - touchPoint.y, 
                                 viewPosition.x - touchPoint.x);
          CGFloat g = 10;

          viewPosition.x -= floorf(cosf(theta) * r * g);
          viewPosition.y -= floor(sinf(theta) * r * g);

          // User space -> text space
          // Note that this is modifying an internal data structure of the CTRun.
          // positions[glyphIndex] = CGPointApplyAffineTransform(viewPosition, inverseTextTransform);
          viewPosition.x -= textPosition.x;
          viewPosition.y -= textPosition.y;
          positions[glyphIndex] = viewPosition;
        }
      }

Finally, finally, we draw the glyphs and move down a line. We move down by adding the previous-calculated descent, leading and then +1. The “+1″ was added because it matches up with how CTFramesetter lays out. Otherwise the descenders of one line exactly touch the ascenders of the next line.

      CGContextShowGlyphsAtPositions(context, glyphs, positions, glyphCount);
    }

    // Move the index beyond the line break.
    start += count;
    textPosition.y -= ceilf(descent + leading + 1); // +1 matches best to CTFramesetter's behavior  
    CFRelease(line);
  }
  free(positionsBuffer);
  free(glyphsBuffer);
}

So there you have it. It’s a whirlwind tour showing how to lay glyphs out one-by-one. Attached is an example project showing it in real life.

TextDemo