Pioneering Software

Out Parameters When ARCing

Tue, 06 Mar 2012 21:51:11 -0800

Be careful when assigning values to out-parameters when Automatic Reference Counting. Ensure that there is no auto-release pool in-between where you assign the parameter and where you use the out-parameter. Easily overlooked when using C blocks, especially when passing blocks to Cocoa APIs which can enclose your handler within an auto-release pool without you realising it.

Problem

See example below. It represents the kind of situation not-uncommonly found within Cocoa software: a message argument passes an error by reference, a pointer to a pointer. Please note, the example is entirely fictional. The program below does nothing useful, except to demonstrate the principle.

    1 /*
    2  * Demonstrates EXC_BAD_ACCESS across an auto-release pool boundary.
    3  */
    4 
    5 #import <Foundation/Foundation.h>
    6 
    7 @interface Something : NSObject
    8 
    9 - (void)doWithError:(NSError **)outError;
   10 
   11 @end
   12 
   13 @implementation Something
   14 
   15 - (void)doWithError:(NSError **)outError
   16 {
   17     @autoreleasepool
   18     {
   19         *outError = [NSError errorWithDomain:@"Emergency" code:999 userInfo:nil];
   20     }
   21 }
   22 
   23 @end
   24 
   25 int main(int argc, const char *argv[])
   26 {
   27     @autoreleasepool
   28     {
   29         NSError *__autoreleasing error = nil;
   30         // The argument type implicitly becomes
   31         //
   32         //  (NSError *__autoreleasing *)
   33         //
   34         [[[Something alloc] init] doWithError:&error];
   35         
   36         // At this point, the main thread gives EXC_BAD_ACCESS.
   37         NSLog(@"%@", error);
   38     }
   39     return 0;
   40 }
   41 
   42 @interface NSError(AutomaticReferenceCounting)
   43 
   44 @end
   45 
   46 @implementation NSError(AutomaticReferenceCounting)
   47 
   48 - (void)dealloc
   49 {
   50     // Place a breakpoint here to see when the error deallocates. You will see
   51     // that the error deallocates at the next autorelease pool, naturally.
   52     NSLog(@"dealloc for %@ at %p", NSStringFromClass([self class]), self);
   53 }
   54 
   55 @end
   56

Under the Automatic Reference Counting using the LLVM compiler, the outError argument implicitly adds __autoreleasing to its type. In other words

NSError **

actually becomes

NSError *__autoreleasing *

See indirect parameters for the rationale. This additional implicit qualifier implies that the NSError pointer exists on the stack frame subject to release by an auto-release pool.

No problem so far.

Looking at line 19, you would never deliberately enclose the out-parameter assignment within an autorelease pool. The situation might arise however inadvertently when you attempt to do the same thing from within a completion handler block sent to one of Apple’s APIs, e.g. sending a synchronous URL requestor running a block with a Core Data context. The handler block makes the assignment but while unwinding the stack, a hidden auto-release pool deallocates and reclaims the error object leaving its auto-releasing pointer sitting on the enclosing stack frame dangling in space. When you attempt to access the error, you of course see EXC_BAD_ACCESS; the error has disappeared.

So look out for that!

Solution

The solution is relatively straightforward. Temporarily assign the out-parameter’s value to an automatic (stack frame) variable with __strong or no qualifier. No qualifier is the same as strong. Then just before returning, assign the out-parameter, e.g.

- (void)doWithError:(NSError **)outError
{
    NSError *error = nil;
    @autoreleasepool
    {
        error = [NSError errorWithDomain:@"Emergency" code:999 userInfo:nil];
    }
    
    // The autorelease pool did not deallocate the error because the automatic
    // "error" variable retains a strong reference.
    if (outError && *outError == nil)
    {
        *outError = error;
    }
}

Reference Is Not a Tree

Fri, 13 Jan 2012 01:03:00 -0800

Git is an excellent version control system. However, it does not offer much hand-holding. Linus expects you to read the documentation and understand what you are doing. How rude! One of Git’s primary features includes: making you feel less intelligent than you thought you were! It satisfies this feature quite well. This can prove especially true when dealing with submodules.

Have you ever seen something like this when working with submodules?

fatal: reference is not a tree: d001e26744dab4b92014a77a4552a535b8fddad8

(Your SHA may vary.) Reference is not a tree?

Dangling Commits

When you clone a repository (from now on: repo for short) as submodules within a super-repo, you can initialise and update any submodules recursively using

git submodule update --init --recursive

After doing so however, your submodules will not sit on a branch. Verify this using

git submodule foreach --recursive git branch

You will notice that all the submodules report “no branch.” They all sit on a ‘detached head’ corresponding to the commit for the branch you originally added as a submodule. (Following it so far?)

So here is the problem: commit any changes to the submodules and you commit to a detached head. When you push the submodule to its origin, Git reports “everything up-to-date” because it pushes the master by default. “Everything up-to-date” refers to the master and not to the detached head. This can easily catch you out, and doubly so if you use a GUI tool where you might not so easily notice the lack of ‘pushing noise’ so to speak.

Prevention Better Than Cure

What to do? When pushing commits within submodules, make sure you move the submodule to a branch, e.g.

git submodule foreach --recursive git checkout master

moves all repos to their master branch for the submodules and sub-submodules recursively. Do this before committing if you can remember.

Reattaching Heads

If you forget, you can conveniently verify that you have dangling commits by checking out the master branch within your submodule. Git warns you about ‘leaving commits behind.’ You can also verify the dangling commit using

git submodule foreach --recursive git fsck --no-reflog

which outputs things like:

dangling commit d001e26744dab4b92014a77a4552a535b8fddad8

(Of course, your SHA may vary again.)

The fix for detached heads temporarily adds a branch, merges it with master and discards the temporary branch. Use the following Git command sequence within your detached head.

git branch detached
git checkout master
git merge detached
git branch -d detached

You can use any name for the detached branch. Hereafter, you can happily push your formerly orphaned changes.

Developing a project as an OS X framework and iOS static library at the same time

Fri, 05 Aug 2011 05:20:00 -0700

You want to develop a framework for OS X and iOS at the same time!

This is not a strange thing since OS X and iOS are substantially similar underneath: a Unix box running Cocoa. With Lion, Apple seem determined to make them even more similar. And why not?

You want both framework and library to share identical sources so that you do not need to maintain two mostly-similar sets. If there are any differences, you want to limit that only to actual differences, not just differences in the way they need compiling and linking.

There is a simple way to do it.

To explain how, take a simple example.

The scenario: You want to develop a framework called MyKit. You want this compiling to a Mac OS X framework for use on OS X and as a static library for use on iOS. The final build products will be:

MyKit.framework with the standard framework layout
libMyKit.a, a static library with headers:
- MyKit.h, the monolithic public header
- MyKit/MyKitPrivateHeader.h, one (or ultimately more) private headers

Developer tools: Xcode 4.2, build 4D139.

How to Use This Article

I have pushed a Git repository for a sample MyKit project at GitHub. I recommend examining the commit history in concert with the steps outlined below. You can see what has changed at each step.

The short version looks like this, where MyKit stands for your framework-library’s product name:

Locate the Product Name build setting for target MyKit (the framework target) and change it from $(TARGET_NAME) to MyKit.
Rename the MyKit framework target from MyKit to MyKitFramework. This just renames the target, not the product since we have disconnected the two; product name no longer depends on target name.
Rename MyKitTests target to MyKitFrameworkTests. This time however, do not rename the Product Name for this target. Let the product name reflect the target name.
Delete MyKit.h and MyKit.m, the stubs supplied by the Cocoa Framework template—really delete, not just delete references.
Add a Cocoa-based Objective-C category file. Category on NSObject. Save as NSObject+MyKit. (This is just for testing. You might also just add existing sources to the framework or create new ones, as many as you like.)
Add implementations along with a little unit testing.
Change the NSObject+MyKit.h header’s target membership for the MyKitFramework target from Project to Private.
Create a monolithic header. Make it Public. It serves only to import the private headers.
Choose the Cocoa Touch Static Library template for the new target. Include Unit Tests. Call the product name “MyKitLibrary.”
Delete the stub object sources: MyKitLibrary.h and MyKitLibrary.m. Not required.
Also delete the test sources, MyKitLibraryTests.h and m. Also not needed.
Change the product name from $(TARGET_NAME) to MyKit for the MyKitLibrary target. Also add NSObject+MyKit.h and m to this target. Make the header a Private member.
Change ___VARIABLE_bundleIdentifierPrefix:bundleIdentifier___ in the bundle identifier for the test bundle to your company name as a reversed DNS string.
Add MyKitTests.m to the MyKitLibraryTests target.
Add -all_load to the Other Linker Flags for the MyKitLibraryTests target. Now the tests run successfully.
Add MyKit.h to the public headers for MyKitLibrary target.
Change your library target’s public and private header folder from /usr/local/include, the default settings, to include and include/$(PROJECT_NAME) respectively.

For the long version of the story, read on.

Create an OS X Cocoa Framework

This is the starting point. Call the product MyKit, include unit tests, and use version control. Xcode stubs out a basic template with two targets: one for the framework and a bundle for the unit tests.

Product Does Not Equal Target

This is a key ingredient. We want multiple targets by different names to create products with the same name, albeit with differing outcomes.

Start by disconnecting the dependency between the target and product names. By default, the product name is the target name. Make the product name equal to MyKit so that you can rename the framework to MyKitFramework.

Step 1: Locate the Product Name build setting for target MyKit (the framework target) and change it from $(TARGET_NAME) to MyKit.

Step 2: Rename the MyKit framework target from MyKit to MyKitFramework. This just renames the target, not the product since we have disconnected the two; product name no longer depends on target name.

You can now rebuild and retest the framework (Command+U shortcut). The project performs both successfully; although the default unit test successfully fails! Check out the framework under Products. You will notice that its name remains MyKit. Control-click and select Show in Finder to see the evidence directly. Framework product name remains unchanged.

Do the same for the unit test bundle.

Step 3: Rename MyKitTests target to MyKitFrameworkTests. This time however, do not rename the Product Name for this target. Let the product name reflect the target name.

Now you have two targets:

MyKitFramework
MyKitFrameworkTests

But you have sources:

MyKit
MyKitTests

Public and Private Headers

The Cocoa Framework template only gives a stub for a class called MyKit. It derives from NSObject and does nothing. Make the MyKit.h header a private header. This will become the monolithic header for importing all private headers.

For something our framework will do, let us make it extend a standard object class with a category. This is the most awkward kind of library-oriented thing for Objective-C. Mac OS X handles it gracefully, but iOS has a little issue with loading category-only libraries. We will need to add an addition linker option -all_load later on.

So, add a new category on NSObject called MyKit which adds a new method to NSObject called -stringFromClass, an instance method which answers the class name as a string for this instance. Not a very useful method; it amounts to NSStringFromClass([self class]). Also, something very similar already exists, i.e. -className appears as a scripting extension, category NSObject(NSScriptClassDescription). Still, it will serve as a piece of very simple functionality, for the purpose of testing out linking against a category-only library.

Step 4: Delete MyKit.h and MyKit.m, the stubs supplied by the Cocoa Framework template—really delete, not just delete references.

Step 5: Add a Cocoa-based Objective-C category file. Command+N is the shortcut. Category on NSObject. Save as NSObject+MyKit. Xcode provides the h and m extensions appropriately for the header and source. Include in the MyKitFramework target, the default selection.

Step 6: Add the implementation along with a little unit testing.

    1 - (NSString *)stringFromClass
    2 {
    3     return NSStringFromClass([self class]);
    4 }

Unit tests:

    1 - (void)testStringFromClass
    2 {
    3     STAssertEqualObjects([[[NSObject alloc] init] stringFromClass], @"NSObject", nil);
    4     
    5     // NSString deploys a class clustering architecture. The actual class is an
    6     // implementation-specific sub-class or compatible class, depending on what
    7     // kind of string, and presumably what version of Cocoa and on what platform
    8     // since the exact underlying class might change. Be prepared for test
    9     // breakage.
   10     STAssertEqualObjects([@"" stringFromClass], @"__NSCFConstantString", nil);
   11     
   12     // This is freaky. You would not expect this to work. But it does; classes
   13     // are also objects. Invoking an instance method on a class: it compiles and
   14     // runs! You would expect the compiler to baulk, but no.
   15     STAssertEqualObjects([NSObject stringFromClass], @"NSObject", nil);
   16 }

Notice that the first assertion does not autorelease the NSObject instance. The project uses Automatic Reference Counting (ARC).

Step 7: Change the NSObject+MyKit.h header’s target membership for the MyKitFramework target from Project to Private. This will mean that applications using the framework cannot directly import the header. Instead they will import the monolithic header MyKit.h which will include all the headers for the framework.

Note that the unit test module still compiles, even though it imports “NSObject+MyKit.h” because Xcode resolves this header against the workspace.

Step 8: Create monolithic header. Make it Public. It serves only to import the private headers.

Now the framework is complete. Next the iOS library.

Add Static Library Target

Add the Cocoa Touch Static Library target for iOS under iOS Framework & Library in Xcode’s template chooser, giving it the product name as the target name MyKitLibrary but then subsequently renaming the product. This differing initial target name prevents the library template from stomping over the existing framework sources.

Step 9: Choose the Cocoa Touch Static Library template for the new target. Include Unit Tests. Call the product name “MyKitLibrary.” This will become the product name, initially, as well as the target name.

Step 10: Delete the stub object sources: MyKitLibrary.h and MyKitLibrary.m. Not required.

Step 11: Also delete the test sources, MyKitLibraryTests.h and m. Also not needed.

Step 12: Change the product name from $(TARGET_NAME) to MyKit for the MyKitLibrary target. Also add NSObject+MyKit.h and m to this target. Make the header a Private member.

There is a bug in Xcode 4.2’s static library template. It writes ___VARIABLE_bundleIdentifierPrefix:bundleIdentifier___ in the bundle identifier for the test bundle.

Step 13: Change this to your company name as a reversed DNS string.

However, you cannot yet run the tests, because there are none defined for the MyKitLibrary target.

Step 14: Add MyKitTests.m to the MyKitLibraryTests target.

The test fails at run-time because linking against the library does not automatically import categories.

Step 15: Add -all_load to the Other Linker Flags for the MyKitLibraryTests target. Now the tests run successfully.

Step 16: Add MyKit.h to the public headers for MyKitLibrary target.

Header Search Within the Workspace

One more thing to do.

You do not need to add special header search paths for your client application—not even for iOS applications. Thanks to Xcode 4, the build process automatically picks up the headers from their respective build locations.

Step 17: To make this searching work though, you need to change your library target’s public and private header folder from /usr/local/include, the default settings, to include and include/$(PROJECT_NAME) respectively. If you want to be fancy and make them interdependent, make the public header folder path equal to include and the private equal to $(PUBLIC_HEADERS_FOLDER_PATH)/$(PROJECT_NAME).

Why include? This is because if you build an iOS application with your library as a dependency and a nested project, Xcode builds everything within a single DerivedData folder. When building the application project and all its dependencies, Xcode automatically adds a header search path at include within this folder. Installing the library headers at this location lets enclosing projects find them easily.

Note however, you might notice that compiling fails when you attempt to archive. The archive step does not automatically search the include directory. Instead you need to add it manually, using HEADER_SEARCH_PATHS = $(DSTROOT)/include in your target build settings.

Renaming the Schemes

You might want to consider renaming the MyKit scheme to MyKitFramework. You then have two schemes: MyKitFramework for the framework and MyKitLibrary for the library. Logical!

ASI HTTP Request Library Review

Thu, 09 Jun 2011 03:19:00 -0700

ASI has a useful class library for wrapping around the CFNetwork framework, the networking Core Foundation framework designed by Apple as a programming interface to lower-level network capabilities of iOS and OS X; CFNetwork framework is available on both platforms.

But how useful does All-Seeing Interactive’s work prove to be for interacting with RESTful web services?

Pros

It works. That’s a good start. But not the only important thing. The codebase should provide a foundation, something to build on. Therefore working is one important quality, but it also needs to provide flexible building blocks on which to build higher level functions of different kinds.

Useful Features

They include:

Handles connection retries and redirections
Handles proxies and authentication
Has data compression and decompression capabilities
Has useful operation sub-class for interacting with Amazon S3 (simple static storage)

Extra Dependencies

Not too many. Apart from Apple’s Cocoa umbrella framework, run-time dependencies include:

libz
libxml2
libtidy

These libraries are part of the standard platform for both iOS and OS X.

For build-time and testing dependencies adds GHUnit framework.

Cons

Cons mainly concern implementational issues.

Request Is an Operation

ASIHTTPRequest is an operation to process a request, not the request itself. Hence the request has a delegate too. This jars with Foundation framework semantics where requests are the request objects not behavioural elements which make requests and collect responses. It’s a English language problem: “request” is a noun and a verb—a source of ambiguity.

Class Architecture

ASIHTTPRequest class carries an enormous amount of ‘state’ and sub-state, in the form of a whopping 141 instance variables! In addition, there is a considerable amount of static state: 35 statics.

In general, this indicates a lack of class-architecture decomposition. Such approaches throw functionality into the one class, making it easy to implement in the short run, but hard to maintain and error prone in the long run. Cross-coupling leads to unexpected and hard-to-debug side effects. You often find that this approach leaves the design with lots of class-scoped methods inappropriately hanging from classes and nested behaviours switching on various internal and typically undocumented states and sub-states.

This is the case with ASIHTTPRequest.

Threading and Locks

The implementation uses old-style threading and locks. Locks have a performance cost, introduce blocking, and scope for race conditions.

However, as of version 4.0, iOS support blocks and Grand Central Dispatch which makes better use of multi-cored hardware and multi-threading systems, and makes implementing concurrent operations more robust.

Other Noteworthy Cons

Other notes:

Uses a cascading tree of request operations. Every request has a main request.
Does not delineate public and private methods, e.g. using categories.
Seems to overlap built-in functionality to some extent. iOS and OS X support some functions such as caching, cookies and authentication at some level.
Request operations run in a single background thread.

Conclusions

The library has some useful functionality, but at mid-2011 needs some additional work. It does not including any kind of routing functionality for RESTful transactions. And at its core lies a heavy-weight HTTP request operation into which the developers have folded most of the available functionality. For ongoing development goals, it would be nice to see some of the class functionality teased apart with less coupling and more flexibility.

-sendAsynchronousRequest:queue:completionHandler:

As it turns out, much of the requirement for handling URL connections will be appearing in iOS 5.0; this includes queuing asynchronous request operations, which is the primary focus of ASI’s library. In the new version of the operating system, you can send -[NSURLConnection sendAsynchronousRequest: queue: completionHandler:] to run an asynchronous URL request in some operation queue. It invokes a completion handler on the main thread when the request completes. Your block receives the NSURLResponse and the data. This new API becomes available in OS X 10.7 Lion and iOS 5.0!

iOS 5.0 defines the following interface.

/*!
    @category NSURLConnection(NSURLConnectionQueuedLoading)

    The NSURLConnectionQueuedLoading category on NSURLConnection
    provides the interface to perform asynchronous loading of URL
    requests where the results of the request are delivered to a
    block via an NSOperationQueue.

    Note that there is no guarantee of load ordering implied by this
    method.
 */
@interface NSURLConnection (NSURLConnectionQueuedLoading)

/*!
    @method       sendAsynchronousRequest:queue:completionHandler:

    @abstract 
                  Performs an asynchronous load of the given
                  request. When the request has completed or failed,
                  the block will be executed from the context of the
                  specified NSOperationQueue.

    @discussion
                  This is a convenience routine that allows for
                  asynchronous loading of an url based resource.  If
                  the resource load is successful, the data parameter
                  to the callback will contain the resource data and
                  the error parameter will be nil.  If the resource
                  load fails, the data parameter will be nil and the
                  error will contain information about the failure.

    @param
         request   The request to load. Note that the request is
                   deep-copied as part of the initialization
                   process. Changes made to the request argument after
                   this method returns do not affect the request that
                   is used for the loading process.

    @param 
         queue     An NSOperationQueue upon which    the handler block will
                   be dispatched.

    @param
         handler   A block which receives the results of the resource load.
 */
+ (void)sendAsynchronousRequest:(NSURLRequest *)request
                          queue:(NSOperationQueue*) queue
              completionHandler:(void (^)(NSURLResponse*, NSData*, NSError*)) handler NS_AVAILABLE(10_7, 5_0);
           
@end

CFNetwork Framework, OS X versus iOS

Mon, 06 Jun 2011 23:18:00 -0700

A question: What are the differences, if any, between CFNetwork framework on iOS and the same framework under OS X?

Mac OS X 10.6 (Snow Leopard)

The framework in question lives under CoreServices in OS X. Look under /System/Library/Frameworks on your Macintosh HD, then under CoreServices.framework, an umbrella framework for the following:

AE.framework
CFNetwork.framework, the framework in question
CarbonCore.framework
DictionaryServices.framework
LaunchServices.framework
Metadata.framework
OSServices.framework
SearchKit.framework

iOS 4.3

Assuming you have Xcode installed with the necessary iOS platform, you will find CFNetwork.framework in a different place, under the platform SDK’s system library frameworks at /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS4.3.sdk then under sub-folder System/Library/Frameworks.

Comparison

Note, you cannot compare the frameworks directly. Frameworks on OS X and iOS have differing layouts. On iOS, frameworks lay out in a much simpler way compared to their OS X counterparts. Also, you cannot compare the binary files. The two different platforms, Intel versus ARM, make machine-code comparisons even more meaningless.

So compare just the framework Headers sub-folder using the following shell command.

$ opendiff /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CFNetwork.framework/Versions/A/Headers /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS4.3.sdk/System/Library/Frameworks/CFNetwork.framework/Headers

Note, opendiff is a command-line front-end to Apple’s FileMerge tool, a graphical ‘diff!’

Results

Only three headers of twelve differ.

CFHTTPMessage.h, 4 differences
CFProxySupport.h, 5 differences
CFSocketStream.h, 1 difference

iOS New Features

The details are slightly surprising. In short, iOS has extra features. Perhaps you might assume the reverse. But no.

Negotiate v2 and Kerberos Authentication in iOS 3.0

iOS 3.0 adds support for HTTP Negotiate v2 and Kerberos authentication schemes. The iOS header promises that this will appear in Lion, OS X 10.7. It carries an __OSX_AVAILABLE_STARTING macro to that effect.
Proxy PAC JavaScripts in iOS 3.0

Provides access to the full JavaScript source of a proxy’s PAC script.
Stream Service Types in iOS 4.0

Supports VoIP.

Conclusion

There is no significant difference between CFNetwork.framework headers on iOS and OS X. Therefore, logically, no significant difference between functionality even if the implementation details differ somewhat. Main conclusion: Develop against one, and you develop against the other at the same time.

As an aside, it appears as though they are two snapshots of the same code base—as you might expect because the two platforms are merging!

Import CSV files in Rails 3

Tue, 13 Jul 2010 07:52:29 -0700

CSV is a ubiquitous importing and exporting format for tables of information conforming to row-column layout. Lines correspond to rows. On each line, commas separate values under each column; hence Comma-Separated Values. Values between commas define table-cell contents, strings or numbers. The format is relatively flexible, relatively easy for machines and even humans to read and write.

In terms of user stories, it answers the following requirement.

As a: database maintainer

I want to: import information from external files, particularly Comma-Separated Value files where rows appear on each line and columns within each row have comma delimiters

So that I can: select columns (and rows) from the CSV files and merge these to create other new pieces of information.

Question is, How do you import CSV files in Rails 3. This article attempts an answer, or at least part of an answer, enough perhaps to point you in the right direction. You can find the sources here at GitHub.

Prerequisites

Rails 3
FasterCSV gem (Ruby parlance for an installable package)
JQuery

Why?

That is an important question. Has this sort of thing not been done before, many times? Well, yes. This is nothing new. But it does offer a fresh look at the subject, especially in the light of the upcoming new version of Rails and with preference for jQuery over Prototype for Rails’ JavaScript library. If you like, consider this a scenic voyage into Rails 3 and jQuery. The seas are fairly calm even though only at Beta.

Check your versions

Entering rails -v on the command line reports,

Rails 3.0.0.beta4

At the time of writing this version is the latest prerelease. Install it using sudo gem install rails --pre; option pre allows installation of prerelease versions, i.e. betas.

Doing similarly for RubyGems using gem -v gives,

1.3.7

Test-driven approach

Begin with a new Rails application.

The command to create a new application has changed since Rails 2. The Rails project now brings together the various disparate scripts and generators under one umbrella, so to speak. New applications now appear when you ask,

rails new APP_PATH

where APP_PATH specifies where your Rails application tree will reside on the local file system. The exact name does not actually matter too much. Although the name does become part of the application proper: it names the application module, and appears in the cookie store key; but you can change these later if you wish.

I will give it a long name. The following command creates a new folder called import-csv-files-in-rails-3. Rails fills it with basic structure and content, enough to get you started.

rails new import-csv-files-in-rails-3

The application is ready to run already. It does not do anything yet, of course. However, you can launch the server and access the application at localhost:3000. Note, syntax for launching a local server has also changed in Rails 3. You no longer enter script/server; it is now in the slightly-less wordy form,

rails s

s for server, naturally.

Test bed

As a simple starting point, imagine a Rails application comprising people and places. People have a first and last name; places have an address. Let the application’s database store these two entities, along with these attributes. Person entity with first and last name attributes, both strings. Place entity with address attribute, a simple string for now.

Generating the basic Rails scaffolding is very easy. Just two commands gives us the entities and their properties, together with corresponding controllers and views to access them via HTML and RESTful XML. g is short for generate in the following two commands.

rails g scaffold person first_name:string last_name:string
rails g scaffold place address:string

Migrate the database using rake db:migrate to update the database schema. By default Rails will use SQLite. Good enough for this exercise. You can then launch a local server to access people at localhost:3000/people and places at localhost:3000/places.

Once set up, the goal will become to import people and place information using CSV files. This is the real challenge.

Test import

One more thing. You may not have a list of people and places handy. But a CSV file or two will definitely be very useful for testing. So just for testing purposes, let us write a script to generate a file.

See below. Copy this and run it to generate some CSV files comprised of random words using Dan Croak’s Webster gem, a package for randomly generating short non-offensive words. You will need to install the gem using sudo gem install webster. Do the same for fastercsv if you haven’t already.

#!/usr/bin/env ruby

require 'optparse'
require 'ostruct'

options = OpenStruct.new
options.rows = 10
options.columns = 10
OptionParser.new do |opts|
  opts.on('-r', '--rows [INTEGER]', 'Number of rows', Integer) do |x|
    options.rows = x
  end
  opts.on('-c', '--columns [INTEGER]', 'Number of columns', Integer) do |x|
    options.columns = x
  end
end.parse!

require 'rubygems'
require 'fastercsv'
require 'webster'

webster = Webster.new

ARGV << '/dev/stdout' if ARGV.empty?
ARGV.each do |file_name|
  FasterCSV.open(file_name, 'w') do |csv|
    options.rows.times do |row_index|
      row = []
      options.columns.times do |column_index|
        row << webster.random_word
      end
      csv << row
    end
  end
end

You will find this script here as a Gist. Download it, make it executable and run using,

./gen-webster-csv.rb test.csv

Two-stage process: upload and merge

For simplicity, the application design will take a two-stage approach to importing.

Stage 1: upload
Stage 2: merge

Here, upload refers to the action of receiving a raw CSV file from a browser and storing it somewhere within the application’s database or otherwise readily-accessible place. Merge refers to the action of moving selected columns to the database proper. In a real application, you might choose to fold both stages into one action. But for this exercise, we make them distinct. The first stage takes a CSV file and stores it. The second stage lets the user pick out one or more columns and choose where to move the data: to people or places within the database. This second stage does have an advantage: it allows the user to see the imported information and select and deselect columns. Only selected columns will merge and become useful to the app.

Uploading, stage 1

First upload the CSV file, then store it.

Uploading files

Uploading a file is straightforward. Just add a form with a ‘file field.’ When submitted, the form will send a POST request with the file contents attached.

Create a CSV controller with an import action.

$ rails g controller csv import
      create  app/controllers/csv_controller.rb
       route  get "csv/import"
      invoke  erb
      create    app/views/csv
      create    app/views/csv/import.html.erb
      invoke  test_unit
      create    test/functional/csv_controller_test.rb
      invoke  helper
      create    app/helpers/csv_helper.rb
      invoke    test_unit
      create      test/unit/helpers/csv_helper_test.rb

Notice, Rails creates a default controller, a GET-request route for the action, an ERB-based view for the action along with test and helper stubs. Hence, when an external browser sends the application a GET request at csv/import, Rails will match the route, find the ERB template, render the HTML response and reply. The ‘view’ only needs to incorporate a standard HTML form which the browser will promptly display.

Navigate to the import view, app/views/csv/import.html.erb and add the HTML form, as follows.

<%= form_for :upload, :html => { :multipart => true } do |f| -%>
  <%= f.file_field :csv %>
  <%= f.submit "Upload", :disable_with => 'Uploading...' %>
<% end -%>

TextMate makes this very easy to type. Type ff then press Tab to lay out the basic form_for skeleton. Use : then Tab to add a key-value pair of the form :key => "value"; type the key, press Tab and type the value. Tab again moves you to the comma delimiter, and so on.

The most important part of the form is the :html => { :multipart => true } option. This ensures that the POST-request parameters will contain the actual file contents. The request will contain a parameter called upload (as defined by the first argument of the form_for) which maps to a dictionary with key csv mapping to a File object which the application can parse to extract the uploaded CSV. Within the Rails controller for handling the POST request, the File object will be found by accessing params[:upload][:csv].

When the user fills in the form and clicks Upload, the browser will send a POST request containing the CSV-file contents. The Rails application needs to respond. Hence it needs a route describing how the request and URL path translates to controller and action. Under Rails 3, this becomes very easy. Just add the following line to your config/routes.rb. Here, csv/import specifies the URL path; while csv#upload specifies controller and action respectively. When Rails routing sees a POST at that URL path, it will invoke method upload of the CSV controller, class CsvController.

  post "csv/import" => 'csv#upload'

This route needs a corresponding action for the CSV controller. Add this method to your CsvController class, source at app/controllers/csv_controller.rb under the application’s source tree.

  def upload
    # While under development, just respond by rendering some in-line text.
    # Send back the request parameters in JSON (JavaScript Object Notation)
    # format, i.e. something reasonably easy to parse with the human eye.
    render :text => params.to_json
  end

Store the CSV contents

Where to store the imported CSV information? We could store the CSV file verbatim somewhere on a local file system; that would be typical. But let us assume that there is no local file system such as for web applications running in the cloud.

Storing the CSV temporarily in some kind of document-based key-value store like MongoDB or CouchDB would be another alternative. It would live outside the database. The application database would contain a link to the external ‘document’ and read it when required. But no. For now, let us keep it fairly simple. Instead, we will conceptually map the CSV to a simple data model and store the CSV information within the database itself. See data model below.

ImportTables have many ImportCells; ImportCells have row and column indices, string contents and belong to one particular ImportTable. The database will index the cells by their import_table_id (back-reference to the owning import table) in order to pull all cells belonging to a table quickly from the database into main memory.

Implementing this model in Rails proves relatively easy.

$ rails g scaffold import_table original_path:string
      invoke  active_record
      create    db/migrate/20100715123031_create_import_tables.rb
      create    app/models/import_table.rb
      invoke    test_unit
      create      test/unit/import_table_test.rb
      create      test/fixtures/import_tables.yml
       route  resources :import_tables
      invoke  scaffold_controller
      create    app/controllers/import_tables_controller.rb
      invoke    erb
      create      app/views/import_tables
      create      app/views/import_tables/index.html.erb
      create      app/views/import_tables/edit.html.erb
      create      app/views/import_tables/show.html.erb
      create      app/views/import_tables/new.html.erb
      create      app/views/import_tables/_form.html.erb
      invoke    test_unit
      create      test/functional/import_tables_controller_test.rb
      invoke    helper
      create      app/helpers/import_tables_helper.rb
      invoke      test_unit
      create        test/unit/helpers/import_tables_helper_test.rb
      invoke  stylesheets
   identical    public/stylesheets/scaffold.css
$ rails g model import_cell import_table_id:integer row_index:integer column_index:integer contents:string
      invoke  active_record
      create    db/migrate/20100715123207_create_import_cells.rb
      create    app/models/import_cell.rb
      invoke    test_unit
      create      test/unit/import_cell_test.rb
      create      test/fixtures/import_cells.yml

To establish the one-to-many association between import table and import cell, add two new lines: a has_many and a belongs_to. Your app/models/import_table.rb and app/models/import_cell.rb should look like this. The import table’s :dependent => :destroy option instructs Rails to delete its cells when you delete a table.

class ImportTable < ActiveRecord::Base
  has_many :import_cells, :dependent => :destroy
end

class ImportCell < ActiveRecord::Base
  belongs_to :import_table
end

For speedy performance, you also need to add an index for the import cells. Generate a migration.

$ rails g migration add_index_for_import_cells
      invoke  active_record
      create    db/migrate/20100715124738_add_index_for_import_cells.rb

Then edit the newly-created migration code, adding instructions to add and remove an index. Without this, whenever you ask Rails for a table’s cells using table.import_cells where table references an instance of ImportTable, the answer will arrive but slowly because the database cannot quickly lookup the cells given a particular import_table_id which uniquely identifies the import table. Note also, the index is non-unique. The answers to table.import_cells which navigate the one-to-many table-to-cell association is an array of many ImportCell instances.

class AddIndexForImportCells < ActiveRecord::Migration
  def self.up
    add_index :import_cells, [:import_table_id]
  end

  def self.down
    remove_index :import_cells, :column => [:import_table_id]
  end
end

Finally, do not forget to run the ‘up’ migrations to upgrade your database schema. Run rake db:migrate.

Upload action

Thus far, we have the means to store CSV table information, but do not yet have the code in place to parse the uploaded CSV into import tables. The CsvController method below uses the FasterCSV gem to parse the incoming file. Edit your app/controllers/csv_controller.rb source to look like this.

require 'fastercsv'

class CsvController < ApplicationController
  def import
  end

  def upload
    table = ImportTable.new :original_path => params[:upload][:csv].original_path
    row_index = 0
    FasterCSV.parse(params[:upload][:csv]) do |cells|
      column_index = 0
      cells.each do |cell|
        table.import_cells.build :column_index => column_index, :row_index => row_index, :contents => cell
        column_index += 1
      end
      row_index += 1
    end
    table.save
    redirect_to import_table_path(table)
  end
end

The code constructs a new import table passing the original file’s path; original_path is a column in the ImportTable database table. The upload method uses the build method on the to-many association between table and cell in order to build new cells and attach them to the table. After parsing, the method saves the table then redirects the user to its ‘show’ page so that the user can see the newly-imported data.

Add the following to Gemfile for Rails 3. This tells Rails to prepare the package for import. Without this you cannot do a require 'fastercsv'. Gemfile appears in the root directory of the application folder. Although, if you are using TextMate on a Mac, locating any file no matter where it lives is very quick using Command+T then start typing the name. A list of matches starts to appear. Just pick the one you want.

gem 'fastercsv'

Then to view the uploaded table, add these lines to your ImportTablesController’s show method.

    @import_cells = @import_table.import_cells
    @row_index_max = @import_cells.map { |cell| cell.row_index }.max
    @column_index_max = @import_cells.map { |cell| cell.column_index }.max

Add the lines above after finding the import table, i.e. after @import_table = ImportTable.find(params[:id]). Then display cell contents in app/views/import_tables/show.html.erb by adding this piece of Ruby-embedded HTML.

<table border="1" cellspacing="1" cellpadding="1">
  <% 0.upto(@row_index_max) do |row_index| %>
    <% row = @import_cells.select { |cell| cell.row_index == row_index } %>
    <tr>
      <% 0.upto(@column_index_max) do |column_index| %>
        <td>
          <%= row.select { |cell| cell.column_index == column_index }[0].contents %>
        </td>
      <% end %>
    </tr>
  <% end %>
</table>

Given the import cells collected by the CSV controller together with the index maxima, this view code builds a standard HTML table comprised of tr and td (table row and table data) tags. The row = line (line 3) optimises the iteration somewhat by first collecting together all cells matching the current row index. This is a smaller number of cells from which to select individual cells matching the column index within the inner loop.

Merging, stage 2

At this point, you can navigate to http://localhost:3000/csv/import. You can generate and drop a CSV file from Finder onto the Choose File button (a Mac shortcut for choosing a file) and click Upload. The Rails application then uploads and builds an import table of cells. By redirection, the app then displays the table. So far so good.

However, this is only half the story. What the user actually wants to do is to create People and Places. The CSV information is just the raw data. Some of the columns will contain names of people (first name, last name) while some columns may contain addresses. The columns just contain random words of course, unless you have some real data to play with. But you get the idea. The data needs merging with the other tables to create workable records.

Sketch below outlines the design for merging CSV import tables.

We will add a row of table headers to the import table viewing page. Initially they will contain empty ‘select’ buttons. The user starts by selecting which table to import into. After choosing, the page will automatically update the column headers with the appropriate choices for input field mapping. If the chosen table is Person, column choices switch to none, first name or last name. For Places, only two choices: none or address. The user can then assign columns to fields, clicking Merge when ready.

We therefore want the import-table show page to become dynamic. When the user selects the table, the selectors for column field must change appropriately. To make this happen, we take a page out of Ryan Bates’ book. Since the database records are static, we can set up a JavaScript array of mappings between tables and columns. When the user selects the table to import into, person or place, JavaScript within the page can respond by updating the column selectors.

Converting to jQuery with UJS

We plan to use jQuery rather than Prototype as our JavaScript framework. By default however, Rails has set up our application to include Prototype and script.aculo.us. Remove the following list of files.

public/javascripts/controls.js
public/javascripts/dragdrop.js
public/javascripts/effects.js
public/javascripts/prototype.js

Replace the rails.js UJS script found in public/javascripts with the jQuery-based version found at GitHub. UJS stands for Unobtrusive JavaScript, new in Rails 3. By making use of HTML 5 enhancements, the UJS feature avoids littering HTML pages with embedded JavaScript. Download and install jQuery 1.4.1 in the same place, at public/javascripts.

There is one more thing to do before jQuery becomes the new default JavaScript framework for our application: create a new ‘initialiser’ to override Prototype and script.aculo.us as the default framework. Create a new Ruby script, config/initializers/jquery.rb; exact name is not critical. This will run when the Rails server initialises. Add the following code.

module ActionView::Helpers::AssetTagHelper
  remove_const :JAVASCRIPT_DEFAULT_SOURCES
  JAVASCRIPT_DEFAULT_SOURCES = %w(jquery-1.4.1.min rails)
  reset_javascript_include_default
end

This will override the default JavaScript sources with jQuery 1.4.1 and its UJS helper. jQuery ready!

Table headers, table selection, Merge button

Next, we need to add the import table headers, a table selection and a Merge button to the current import-table viewing code. Add the following. I’ve commented out where the table contents go, just to make it a little clearer.

<% form_for :merge do |f| -%>
  <table border="1" cellspacing="1" cellpadding="1">
    <tr>
      <% 0.upto(@column_index_max) do |column_index| %>
        <th>
          <%= f.select(column_index, []) %>
        </th>
      <% end %>
    </tr>
    # same as before goes here
  </table>
  <label><%= 'Select a table for merging:' %></label>
  <%= f.select(:table, @tables.collect { |t| [t, t] }, { :include_blank => true }, { :onchange => "mergeTableSelected();" }) %>
  <%= f.submit "Merge", :disable_with => 'Merging...' %>
<% end -%>

Some things to note about this code:

It encloses the entire table within a form. That means that the form will send a POST request with the form parameters as fields. Within the receiving controller method, expression params[:merge] will answer a Hash of values: one for each of the columns, and one for the selected table.
Initially, the table headers are null selections. It will produce select tags with no options. We will fill this with content using JavaScript when the user chooses which table (people or places) for merging into.
Whenever the user changes the table selection, the browser will invoke a JavaScript function named mergeTableSelected.

The code accesses an instance variable @tables. This is not yet defined. But we will now define it in the import tables controller as follows.

@tables = ActiveRecord::Base.connection.tables.select { |t| t != 'schema_migrations' }

This code performs a little bit of Active Record introspection. It asks Rails for the tables associated with its database connection. The answer is an array of table names, strings. The Array select method de-selects the schema_migrations table, an internal table used by Rails. We want our app to ignore this one, in future possibly others too.

Merge table selected

Next, add the following JavaScript code to the show.html.erb amoung the import tables views. It injects a blob of JavaScript into the show page.

<%= javascript_tag do -%>
  var columns = new Array();
  <% for table in @tables -%>
    <% for column in ActiveRecord::Base.const_get(ActiveRecord::Base.class_name(table)).columns -%>
      columns.push(new Array('<%= table %>', '<%= column.name %>'));
    <% end -%>
  <% end -%>
  function mergeTableSelected() {
    var o = $('select#merge_table option:selected');
    $('th select option').remove();
    $('th select').each(function(i, select) {
      select.options.add(new Option());
      $.each(columns, function(j, column) {
        if (o.text() == column[0]) {
          select.options.add(new Option(column[1]));
        }
      });
    });
  }
<% end -%>

When Rails renders the page, it iterates the introspected @tables, building an array of arrays called columns. Each element contains a table name and a column name. When the user picks a table, the mergeTableSelected() function accesses the selection, removes all existing options from the table header select tags and fills them with new options based on the column names matching the newly selected table name. Does that make sense? The JavaScript relies on jQuery of course. See jQuery documentation.

The merge

Finally, the actual merge operation. When the user clicks Merge, their browser posts the parameters to import_tables/:id where :id stands for the unique identifier of the relevant import table. We need to match this to a method on the import tables controller. Add the following route to routes.rb.

  match 'import_tables/:id' => 'import_tables#merge'

And add the merge method implementation.

  def merge
    # Uncomment the following line if you want to debug this method. However,
    # do not forget to uncomment the gem 'ruby-debug' in Gemfile also; this
    # assumes you have the ruby-debug gem installed too, and do the usual
    # thing if not, i.e. sudo gem install ruby-debug. Calling "debugger"
    # inserts a break point when the server hits this point in the
    # application. From that point you can interrogate the Rails state, single
    # step over or into message-sends and so forth. Very useful for debugging!
    #
    # debugger
    
    import_table = ImportTable.find(params[:id])
    import_cells = import_table.import_cells
    row_index_max = import_cells.map { |cell| cell.row_index }.max
    column_index_max = import_cells.map { |cell| cell.column_index }.max
    
    # Pull the merge parameters from the POST request. The form sets up the
    # column mappings and merge table choice. Then use a little bit of
    # ActiveRecord introspection to derive the table's class name followed by
    # the class object. We will use this latter object to instantiate the
    # merged records.
    merge = params[:merge]
    merge_table = merge[:table]
    klass = ActiveRecord::Base.const_get(ActiveRecord::Base.class_name(merge_table))
    
    # Determine which columns have been mapped. Ignore the rest. Intersect the
    # requested column names with actual column names. Perhaps we should abort
    # and display some error message if the intersection proves empty because
    # the user did not select any columns.
    inverted_merge = merge.invert
    column_names = inverted_merge.keys & klass.column_names
    
    # Finally, create new instances, one per row. Iterate the rows, then for
    # each row, iterate the mapped columns. Select the matching cell and
    # update the record's corresponding column. Redirect to the merged table
    # when done.
    0.upto(row_index_max) do |row_index|
      row = import_cells.select { |cell| cell.row_index == row_index }
      instance = klass.new
      column_names.each do |column_name|
        column_index = inverted_merge[column_name].to_i
        contents = row.select { |cell| cell.column_index == column_index }[0].contents
        instance[column_name] = contents
      end
      instance.save
    end
    eval "redirect_to #{merge_table}_path"
  end

Hopefully, the in-line comments will help to elucidate what this method is doing and why.

Conclusions

What have we learned? Importing CSV is relatively straightforward using Rails 3 and jQuery. But the exercise has raised some outstanding questions and pointed forward to some interesting future directions.

Outstanding questions

Uploading of CSV files does not deal with encoding issues. For the English language that might not be an enormous problem. But supporting other languages might begin to throw up bigger and bigger issues depending on how exotic the language needing conversion to Unicode (the internal encoding). There may also be number-formatting issues, e.g. some languages use commas for full stops (periods) within numbers.
Column typing has not been considered. Some CSV columns may contain strings, others numbers. How these convert from CSV to real data within the application will depend on the conversion path between uploading and merging of data.

Future directions

Some CSV files may have headings in the first row. Import merging might be friendlier if the user could ignore the first row by checking a box entitled, “Ignore first row.”
The software does not yet support any form of row selection and de-selection. It merges all rows. A real application would need to trim off the fat: remove duplicates, sort by columns, combine rows by dragging and dropping cell contents in-between rows, etc.

Retaining C++ Objects

Tue, 09 Mar 2010 22:31:53 -0800

C++ and Objective-C do not always mix too well, although the two languages are fully compatible.

With respect to compilation phases, alternating between C, C++, Objective-C and Objective-C++ presents no problem. You just write your software accordingly, freely mixing C, Objective-C, C++ and Objective-C++. Just make sure that the compiler knows what each translation unit contains, normally by source file extension: c for C; cc or cpp or cxx for C++; m for Objective-C; and mm for Objective-C++.

That being specified correctly, your sources compile and link without issues. Only when you start running the code side-by-side however, you might find that memory management can become something of a mishmash. Basic C++ fundamentally has no memory management; I mean nothing automatic. Asking for memory when you want it, freeing memory when you’re done with it: that is all bare C++ gives you. So you typically just roll your own, or find some more-advanced alternative such as Boehm’s garbage collector.

Wouldn’t it be nice however, if you could just simply retain, release and auto-release your C++ objects, just like you do with NSObjects? It would be very nice.

This article outlines a little “trick” for retaining C++ objects on Mac and iPhone. It takes advantage of C++ inheritance; and is simple, so no dependence on internals of CFTypeRef or NSObject. The memory cost is low; just one additional NSObject-derived object per retained C++ object. Final feature, implementation separates from interface by design. Hence you can include retaining within bodies of source without the things being retained having any knowledge of Objective-C. Translation units remain plain old C++, no additional dependencies. The implementation exists apart.

MIT-licensed code provided in full.

Who Cares?

The audience for this topic are developers wanting to integrate C++ libraries with Objective-C on iPhone or Mac under the retain-release management paradigm. C++ objects become subject to standard Apple memory management, even without classes being aware of it.

Start With a Test

Always best: start with a test.

Basic failing test

Allocate a C++ object. Then autorelease it. Place this within an autorelease pool context. Initially test-failing gives a leak. Test-success eliminates the leak. This is the test.

In code form, it looks like this.

// Instantiate an instance of TestClass. Retain then auto-release it. Within
// the context of an auto-release pool, will it delete?
pool = [[NSAutoreleasePool alloc] init];
(new TestClass)->retain().autorelease();
[pool release];

TestClass has the following basic definition.

class TestClass : public c_plus_plus_retained
{
public:
    TestClass()
    {
        log("hello");
    }
    ~TestClass()
    {
        log("goodbye");
    }
private:
    void log(const char *message)
    {
        // print message
    }
};

While c_plus_plus_retained has the following declaration.

struct c_plus_plus_retained
{
    // constructors & virtual destructor
    c_plus_plus_retained() : retainer_object(0) {}
    c_plus_plus_retained(const c_plus_plus_retained &copy) : retainer_object(0) {}
    virtual ~c_plus_plus_retained() {}

    // mirroring NSObject semantics
    c_plus_plus_retained &retain();
    void release();
    c_plus_plus_retained &autorelease();

    // accessor
    void *retainer();

private:
    void *retainer_object;
};

The above code declares a pure C++ public interface; notice there are no includes hence no other dependencies. Any general C++ translation unit can see this interface declaration without baulking. It contains nothing about Objective-C or Objective-C++. Such dependencies appear at link- and run-time, not at compile-time.

This is by design. You can include this header in any C++ source.

Successful Implementation

The implementation allocates an NSObject-derived retainer which deletes the retained C++ instance during its -dealloc method.

c_plus_plus_retained &c_plus_plus_retained::retain()
{
    if (retainer_object)
    {
        [(CPlusPlusRetainer *)retainer_object retain];
    }
    else
    {
        retainer_object = [[CPlusPlusRetainer alloc] initWithRetained:this];
        // Hereafter you must never delete this object. Doing so will break the
        // implicit design contract. Asking to retain actually places the
        // retained object within the scope of retain-release memory
        // management. Only releasing hereafter will release the retainer and
        // delete the object. Henceforward, delete becomes release. Calling
        // delete will double-delete this object.
    }
    return *this;
}

void c_plus_plus_retained::release()
{
    if (retainer_object)
    {
        [(CPlusPlusRetainer *)retainer_object release];
    }
}

c_plus_plus_retained &c_plus_plus_retained::autorelease()
{
    if (retainer_object)
    {
        [(CPlusPlusRetainer *)retainer_object autorelease];
    }
    return *this;
}

void *c_plus_plus_retained::retainer()
{
    if (retainer_object == 0)
    {
        retain().autorelease();
    }
    return retainer_object;
}

Semantically, the c_plus_plus_retained class initially has no retain count. It remains initially invisible respecting retain-release memory management. This changes when you retain it for the first time. At that point, you no longer delete the object. You must release it, doing so manually or by auto-releasing.

Beware the C++ Copy Constructor!

By default the compiler supplies a bitwise copy that copies everything verbatim. If you create a copy however, the duplicate has no retainer. If you want duplicates to retain then invoke retain(). If you want duplicates to auto-release invoke retained->retain().autorelease() assuming ”retained” is a pointer to c_plus_plus_retained or its sub-class.

Grab a Copy

The repositories live on GitHub at Library of useful C++ classes for iPhone and Mac and iPhone app project encapsulating tests for c_plus_plus_retained. Drop the iPhone app project into the Library folder then open, build and run. You’ll see something like this (memory addresses will vary, of course).

Gloss Caustic Shader for iPhone

Thu, 04 Mar 2010 20:47:00 -0800

Ole Begemann has done some very useful work, porting the gloss-caustic shader to iPhone OS. This work has been long overdue. Many thanks Ole!

I’ve merged Ole’s changes into my Git repo, albeit with a few modifications. This article outlines the changes to the changes.

What has changed?

I’ll not dig too much into Ole’s enhancements. Ole does that on his page. In summary:

Shader sources compile for OS X and iPhone OS.
Has two sample projects, one for OS X and the other for iPhone.

Why does it need porting?

The fundamental difference between iPhone and OS X for the gloss-caustic shader concerns its use of NSColor. Simply put, iPhone does not have such a class; instead, it has UIColor with limited support for colour spaces and no support for RGB-to-HSB conversion.

Strangely, you can make the reverse conversion, from HSB to RGB by invoking class method

+colorWithHue:saturation:brightness:alpha:

and UIKit answers with a colour with triple RGB components whence you can access the red, green, blue and alpha component values. But iPhone has no instance method

-getHue:saturation:brightness:alpha:

as does OS X.

The basic approach

The simple approach Ole takes is to surround the difference between OS X and iPhone with #if TARGET_OS_IPHONE, #else, #endif blocks. Xcode defines TARGET_OS_IPHONE only when compiling for iPhone.

Under normal circumstances, I would avoid this approach since it makes code difficult to read and maintenance harder in the long run. Nevertheless, the pre-processor conditional blocks turns out relatively small in number and in extent. Each one in general surrounds just one line of code: the first condition presents the iPhone’s UIColor-based alternative, while the second condition presents OS X’s NSColor-based alternative.

Maintaining semantics

Ole has included a number of useful functions for colour conversion, based on work by Erica Sadun and some standard RGB-to-HSB conversion code. I’ve repackaged these as extensions on UIColor with semantics mirroring that of NSColor.

Interface for UIColor(RRUIKit) as follows.

// Fill in some of the "missing" UIColor methods, making UIColor more compatible
// with NSColor. This includes access to individual colour components and HSV
// conversion. Use similar method signatures and semantics as far as possible.

@interface UIColor(RRUIKit)

- (UIColor *)colorUsingColorSpaceModel:(CGColorSpaceModel)model;

- (CGColorSpaceRef)colorSpace;
- (CGColorSpaceModel)colorSpaceModel;

- (NSUInteger)numberOfComponents;
- (void)getComponents:(CGFloat *)components;

- (CGFloat)redComponent;
- (CGFloat)greenComponent;
- (CGFloat)blueComponent;

- (CGFloat)alphaComponent;

//---------------------------------------------------- Hue-Saturation-Brightness

- (void)getHue:(CGFloat *)hue saturation:(CGFloat *)saturation brightness:(CGFloat *)brightness alpha:(CGFloat *)alpha;
    // Answers the components of RGB colours as hue, saturation and
    // brightness. Adds the missing UIColor method for converting RGB to
    // HSB. This method assumes that self (an instance of UIColor) has RGB
    // colour model.

@end

Serialisation

Ole implements a set of extensions on RRGlossCausticShader for saving shader setting to a dictionary. I’ve enhanced this functionality by implementing the NSCoding protocol for shader and matcher classes.

Coding adds a level of extra flexibility. The coding protocol lets you serialise trees of objects (the shader and its enclosed matcher) by archiving to any data stream.

Key-value coding

I’ve also simplified the iPhone sample’s view controller. iPhone OS does not include bindings. Hence implementing the update cycle within the controller can become rather tedious when there are many control elements, such as with gloss-caustic shaders.

My enhancement on Ole’s enhencement employs key-value coding, i.e. accessing properties via -valueForKeyPath: and -setValue:forKeyPath:. It makes things a little less long-winded.

Your feedback

As always, very welcome. Thanks again Ole. Your improvements are much appreciated. You can find Ole’s work at GitHub. Mine also at GitHub.

Gloss-Caustic Shading

Tue, 09 Dec 2008 19:25:45 -0800

Matt Gallagher recently wrote a very useful article about drawing gloss gradients using Core Graphics. In his article, Matt describes how to reproduce the oft-seen glossy gradient effect. Thanks Matt! It’s a nice article. “Cocoa with Love” lovingly provides the working source code. This little article aims to complement Matt’s work.

I’ve also re-factored the software and packaged the result within an Objective-C class called RRGlossCausticShader. This packaging automatically adds support for key-value coding and observing. Bindings then let you easily wrap the class within a little application able to adjust the many parameters interactively.

Show me the money

Before going any further, you might first want to see the results. I know I would! Download the project to play with the shader. Compile and run the application using Xcode. The main window appears as follows. Inspector panels also appear from where you can interactively adjust the many shading parameters.

Gloss above, caustic below. Those are the two halves of the shading. Gloss appears in the top half, caustic in the bottom half. Within each half, an exponential function blends a given non-caustic base colour with whiteness above and with matching caustic colour below. The main window displays these basic elements of the shading process: the shading itself (left), non-caustic base colour (top right) and coefficient of the exponent function (bottom right).

Re-factoring

My version of Matt’s work includes some re-factoring. Rather than just the one functional interface, the version that you’ll find within the project involves three modular utility classes. - RRGlossCausticShader encapsulates shading - RRCausticColorMatcher finds caustic matches for given colours - RRExponentialFunction wraps the exponential maths - RRLuminanceFromRGBComponents returns luminance from RGB triples

This re-factoring has some benefits. First, it promotes re-use. Other requirements may call for RGB-to-luminance conversion, or a generic exponential function for example; or you might want to sub-class the caustic colour matcher and add some customisation. Separating out the individual sub-requirements adds some extra scope for growth. I think it also makes the source code somewhat easier to follow and therefore comprehend. But that might just be my own personal opinion. Your mileage may vary, as they say!

Anyway, the purpose of this article is to outline some of the differences in implementation and the underlying thinking. Comments welcome, of course.

Twinkle, twinkle

The gloss effect on the upper half of the gradient derives from the base colour’s luminosity. So obtaining luminance from a given Red, Green and Blue is part of the glossing requirement.

As you can see below, I replace Matt’s conversion with the one described at OpenGL. The difference is small, almost non-existent. Only, the comment that “Haeberli notes that [the YIQ colour conversion values used by Matt] are incorrect in a linear RGB colour space” convinced me to make the small change. Just call me fussy.

CGFloat RRLuminanceFromRGBComponents(const CGFloat *rgb)
{
    // 0.3086 + 0.6094 + 0.0820 = 1.0
    return 0.3086f*rgb[0] + 0.6094f*rgb[1] + 0.0820f*rgb[2];
}

Notice a few other things. I’m using CGFloat type for floats because this corresponds exactly with Apple’s usage for colour components. Can we always assume that CGFloat equals float? No. In fact, CGFloat is sometimes double! This happens when 64-bit compiling for instance. Argument const-correctness is another feature.

Caustic nature

The lower half of the gradient blends the base colour into shades of yellow (yellow is the default caustic hue). The lower end of the gradient starts to look more-and-more tinted towards the yellow. However, the precise colour depends upon the non-caustic base colour. For blue colours, for example, the default caustic switches to magenta. Blue looks better fading toward magenta, rather than yellow. In all cases though, the amount of caustic to non-caustic blending varies according to the cosine of the distance between the caustic and non-caustic hues.

So rather than having a function handle the caustic colour matching and implementing the adjustable parameters as manifest constants, the new version implements a “colour matching” class. Interface listed below. The class implements adjustable parameters as read-write class properties with all the necessary setters and getters. This approach buys some advantages. It automatically adds support for key-value coding and observing. In practice, it means that user-interface controls can bind to these properties using Cocoa Bindings.

@interface RRCausticColorMatcher : NSObject
{
    CGFloat causticHue;
        // Yellow by default.
    CGFloat graySaturationThreshold;
        // Saturation level at which colours appear grey. Below this level,
        // matcher response snaps to pure caustic.
    CGFloat causticSaturationForGrays;
        // Defines the caustic saturation for grey colours. Grey colours fall
        // below the grey saturation threshold. When saturation drops too low,
        // everything looks grey.
    CGFloat redHueThreshold;
        // Colours at this threshold and above match to default caustics rather
        // than default magenta for blues.
    CGFloat blueHueThreshold;
        // Triggers a switch to magenta caustics. Hues at blue and beyond
        // display magenta-modulated caustics by default.
    CGFloat blueCausticHue;
        // Magenta by default. Magenta caustics for blue colours.
    CGFloat causticFractionDomainFactor;
        // Expands or contracts the caustic fraction's domain. With factor equal
        // to 1, non-caustic and caustic hues blend according to the cosine of
        // their difference. Smaller the difference, greater the amount of
        // caustic hue. Defaults to 1.4 meaning that the point of absolutely no
        // caustic blending occurs at 1/1.4 difference from caustic hue. Try
        // plotting cos(x*pi*1.4) in the -1,1 interval.
    CGFloat causticFractionRangeFactor;
        // Scales the caustic fraction which without a factor outputs a blending
        // fraction between 0 and 1 in favour of the caustic blend. Defaults to
        // 0.6 which scales down the amount of caustic hue-and-brightness by
        // that amount.
}

- (NSColor *)matchForColor:(NSColor *)aColor;
    // Matches the given colour. Answers a matching caustic colour. The result
    // shifts hue and brightness towards yellow. Saturation remains unchanged.
- (void)matchForHSB:(const CGFloat *)hsb caustic:(CGFloat *)outHSB;
    // Does the work.

@property(assign) CGFloat causticHue;
@property(assign) CGFloat graySaturationThreshold;
@property(assign) CGFloat causticSaturationForGrays;
@property(assign) CGFloat redHueThreshold;
@property(assign) CGFloat blueHueThreshold;
@property(assign) CGFloat blueCausticHue;
@property(assign) CGFloat causticFractionDomainFactor;
@property(assign) CGFloat causticFractionRangeFactor;

@end

Exponentially

Shading employs an exponential function. As the gradient progresses from 0 to 0.5, gloss white blending increases exponentially in the 0,1 interval. Progressing through the bottom half of the gradient from 0.5 to 1, caustic blending increases exponentially. The implementation re-factors the exponential function. Interface listing below. It becomes a C-style object class and thereby minimises its dependencies: just plain C types and standard library math.h and hence very suitable for repeated invocations from the bowels of a shading function.

// Encapsulates an optimising generic Exponential Function where
//      y=1-(exp(x*-c)-exp(-c))/(1-exp(-c))
// and where 0<c is a general coefficient describing the exponential
// curvature. The function's input domain lies within the 0..1 interval, its
// output range likewise. The implementation optimises by pre-computing those
// constant terms depending only on the coefficient whenever the coefficient
// changes. Repeated evaluation takes less computing time thereafter.
struct RRExponentialFunction
{
    float coefficient;
    float exponentOfMinusCoefficient;
    float oneOverOneMinusExponentOfMinusCoefficient;
};

typedef struct RRExponentialFunction RRExponentialFunction;

void RRExponentialFunctionSetCoefficient(RRExponentialFunction *f, float c);
float RRExponentialFunctionEvaluate(RRExponentialFunction *f, float x);

Shading

And finally, the “gloss-caustic shader” class brings all the disparate pieces together. The design aims for reusability. Idea is that you instantiate a shader, set up the parameters such as non-caustic colour or any other necessary adjustments to defaults, then re-use it over-and-over whenever you need to draw the shading.

@interface RRGlossCausticShader : NSObject
{
    struct RRGlossCausticShaderInfo *info;
    RRCausticColorMatcher *matcher;
}

- (void)drawShadingFromPoint:(NSPoint)startingPoint toPoint:(NSPoint)endingPoint inContext:(CGContextRef)aContext;

- (void)update;
    // Send -update after changing one or more parameters. Setters do not
    // automatically update the shader. This is by design. It applies to the
    // caustic colour matcher too. Change anything? Send an update. Otherwise,
    // if the setters automatically update, multiple changes trigger unnecessary
    // multiple updates. It's a small optimisation.
    // Updating follows the dependency chain. Caustic colour depends on
    // non-caustic colour along with all the caustic colour matcher's tuneable
    // configuration settings. Updating also re-computes the gloss. Gloss
    // derives from non-caustic colour luminance, among other things.

//---------------------------------------------------------------------- setters

- (void)setExponentialCoefficient:(float)c;
- (void)setNoncausticColor:(NSColor *)aColor;
    // Converts aColor to device RGB colour space. The resulting colour
    // components become the new non-caustic colour. This setter, like all
    // others, does not automatically readjust the dependencies. Invoke -update
    // after adjusting one or more settings.
- (void)setGlossReflectionPower:(CGFloat)powerLevel;
    // Assigns a new power level to the gloss reflection.
- (void)setGlossStartingWhite:(CGFloat)whiteLevel;
    // White levels range between 0 and 1 inclusive. Gloss starting white levels
    // typically have higher values compared to ending white level.
- (void)setGlossEndingWhite:(CGFloat)whiteLevel;

//---------------------------------------------------------------------- getters

- (float)exponentialCoefficient;
- (NSColor *)noncausticColor;
    // Returns the non-caustic colour.
- (CGFloat)glossReflectionPower;
- (CGFloat)glossStartingWhite;
- (CGFloat)glossEndingWhite;

// Key-value coding automatically gives access to colour matching for caustic
// colours. Special note though, changing caustic matcher thresholds does not
// (repeat not) automatically adjust the shader's caustic colour. You need to
// update the shader when ready.
// Currently, the matcher property offers read-only access. You cannot set the
// matcher! However, perhaps future versions will allow setting in order to
// override the default caustic matching behaviour. Developers might want to
// customise the colour matching algorithmically as well as by tweaking
// parameters.
@property(readonly) RRCausticColorMatcher *matcher;

@end

The sample project explains how to use the shader. I hope it’s clear enough. The sources have an MIT license. I haven’t tried it out on iPhone or pre-Leopard Mac, so can’t make any comments about portability except to say that porting should be fairly straightforward. But no doubt you’ve heard that before!

Outline view, tree controller and itemForPersistentObject

Wed, 10 Sep 2008 07:42:56 -0700

This is the scenario: your user interface comprises an outline view and a hierarchical data model. You want the outline view to display the hierarchy and remember the expansion state automatically in preferences. Hence when the user re-runs the application, the items that were expanded are still expanded, and vice versa: what was collapsed remains so. Outline view, tree controller, hierarchical model, bindings. That’s the recipe.

According to some, storing expansion state of an outline view when used with a tree controller is not just difficult: it’s impossible! But is it? No, is the simple answer. It’s actually quite easy. In this article I introduce a new helper class called RROutlineViewExpandedItemsAutosaver! Original aren’t I? It does not involve sub-classing or access to private methods. The solution presented uses only documented interfaces and only requires a small stateless class instance for handling all outline view auto-saving technicalities. It does assume Core Data use for modelling. But you can easily adapt the technique for other data-model implementations.

It’s impossible?

At first it seems so. For a start, if you read the documentation for setAutosaveExpandedItems: you can see that you also need to implement two data-source methods: outlineView:itemForPersistentObject: and outlineView:persistentObjectForItem:. Goes without saying, you also need a data source implementing those methods. Otherwise it does not work. In fact, it’s not exactly clear what these two necessary methods are supposed to do. The document talks about translating to and from an “archived object.” What does that mean exactly?

You can find a number of posts at CocoaBuilder on this issue. Keith Blount started a thread in 2005; Nick Briggs asked about it again in 2006. But no answers forthcoming! Some discussion exists and some solutions have been posted at CocoaDev (OutlineViewCoreDataBindings and NSOutlineViewStateSavingWithNSTreeController). But there is nothing definitive. Plus, the solution supplied relies on private methods and sub-classing. Messy!

Start with a test

In the spirit of Agile, let’s start with a test that fails.

Apple’s Developer Connection provides sample code. Their AbstractTree demonstrates Core Data, use of bindings along with an NSTreeController. Download the sample here as a ZIP file. It contains two subdirectories: AbstractTree and Tutorial. The tutorial is pretty good. It should help you get up-to-speed if you need some familiarity with the basic technology. Build and run the project living under the AbstractTree folder. It presents you with a window comprising an empty outline view and two buttons.

Click Add a few times. You get a series of nested nodes named “untitled node”. Quit the application, re-run and you find that, although data saved, the expansion state of the outline view resets. You have to manually expand each item. Of course, you can Option-click the disclosure triangles to auto-expand an item and all its sub-items. But that’s not really what we want. We want the expand or collapse state of each element displayed in the outline view to persist! That’s the goal.

So far so good. We have a test case that fails. Just what we wanted.

Outline view’s auto-save name

Open the sample’s MainMenu.xib. Select the NSOutlineView. Set its Autosave name to (say) Nodes. Exact value does not matter. This is just a little piece of text for constructing the preferences key. The application’s defaults will contain an array containing the expanded items. The key will become “NSOutlineView Items Nodes” where Nodes corresponds the the specified Autosave name.

Leave the Autosave Expanded Items check box unchecked! Strange but important. If you check this box, the framework resolves the expand-collapse question straightaway. It needs to wait until bindings have been fully established and the data store has been added. The application will instead enable this option programmatically when the application finishes launching. Not before.

Add the new class

Add the interface header, RROutlineViewExpandedItemsAutosaver.h, to the project. Contents as follows. Download header and module sources here.

#import <AppKit/AppKit.h>

//------------------------------------------------------------------------------
// RROutlineViewExpandedItemsAutosaver
//------------------------------------------------------------------------------
// This class acts as a "dummy" outline-view data source. Dummy because it does
// not source any data. But you wire it up as though it did. In reality, its
// sole purpose is to enable automatic saving of an outline view's expanded
// items. It sounds counter-intuitive, but you can apply bindings through a tree
// controller and at the very same time hook up a data source (such as one of
// these) in order to provide the necessary persistence translations.
//
// Connect an outline view to an instance of this class. Note, you can connect
// multiple outlines views to the same instance! The instance carries no
// state. Message interactions with outline views pass the outline view
// instance. So no state needed.
//
// The implementation makes a number of assumptions. It assumes you connect
// using bindings to a Core Data model. Hence for every item, it sends [[item
// representedObject] objectID] which assumes that the represented object
// responds to -objectID. Core Data's managed objects respond with a unique
// object identifier.

@interface RROutlineViewExpandedItemsAutosaver : NSObject
@end

Add the source module RROutlineViewExpandedItemsAutosaver.m. Contents follow.

#import "RROutlineViewExpandedItemsAutosaver.h"

@implementation RROutlineViewExpandedItemsAutosaver

//------------------------------------------------------------------------------
#pragma mark Outline View Data Source
//------------------------------------------------------------------------------

// Although linked to the core-data context via bindings and a tree controller,
// the outline view also has an instance of this object connected to the outline
// view's data source.

- (id)outlineView:(NSOutlineView *)outlineView itemForPersistentObject:(id)object
{
    // Iterate all the items. This is not straightforward because the outline
    // view items are nested. So you cannot just iterate the rows. Rows
    // correspond to root nodes only. The outline view interface does not
    // provide any means to query the hidden children within each collapsed row
    // either. However, the root nodes do respond to -childNodes. That makes it
    // possible to walk the tree.
    NSMutableArray *items = [NSMutableArray array];
    NSInteger i, rows = [outlineView numberOfRows];
    for (i = 0; i < rows; i++)
    {
        [items addObject:[outlineView itemAtRow:i]];
    }
    for (i = 0; i < [items count] && ![object isEqualToString:[[[[[items objectAtIndex:i] representedObject] objectID] URIRepresentation] absoluteString]]; i++)
    {
        [items addObjectsFromArray:[[items objectAtIndex:i] childNodes]];
    }
    return i < [items count] ? [items objectAtIndex:i] : nil;
}

- (id)outlineView:(NSOutlineView *)outlineView persistentObjectForItem:(id)item
{
    // "Persistent object" means a unique representation of the item's object,
    // representing the objects identity, not its state. Outline view writes
    // this to user defaults as soon as the item expands. That's when it asks
    // for the persistent object, sending -outlineView:persistentObjectForItem:
    // and execution arrives here. A minor problem arises when adding new
    // items. The new item represents a new unsaved managed object. The managed
    // object only has a temporary object identifier. It will receive a
    // permanent one when saved. So, if the objectID answers a temporary one,
    // ask the context to save and re-request the objectID. The second request
    // gives a permanent identifier, assuming saving succeeds. Don't worry about
    // committing unsaved edits at this point.
    NSManagedObject *object = [item representedObject];
    NSManagedObjectID *objectID = [object objectID];
    if ([objectID isTemporaryID])
    {
        if (![[object managedObjectContext] save:NULL])
        {
            return nil;
        }
        objectID = [object objectID];
    }
    return [[objectID URIRepresentation] absoluteString];
}

@end

Add this header and module to the project, under the Classes group. An instance of this class will become the new data source for the outline view.

Add the new data-source

Add a new instance of RROutlineViewExpandedItemsAutosaver to the nib and connect the new instance to the NSOutlineView as the outline view’s dataSource. Use Interface Builder. Drag an Object from the palette to the nib. Change its class in the Identity Inspector panel (Command-6). Then connect the NSOutlineView to it as the new data source. It replaces the pre-existing connection between outline view and app delegate.

After this, the nib is ready.

App delegate

Apple have used the standard Core Data application delegate. It builds a managed object model, persistent store co-ordinator and managed object context. The classic Core Data stack! Our changes need to add a message-send for enabling auto-save expanded items to the outline view. But it’s not the only change required.

There is a subtle problem. It concerns bindings and Core Data. The important requirement is that when the outline view tries to restore auto-saved expanded items, the outline view has its items already loaded. Otherwise how can it translate “persistent objects” to items. Persistent objects, in this context, refers to keys used in the application defaults to identify the set of expanded items; everything not expanded defaults to collapsed.

Moving where the application adds its persistent store

When the application loads, it automatically loads MainMenu nib. All the object instances therein become instantiated. Indirectly, the managed object context and its other Core Data stack components come to life at this time. Therein lies the subtle problem. The Cocoa framework will not immediately load the Core Data objects if the context already has its store when nib-loading establishes the bindings. That’s a mouthful. Basically, the order must go: 1. Establish bindings. Do this wholesale. 2. Add persistent store. At this point, the outline view grabs its contents through bindings to the tree controller. 3. Enable auto-save expanded items.

Simplify the -persistentStoreCoordinator implementation to:

- (NSPersistentStoreCoordinator *)persistentStoreCoordinator {
    if (persistentStoreCoordinator == nil) {
        persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc]
            initWithManagedObjectModel:[self managedObjectModel]];
    }
    return persistentStoreCoordinator;
}

And second, add a new method:

- (void)applicationDidFinishLaunching:(NSNotification *)notification
{
    NSFileManager *fileManager;
    NSString *applicationSupportFolder = nil;
    NSURL *url;
    NSError *error;

    fileManager = [NSFileManager defaultManager];
    applicationSupportFolder = [self applicationSupportFolder];
    if (![fileManager fileExistsAtPath:applicationSupportFolder isDirectory:NULL]) {
        [fileManager createDirectoryAtPath:applicationSupportFolder attributes:nil];
    }

    url = [NSURL fileURLWithPath:[applicationSupportFolder
        stringByAppendingPathComponent:@"AbstractTree.xml"]];
    if (![[self persistentStoreCoordinator]
        addPersistentStoreWithType:NSXMLStoreType
                     configuration:nil
                               URL:url
                           options:nil
                             error:&error]) {
        [[NSApplication sharedApplication] presentError:error];
    }    
    // Watch out for this! Make sure that the data is available before enabling
    // auto-save for expanded items. You can switch it on within the nib. Problem
    // though is that the data store must be available at nib awaking time!
    // Otherwise, how can the auto-saving of expanded items compare against
    // existing items in order to determine whether or not they should be restored
    // as expanded or not.
    //
    // There's another caveat. You need to let the bindings make the necessary
    // connections first, before connecting the Core Data context and persistent
    // store coordinator to the store. In other words, the store must be added
    // last. Here is a good place. Application-did-finish-launching occurs after
    // all nib awaking methods, after bindings have been established. Hence, when
    // the store gets added, the outline view immediately sees the contents
    // through its bindings. Otherwise the outline-view expanded items auto-saver
    // cannot see the items at all.
    [outlineView setAutosaveExpandedItems:YES];
}

That’s it!

Successful test

Build and go. This time, when you expand nodes, quit then re-run, the sample correctly expands the previously-expanded items. Great. You can view the sample application’s user defaults by typing, in Terminal:

defaults read com.apple.dts.AbstractTreeApp

It will list the saved expanded items array, resembling something along these lines:

{
    "NSOutlineView Items Nodes" =     (
        "x-coredata://68A35129-6EA4-45E7-B3FB-F1C15FDC02D9/Node/p104",
        "x-coredata://68A35129-6EA4-45E7-B3FB-F1C15FDC02D9/Node/p102"
    );
}

Download the complete sample project here.