Pensieri di un lunatico minore

Times they are a’changing

2009-10-16T13:42:33Z

I’ve decided to close down this blog … sorta. Something else will replace it, and the mindless posts that exist here will be maintained. A change of scenery is required.

Emacs macro trick

2009-09-07T16:28:52Z

I don’t know when this was added, but as someone who has used Emacs pretty constantly since 1985, I can’t believe I never knew this. Most people know you can record a simple macro with C-x ( to start and C-x ) to end. You can then replay it back with C-x e. Nothing new there, right? Well, obviously you can combine the C-u prefix, which will repeat it. So C-u 10 C-x e will execute the macro 10 times. If you say C-u 0 before, then it’ll run until it has an error (usually the end of the file, or something).

But here’s the trick. If you type C-u C-x e, it will run the macro once, and then allow you simply to press e to keep running it. This is very handy when you’re trying to go through a list in the middle of a document.

Funny how I never noticed.

Python AMQP logging

2009-09-02T03:04:33Z

For a project I’ve been working on, I needed to distribute logging information to a bunch of destinations. As part of this, I decided to write a Python logging module piece that will gateway all the log records to an AMQP exchange. I’ve posted it on PyPI as py-amqp-logging. Right now it’s very alpha, but it does work. It’s only been tested with RabbitMQ at this point.

You can track development on bitbucket.

The Trouble with Frameworks

2009-08-20T12:51:48Z

Everyone loves a “good framework”, even if they do tend to multiply uncontrollably. Frameworks provide a starting point; a place to hang your code initially, while you’re still trying to figure out what’s going on. The web seems to have more frameworks-per-capita than anywhere else, but that doesn’t mean it has anything that fits your specific needs. Brandon Bloom writes about his experiences with Django, and why he’s been slowing dropping it. I feel his pain.

To me, there are a couple reasons to use frameworks.

First, frameworks fill in all the fiddly bits around the edges of our work. What you’re building, when you’re building a website, isn’t a URL dispatcher, or an authentication engine. You’re not building an ORM, nor are you building middleware. All of that is simply the foundation upon which you build your work.

Next, frameworks jumpstart your work. By providing you with a starting point, you can focus more on the actual interesting work and not so much on the grunt work of just getting things up on the web. This is a great thing, at least initially.

Finally, frameworks provide a way to think about a problem. Staring at a clean white sheet of paper is the most intimidating thing in the world. Frameworks put down a huge chunk of best-practices and assumptions that can really help you think about the problem at hand.

None of this, however, means that frameworks are forever. Frameworks only work if they fit what you need, and often that’s not true as you dig further and further into the problem domain. This isn’t to say they don’t serve a purpose. Django has gotten me running with a bunch of projects, and generally stayed out of the way. It isn’t perfect, and neither is any other framework. The important thing in all projects is to continually re-evaluate your assumptions, and that includes choice of frameworks.

At the end, the knowledge in your application-specific problem domain is where all the value is, not the framework. You can translate it to a new framework, or remove pieces of the framework if you need to. I would, however, make sure you understand why you’re replacing something, rather than just choosing to be different. Every line of code you write, is another line of code you’re solely responsible for. Sometimes a 80-90% solution with hundreds of developers is better than a 99% solution with one.

Funny thought

2009-08-19T10:59:03Z

Where were these people who are so concerned with spending money to provide healthcare when we were, and still are, spending even more to kill them?

District 9

2009-08-18T01:35:34Z

Those who know me know that I tend toward a love-affair for the dystopian future. Whether it’s Brazil, Blade Runner or Children of Men, the idea that the future is darker than the present rings truer than any fairy tail of human redemption. So, when I saw District 9 over the weekend, I instantly loved it. It turns the ever-constant “alien story” on its head. No longer is it “what will the aliens do to us”, but “what will we do to them?”

The darkest drive of humans is not hatred but fear; fear of that which is different. District 9 explores this fear with great aplomb. The acting is uniformly excellent, and more importantly believable. The special effects disappear into the story, as they should. And, in the end, enough is left unresolved to leave one wondering what might be and what could have been.

In the end, it is a story we’ve seen before. It is repeated over and over throughout our history, and as the movie visualizes, it will likely be with us until we are gone.

Obvious

2009-08-12T14:41:11Z

Deathers,

You already have a “death panel”, but they’re into it for the money right now. They let you die, they make more money. Idiots.

Simple

2009-08-12T14:41:28Z

Deathers,

It’s not a “debate” if all you do is scream. It’s nothing but bullying by cowards and idiots.

The strength to not fight any more

2009-08-07T13:49:55Z

Reading Terry Pratchett’s moving thoughts on the right to live, and die, in our own way. It closes with this line:

Life is easy and cheap to make. But the things we add to it, such as pride, self-respect and human dignity, are worthy of preservation, too, and these can be lost in a fetish for life at any cost.

I believe that if the burden gets too great, those who wish to should be allowed to be shown the door.

In my case, in the fullness of time, I hope it will be the one to the garden under an English sky. Or, if wet, the library.

It is not easy to think of the end of someone’s life. I certainly know that with everything that I have been through this year, that I would rather not think of it; and yet, there is power in thought. There is power in decision, and strength in not fighting.

My sister and I were fortunate to be present when our mother passed earlier this year. Because she had long suffered from COPD, the issue of what happened tomorrow often included a matter-of-fact discussion of what happened at the end. We both knew, deeply and with complete certainty, what mom wanted when the time came. There was no doubt there.

Yet the decision to let go was not easy, or simple. It was nothing less than a battle of selfish instinct versus selfless desires: my desire to have my mother in my life; her desire not to suffer any longer and not to become a burden on her children. We were fortunate to be surrounded by people who understood what we were going through. They knew how difficult the decision was, but they also knew what our mom wanted, as she was an adopted mother to them as well. Their certainty reinforced ours and gave us strength to make a decision that we did not want to make.

When mom finally took her oxygen off for the third time, and then decided to keep it off, we knew the time had come. So many years of suffering—of fighting. Nearly a decade with a disease the doctors insisted would take her within a year or two. More strength than either of us could imagine wielding, especially in the face of insurmountable opposition. The time had come to stop fighting.

Fortunately, we were able to reunite my mom with people who, for reasons small and large, had drifted apart. Clara, her friend and margarita buddy, for so many years. Caroline, the woman who had raised me—taught me to like liver and onions—and whose spirituality made some discussions difficult, so that she had hid things from her friend for years. In the end, when the last moments tick away, concerns about the future become meaningless and my mom was able to reveal things she had kept to herself. In those last moments, there was nothing but love and acceptance, unfiltered by society’s antiquated moors.

When the house finally stood still—the oxygen machine off for the first time in many, many years—and my mom lay peaceful for once, I realized that no matter how much it hurt, we had done what children are often called to do. To say goodbye, and to realize that we all live forever in the hearts of others. Our legacy is not money or property; our legacy are the memories we leave embedded in other’s lives. Our mom had battled harder than anyone had any right to expect, and lived to see her daughter finish graduate school and married; her son happily settled. The weights on her heart were lifted in those last days and she was finally at peace.

When the day finally comes for me that I might face the dimming days, I can only hope that I have inherited a measure of my mom’s strength to fight and to finally know when to fight no longer. I can only hope that those around me can find the strength to let it happen and find the peace in knowing that I find in my mom’s own graceful exit.

To my sister Kim and my brother-in-law Chad; to Cary; to my friends Sharlene and Brettany; to Clara and Caroline, and the truly amazing people at Hospice Austin, I can never say thank you enough. And to my mom, who never leaves my thoughts for more than a moment, thank you.

Brown M&Ms

2009-08-06T17:07:25Z

Why the brown M&M clause should be integrated into your project.

Python logging

2009-08-01T22:17:02Z

The current Python logging library is not well loved. Whether that’s because it feels derived from the Java logging tools, or just because it seems to require more incantations than feel Pythonic is irrelevent. It’s not loved; not even close.

Reading a posting by Edward Ream about improving the Python logging world, it strikes me as an interesting idea to wonder if logging should be less about hierarchies, and more about matrices. Strict hierarchical systems—think PKI, security classification, company org charts—tend to be brittle and require more than a little jerry-rigging to make function. With the rise of tagging in the Internet sphere, why not approach it from that perspective?

For example, instead of the debug levels:

CRITICAL
ERROR
WARNING
INFO
DEBUG

plus the odd one of “UNSET’, we would use a simple tagging system. Rather than having a single filtering identifier, we could use n-filters. What this API looks like is a bit more uncertain. It loses it’s clear logging.debug() API, and requires a bit more information, but I think that a lot of this could be simplified through how you grab the logger object.

More thinking is necessary, but it would seem that this could also help address issues around auditing as well.

Police and racism

2009-07-25T19:41:00Z

With the continuing obsessive coverage of the whole Gates arrest situation in Cambridge, it reminds me of a few things. The first is how totally worthless the “mainstream media” is in this country, that in the most deadly month yet in Iraq, with healthcare on the line, etc., all they care about is blowing up something that isn’t controversial into something gigantic. The media, as always, is more obsessed with playing news-maker than news-reporter.

Second, what President Obama said may have been poor word choice, but it was not incorrect. Arresting someone who had not broken the law—in their own home—because they seem to have not demonstrated the proper level of subservience and deference to cops who were mistaken is stupid. It is glaring stupid. It is a level of stupid derived from people so intoxicated with their own sense of self-righteousness and power-obsession.

Thirdly, anyone—black or white—who was being accused of a crime, in their own home, after having demonstrated that they were who they said they were, and who was being harassed by the police would be agitated. If I had just arrived back from a 15+ hour flight from China to not be able to get into my house, and then quickly found someone accusing me of being a criminal, I’d be a lot more than just agitated. I’d be seriously upset and argumentative.

Lastly, that whether people care to acknowledge it or not, people are treated differently based on their color of their skin, especially by the police. To not see that is to have ones head in the sand. This is most emphatically not a post-racial society. The election of an African-American man with a Islamic name is a demonstration of the total and complete abject failure of the Bush administration and Republicans, not the movement of a society past racial concerns.

Pet Peeve – Excel is not a database

2009-07-14T18:35:33Z

Please stop using Excel as a “database”. It’s not. It’s for doing calculations.

If you enter a page of data in a single cell, you are using it as a database, or something worse.

Just stop.

That is all.

It’s alive! Mac OS X on non-Apple hardware

2009-07-23T02:25:07Z

I tried. I really tried to use Linux on the Desktop. I tried Ubuntu, I tried Fedora. I tried OpenSolaris, and I tried my favorite free OS, FreeBSD. In the end, I was dissatisfied. From each, I liked different things; but, that’s not what this post is about. This post is how I got Mac OS X running with 4 cores, hyper-threading and 12GB of RAM.

First, let’s set the stage of the hardware involved. This is what I discussed in a previous post, and the only thing that has changed is the HD.

Antec ThreeHundred case
Corsair HX520W modular power supply
Asus P6T motherboard
Intel Core i7-920 processor
12GB of OCZ DDR3-1600 memory
EVGA nVidia GTS 250 video card
Hitachi DeskStar 7K100.B hard drive
Pioneer DVR-1910LS DVR-RW drive

If you have “identical” hardware, then this will get you through it. If you don’t, then hopefully this will at least share a few of the issues I ran into.

Before I start, I can’t thank enough everyone on the interwebs who have shared their experiences—both good and bad—because without those, I’d never have gotten this far. I also need to thank someone that goes by soniclmusic for this wonderful writeup that got me 98% of the way there.

Preparing Your Tools

Before you start, you’ll need the following software:

Licensed copy of Mac OS X v10.5 (Leopard)
iDeneb v1.3, based on MacOS X v10.5.5
DSDT Patcher v1.0.0e (get here)
DSDT Patcher GUI v1.0 (get here)
GFX Strings (get here)
Kernel extensions (kext) to enable audio (get here)
Kernel extension (kext) to enable PATA (get here)

Let’s explain the first two. At least in my instructions, you won’t be using the copy you bought, that’s simply something you need to do to pay for the OS. If you just steal the OS without paying for it, you’re a dick, and we’ll leave it at that. Once you’ve bought it, place it lovingly on your desk and admire the packaging. Oooooo ahhhhhh.

The iDeneb distribution can be found somewhere on the interwebs, just search your favorite search engine or torrent site. It will make this much easier. Do not use the v1.4 iDeneb installer. I was never able to get it to boot on my machine, no matter what magical invocations I tried. We’ll get the system up to 10.5.7 soon enough.

Step 1 : BIOS configuration

The first thing we need to do is change a bunch of settings in the BIOS. If you don’t do this, the installation will never work. Period. Some of these may make creaky old operating systems, like Windows XP, go down in a flaming spiral, but then you weren’t running Windows XP, were you? Linux, OpenSolaris and FreeBSD all deal perfectly fine with the changes that we are going to make. The high-level changes that need to happen are:

Enable AHCI for all SATA drives
Disable Hyper-threading
Disable all but the first core
Enable ACPI 2.0 Support
Enable ACPI APIC support

You can enter the BIOS configuration by pressing the DEL key on start-up. From there, you can navigate according to the instructions. You need to make sure the above changes are made, otherwise the DVD will never boot. Once that’s done, press F10 to save, and reboot the system.

Step 2 : Booting from DVD

I don’t change the boot order of my system. Instead, you can just press F8 when the system starts up to select a different boot device. On my system, the DVD is displayed as IDE:PIONEER DVR-113NP. Select that, or whatever yours is, and hit ENTER. This will start the boot process.

When the disc actually starts loading, it will ask you if you want to boot from the disc, or press F8 to enter additional options. I usually press F8 and enter one additional option: -v which will put it into verbose mode as it boots.

It will take a little bit, perhaps a few minutes, to start the OS, and then you will be in a “normal” Mac OS X installation environment.

Step 3 : Partition your hard drive

From the Utilities menu at the top of the screen, select Disk Utility, and then proceed to partition and format your drive. For me, I simply created one partition on the 1TB drive. This only takes a few seconds to complete, and then you can quit the tool to return to the installation process.

Step 4 : Configure the installation

As you go through the process, you will see a button labeled Customize. You want to click that and opening the hierarchy of options, select the first kernel (VooDoo), which is for both AMD and Intel machines. Next you want to go down to Graphics and nVidia, and select the NVInject extension that has the right amount of memory. I tried skipping this step—we will later use GFX Strings to fix it—but the system won’t boot since it won’t be able to see your graphics card the first time.

After that, you’ll want to go through the list of applications. You’ll want to select all of them at this point, you can delete things later.

Step 5 : Start installation

Once you’ve selected your options and installation drive, you can continue with the installation. I generally go up to the Window menu and open the log so that I can see what’s happening. On my system, it takes about 10-12 minutes to do an installation, but it may take longer. When it is done, there will be a button to reboot. Do that.

Step 6 : Initial OS configuration

When the system boots for the first time, you’ll get the fancy Apple welcome video, as well as a wizard to guide you through configuration. This stuff should be pretty self-explanatory.

Step 7 : Enable Hyper-threading and multiple cores

At this point, we only have 1 CPU core running, with all Hyper-threading disabled. Nice, but not really what we’re looking for. In order to make the system detect properly all the cores and allow Hyper-threading, we have to insert a configuration table for the ACPI so that it can see things properly. This configuration table is called the DSDT, or Differentiated System Description Table, and will be different for every system motherboard/CPU combination.

To install this, we need to do two things. First we need to create the DSDT for our system, and then second we need to make sure that it is loaded early in the boot process.

To create the DSDT configuration file, we’ll use the DSDT Patcher that I mentioned earlier. First, copy the DSDT_Patcher1.0.1e.zip file to your desktop and double click it to extract it. Then you need to open a Terminal window (Application -> Utilities) and then type the following commands, where the $ represents the default shell prompt.

$ cd ~/Desktop/DSDT_Patcher1.0.1e
$ ./DSDT\ Patcher

At this point you’ll get a question about what platform to emulate. Choose 0 (zero) for Darwin. The program will then generate a bunch of files, and may tell you about a few warnings. These can be safely ignored at this point. Next we need to edit the file to change some configuration. In the same Terminal window, continue:

$ nano Debug/dsdt.dsl

This will open an editor window to change the dsdt.dsl file to match what you need. When in this editor, you want to remove all the lines in the first section about CPUs that start with Alias. You can do this with the delete key on your keyboard and the arrow keys. When you’re finished, press Control-O (the control key, plus the letter ‘o’ at the same time) and then RETURN to save the file. Then you can press Control-X to exit the editor.

Back in our Terminal window, we will continue:

$ ./DSDT\ Patcher Debug/dsdt.dsl

When you run it, you’ll get a few errors about pre-existing files, you can ignore this. It will create a file in the current directory called dsdt.aml. You will copy this file to the root of the main boot drive:

$ cp dsdt.aml /DSDT.aml

Please note that the uppercase name is required. If you don’t do it in upper case, this won’t work.

Next, we need to monkey patch the Chameleon loader to make sure that it actually loads the DSDT configuration file we’ve created. To do this, we need to copy the DSDTPatcherGUI_1.0.zip file to the desktop of the machine and double click to extract. Using the Finder, browse down into that folder to Tools and double click on Bootloader.pkg. This will start up a standard Mac OS X installer. Go through it to the finish.

Step 8 : Configure your boot.plist

To make sure things boot correctly, you want to edit the file in:

/Library/Preferences/SystemConfiguration/com.apple.Boot.plist

You can use the TextEdit program to do this, or use nano as you did before. You want to make sure that when you’re done editing at this point, that it looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Kernel</key>
    <string>mach_kernel</string>
    <key>Kernel Flags</key>
    <string>busratio=20</string>
    <key>Timeout</key>
    <string>5</string>
    <key>device-properties</key>
    <string></string>
  </dict>
</plist>

Step 8 : Enable all cores and Hyper-threading

At this point, we’re ready to give it a whirl. You want to go to the Apple menu, select Reboot, and then when the system reboots, hit the DEL key to enter setup and enable both Hyper-threading and all cores. Press F10 to save, and then boot the OS as usual.

If everything has gone according to plan, and my scribbled notes, then the system should boot with all the CPUs activated.

Step 9 : Replace NVinject with GFX Strings

The final thing that I had to do, because the system would random hang the graphics subsystem, was move from NVinject to the GFX Strings solution. The solution mentioned here works perfectly and can be followed to the letter. I’m not going to copy and paste it here.

Once you’ve made those changes, you can reboot again.

Step 10 : Upgrading to 10.5.7

For me, I downloaded the combo updater from Apple’s website, and ran it as normal. When I finally rebooted, I made sure to change the boot options to -f -v, just to make sure that the caches were rebuilt. The first time I rebooted, though, it hung part-way through. I rebooted again and I’ve had no problems since. You can then proceed to use Software Update to bring the system up-to-date with iTunes, etc. I had no issue doing it that way.

Step 11 : Enabling audio support

When I first installed all this, there was no audio support. System Profiler reported that there was no audio subsystem. After a bit of searching, I discovered and tested using a couple of kernel extensions (kext) to enable it. They are linked at the top of this post, and you can install them using kExt Helper, which was installed when you installed iDeneb. It will require a reboot, and you will need to select “line out” as your output. I’ve not tested the digital output yet.

Step 12 : Fixing the date/time clock issues

One of the first things I noticed when I first got the machine up and running was that the date/time in the menu bar was constantly fast. So fast, in fact, that it would be at least 1-2 minutes ahead for every 5 minutes that went by. After an hour, 12 minutes ahead, etc. It was insane. Opening the Date & Time Preferences panel would fix that, as it would force an NTP resync and call settimeofday(2) to fix up the clock. Then it would drift more and more and more. A solution was to be found in a posting in the InsanelyMac forums. It was the FSB on the processor. There’s multiple ways to fix this, but I found that downloading the Chameleon v2 installer and running it got you what you needed. The latest release candidate (RC1) includes support for auto FSB-speed detection. Problem solved.

Step 13 : Installing support for the JMicron PATA interface

Using the kExt Helper, you will want to install the JMicronATA.kext extension and reboot. That’s all you need. For me, it also enables burning of DVDs.

Open Issues

Right now, there’s no major hardware support issues outstanding. What I am still seeing is that sometimes certain preferences (like requiring a password when coming out of a screen saver) are not saved sometimes. This seems to be random.

Conclusion

Hopefully this provides some help to people who want to play with Mac OS X on non-Apple hardware. Good luck.

Updated (12-July-2009): Now includes instructions on enabling audio.
Updated (13-July-2009): Included how to resolve the clock issues.
Updated (21-July-2009): Now have working parallel ATA (PATA) with JMicron controller.

Slides From the Past, Part Deux

2009-06-16T20:00:28Z

I’ve started posting the slides, and you can find them on my flickr account. I’ve still got a long way to go, but I’m going to try and make some constant progress.

Bobo is Back Baby!

2009-06-16T19:58:33Z

Ohhhhhhhhhhhh yeah! Or sorta.

The first “object publishing” framework I ever used—and this was many, many moons ago—was called “bobo”, and was the hurried effort of Jim Fulton, a wielder of much Python magic. It soon got subsumed into Zope as the foundation of object publishing and disappeared from the interwebs. Now, it’s back, but this time it’s got a fresh coat of WSGI and decorators to give it that new framework smell.

Will have to take a peek and see where it fits in this day and age.

Slides from the past

2009-06-15T13:45:51Z

In going through my mom’s house to prepare it for sale, I stumbled upon a box of slides that I had been looking for for many, many years. It contains several hundred slides that my dad shot back between 1960 and 1962 when he was stationed in Korea. Now I’ve got to work on scanning them all in.

The funny thing, to cite Paul Simon, is that the Kodachrome slides are stunning. The Ektachrome (E-2 processed) slides suck unbelievably.

Apple pulling a “Microsoft”

2009-06-08T19:39:06Z

Let’s face it … nobody buys a Windows machine for it’s stellar functionality or beautiful interface. They buy it for the ecosystem (software) around it, and the fact that Microsoft dominates the desktop domain. While Apple may have lost out on the desktop to Microsoft, they didn’t ignore the lessons to be learned.

With today’s announcement of the new iPhone 3GS and the reduction of the iPhone 3G to $99 (effective immediately). Let me explain why the Palm Pre has a sisyphean task:

1,000,000 SDKs downloaded
50,000 applications
40M iPhones/iPod Touches

That is the iPhone/iPod ecosystem. That leaves out the thousands and thousands of accessories that are “Made for iPhone”. It’s a gigantic impediment to adoption, and more importantly a huge risk to consider for any developer. You don’t unseat the “best” while being only incrementally better. You have to change the game.

The iPhone Compass

2009-06-04T20:47:39Z

Some rather horny commentators are dismissive of a compass and its importance in the iPhone, but here’s why I think it’s actually more interesting than people think:

Knowing which way the phone is facing when it’s on can tell you which way the user is facing.
Knowing which way the user is facing allows you to do fun things with mapping and social media. Think turning around in place and watching Google StreetView spin to match.
Navigation, especially when walking, would be strongly assisted by knowing which way the user is facing, and therefore probably walking.

Not a huge change, but it’s a refinement, and since you can get a very small compass, it won’t consume much space. More sensors is better than less.

D’oh Moment

2009-06-02T14:33:58Z

There’s nothing worse than realizing that the reason the audio on your new machine isn’t working is not because of stupid Linux drivers, but because you’re a moron and somehow disabled it in the BIOS. D’oh!