Meet modern Bumblebee,

This transformer was crazy hard to transform, in the couple of days I was visting with the 4 year old owner, the head “popped off” multiple times, and I couldn’t get it back on without feeling I was going to break the toy. Lucky said child’s mother had the magic touch.

But transforming challenge aside, it was a pretty cool model/toy. Just not like the Starscream or Optimus Prime of my days gone by….

To this end, I point would-be-helpers to nikonhacker.com, I’ve been contributing there, as best I can.

It’s very addictive having such large inflows of traffic to your site/blog, that I was reluctant to redirect it, but I have also felt since November a presure not to post stupid stuff like I used too, or am about to post, thus have been silent (besides the travel and working on things that are not up for chatter).

So there’s my Nikon update…. I have been having a good family holiday..

My first game was a six person double deck game, which was slower, and very brutal. Since then I have played many a game with Jacob (single player) and it’s fast and fun.

In fine to twenty minutes, you can compact all the fun and rage of the full 4 hour game, but it’s done, and you can move on so much quicker. It has stealing, ripping people off, double crossing, saying no (it’s a card, but it the best feeling in the game rejecting a big play, just watch out for the reverse no,back at you…).

It’s a very fun game, and while there it can have all the rage of the original it, also can be started and finished within 30 minutes.. so many hands can be played giving the “balance” missing from Monopoly.

After telling him the shortcuts for manually fixing the issue (press O for the data segment, or Alt-R for any segment offset), he wrote an IDC script to do it en mass.

Thus (made up example code)

push ds;

mov ax, 0x1234;

push ax

should look like:

push ds;

mov ax, ds:dword_1234;

push ax

Here’s his script:

#include <idc.idc>

static main()
{
    auto seg, loc;
    auto movloc, movtarget;
    auto xref;
    auto dsegbase;

    dsegbase = SegByName("dseg") * 16;
    Message("dsegbase=%x\n", dsegbase);

    Message("========================================\n");
    seg = FirstSeg();

    while(seg != BADADDR )
    {
        Message("----------------------------------------\n");

        loc = SegStart(seg);

        if( Byte(loc) != 0xCD || Byte(loc+1) != 0x3F)
        {
            Message("Fixing indirect push [ds:xx] refs from %s\n", SegName(seg));

            while(loc != BADADDR && loc < SegEnd(seg))
            {
                if (GetMnem(loc) != "push" || GetOpnd(loc, 0) != "ds")
                {
                    loc = NextHead(loc, BADADDR);
                    continue;
                }
                loc = NextHead(loc, BADADDR);

                if (GetMnem(loc) != "mov" || GetOpType(loc, 1) != o_imm)
                {
                    loc = NextHead(loc, BADADDR);
                    continue;
                }
                movloc = loc;
                movtarget = GetOpnd(movloc, 0);
                loc = NextHead(loc, BADADDR);

                if (GetMnem(loc) != "push" || GetOpnd(loc, 0) != movtarget)
                {
                    continue;
                }

                // At this point, we know we're pushing a [ds:x] combo.
                //Message("%x: mov %s, %s\n", movloc, movtarget, GetOpnd(movloc, 1));

                // Abort if there already exists a Dxref
                xref = Dfirst(movloc);
                if (xref != BADADDR)
                {
                    continue;
                }

                Message("  Updating %s:%04x\n", SegName(seg), (movloc - seg) & 0xffff);
                OpOff(movloc, 1, dsegbase);
            }
        }

        seg = NextSeg(seg);
    }
}

I’ve got most the area’s of code identified (where they are, not what they do), but there are some puzzles, with some chunks of code that are used (eg selects a picture to be shown based on shooting mode) but the code it self is not directly linked to, and it’s address in not present in the image. There is defiantly some form of jump/call table compression/encoding done, as there are functions that do some maths, and then call the result. So that needs to be puzzled out.

So to help map the data (and thus remove possible options from above puzzle), I previous mentioned mapping the jpg’s out:

As can be seen in this small sample, there’s the icons for the different shooting modes, and three colour schemes.

Last night I was working on using a modified version of BinViz (original found here) and have found how the font’s and overlay text/images are packed, and I am in the process of tracking down how the width/size information is encoded in the associated data tables. Shown below is the same block of data shown, but at two different widths, showing the “Dial” overlays and the “Bulb Time” text:

It’s quite neat looking at the Asian font sets, as the fonts/overlays use subpixel rendering, which can be seen in the green arm of the sports mode dial icon above. Much simpler, the normal overlays are just black/white, and now look better X/Y scaled.

I started a Google Code Project called Nikon Firmware Tools in which I’m placing the tools I’m using and the changes I’m making to them as I go. So interested developers can look there. Sorry only code so far.

The code they used was the big endian form of the CRC16, but running on little endian CPUs.

Turning the Wikipedia CRC code into C# follows:

For the D7000 A & B, D300S A&B, D3S A & B, D3100 A, D5100 A firmware files use start: 0×0000, mask=0×1021

For all models (D3S, D300S, D7000, D5100 & D3100) the CRC is a normal Xmodem CRC16, the originally reported difference for D5100 & D3100 was due to a XOR error (by me), that is now found & fixed.

static int crcBig(byte[] data, int len)
{
    int rem = 0x0;

    for (int i = 0; i < len; i++)
    {
        rem = rem ^ (data[i] << 8);
        for (int j = 0; j < 8; j++)
        {
            if ((rem & 0x8000) != 0)
            {
                rem = (rem << 1) ^ 0x1021;
            }
            else
            {
                rem = rem << 1;

            }
            rem = rem & 0xFFFF; // Trim remainder to 16 bits
        }
    }
    // A popular variant complements rem here
    return rem;
}

For the D5100 B firmware use start= 0x4ed4, mask= 0×1021

D5100 checksum passed, now loading firmware

D5100 HaCkEd firmware running.

The D3100 B firmware also matches the same start value, so I’d assume it’s common to both. And the only thing I modified here was the firmware help message, and I’ve not tested if you can load the 1.01 firmware over top it’s self. That is the next step. Also to make a tool (Vitaliy?) the patches and encrypts for you to avoid typo’s.

[Update]
Very import and good news, firmware lets you flash same version over top it’s self. Thus my camera is now back running normal Nikon 1.01 firmware. Warranty unbroken!
[Update 2: 27 Nov] Fixed CRC code in light of XOR code change.

I also said I would be happy to accept money in order to pay for a legit version of IDA Pro. I’ll return the one donation I did get.

I then proceeded to have many emails with Vitaliy about how the work could be done with the free version. While his methods will technically work, I’m just not sure I want to publicly endorse them.

So to avoid offending ‘the scene’ with personal greed, I’m pulling my hat from the race. I’ll just be the guy that solved the encryption and stick with that.

Looking at the B firmware you can see text, for example English @ 0x5ACC68 to but the table of addresses prior to this text block have the same relative spacing of values but are 0×40000 bytes higher in value.

Browsing elsewhere in the file, shows this pattern holds up for the entire file. You can check by finding some text, ad 0×40000 to the text file address, then search for that resulting hex value, ta-da

So we can assume the code is loaded starting at 0×40000.

Next is what we are going to decode the file with. If you have commercial version of IDA Pro then your in luck as it comes with the Fujitsu FR CPU decoder built in. I only have the free version, so am not so luckly.

Kevin Schoedel however has written a pretty good dissembler for the FR CPU, as part of previous Pentax firmware work. I struggled to get the original code working correctly, so I rewrote it in C# and now understand how to use it, and really appreciate his efforts. But I am left oh-so-much wanting IDA Pro style everything. The worst thing is I almost want to pay the $500USD to have that IDA Pro magic, but if I have that money ‘spare’ why didn’t I just buy the D7000 to start with.

Anyway, my present D5100 B firmware DFR import file looks like:

# File map: D5100
-i 0x00040000-0x00BFFFFF=0x00000000

# Memory map:
-m 0x00040000-0x00040947=CODE
-m 0x00040948-0x000409b3=DATA:L
-m 0x000409b4-0x00050068=CODE

# interupt vector table.
-m 0x000dfc00-0x000dffff=DATA:L

#-m 0x000e0000-0x0x000fffff=DATA:L // this is 0xFF rubbish

-m 0x00100000-0x00236A4A=CODE

and heres my C# port of DFR, to use the above with.

If Ilfak wants to give me a free copy of IDA Pro, or even just the FR processor that works with 5.0, I would be very humbled.

The FR – CM71-00101-5E.pdf instruction manual from Fujitsu is very useful.

Here the first 0x4a bytes of the firmware, you can see the interupt vector table being loaded at 0x4002c.

Disassemble 0x00040000-0x00040947 (file 0x00000000) as CODE

00040000 (00000000)  9F80 5000 0000         LDI:32  #0x50000000,R0
00040006 (00000006)  C011                   LDI:8   #0x01,R1
00040008 (00000008)  1501                   STH     R1,@R0          ; 0x50000000
0004000A (0000000A)  9B00 03E0              LDI:20  #0x003E0,R0
0004000E (0000000E)  9F81 4700 0000         LDI:32  #0x47000000,R1
00040014 (00000014)  1401                   ST      R1,@R0          ; 0x000003E0
00040016 (00000016)  9B00 03E7              LDI:20  #0x003E7,R0
0004001A (0000001A)  C031                   LDI:8   #0x03,R1
0004001C (0000001C)  1601                   STB     R1,@R0          ; 0x000003E7
0004001E (0000001E)  8710                   STILM   #0x10
00040020 (00000020)  9F8F 6800 0800         LDI:32  #0x68000800,R15
00040026 (00000026)  9F80 000D FC00         LDI:32  #0x000DFC00,R0
0004002C (0000002C)  B300                   MOV     R0,TBR
0004002E (0000002E)  9B00 0600              LDI:20  #0x00600,R0
00040032 (00000032)  9B41 0185              LDI:20  #0x40185,R1
00040036 (00000036)  1401                   ST      R1,@R0          ; 0x00000600
00040038 (00000038)  9B00 0640              LDI:20  #0x00640,R0
0004003C (0000003C)  C401                   LDI:8   #0x40,R1
0004003E (0000003E)  1401                   ST      R1,@R0          ; 0x00000640
00040040 (00000040)  9B00 0680              LDI:20  #0x00680,R0
00040044 (00000044)  9F81 799F 9910         LDI:32  #0x799F9910,R1
0004004A (0000004A)  1401                   ST      R1,@R0          ; 0x00000680

The second address in parens is the original file offset to help with looking up the file.

As noted in the previous post, the bundle is checksum’ed, but I didn’t mention that each sub-file is also checksum’ed. But you can see the checksum at the end of the files:

Now on some Pentax forum I was reading about people wondering if the checksum was checked. So I altered the help file text for the viewing firmware information to make the ‘c’ of camera upper case.

Now the file is just XOR’d with the larger one time pad discussed earlier, and ‘c’ to ‘C’ is just XOR’ing with 0×20, applying the same XOR to the encrypted file at the same location, will have the net result of making the output upper case.

I put this altered firmware on a SD stick, and popped in into the D5100. Long story short, it recognized the update, but once I started the update, it quit really early in the process. The amount of time I’d expect for the checksum to be checked…

So I then turned to the internet for checksum algorithms, and it doesn’t appear to be a CRC-16-anything, as I brute forced the full possibility space, that was a fun adventure in multi-threading batch processing. Nor does it seam to be a Fletcher’s Checksum.

So now I’m up to searching for the answer in code…

[Update: 25 Nov] Answer found by Roos, see here

Firstly after removing the encryption from the Nikon DSLR firmware bundle file, we should extract each file from the bundle.

Form my looking at the files I have, there is 0×20 bytes of fluff, then there is a file count, header length, and a couple of dummy int32′s. Then there’s 0×10 bytes of file name, file start, length, and two more dummy int32′s. After that there’s a word ‘checksum’ and some padding bytes.

Thus this code (C#) pulls the files out of the bundle file decoded by the previous post:

static void ExactFirmware3(string fileName)
{
    if (File.Exists(fileName))
    {
        BinaryReader br = null;

        try
        {
            br = new BinaryReader(File.Open(fileName + ".out.bin",
                FileMode.Open, FileAccess.Read, FileShare.ReadWrite));
            br.BaseStream.Seek(0x20, SeekOrigin.Begin);

            uint count = ReadUint32(br);
            uint headerlen = ReadUint32(br);
            uint dummy1 = ReadUint32(br);
            uint dummy2 = ReadUint32(br);

            var header = new List<Tuple<string,uint,uint>>();

            // Read Header
            for (int c = 0; c < count; c++)
            {
                string firmwareName = Path.Combine( Path.GetDirectoryName(fileName),
                    ReadString(br, 16));
                uint start = ReadUint32(br);
                uint len = ReadUint32(br);
                uint hdummy1 = ReadUint32(br);
                uint hdummy2 = ReadUint32(br);

                header.Add(new Tuple<string, uint, uint>(firmwareName, start, len));
            }

            foreach (var t in header)
            {
                DumpFile(br, t.Item1, t.Item2, t.Item3);
            }
        }
        finally
        {
            if (br != null)
                br.Close();
        }
    }
}

static void DumpFile(BinaryReader br, string fileName, uint start, uint len)
{
    BinaryWriter bw = null;

    try
    {
        bw = new BinaryWriter(File.Open(fileName, FileMode.Create, FileAccess.Write,
            FileShare.ReadWrite));

        br.BaseStream.Seek(start, SeekOrigin.Begin);

        var data = br.ReadBytes((int)len);

        bw.Write(data);
    }
    finally
    {
        if (bw != null)
            bw.Close();
    }
}

Now for the D5100 we have two files, a640m010100.bin and b640101b.bin. The D7000 firmware is x75xxxx.BIN, the D3100 is x74xxxx.BIN, D300S is x81xxx.BIN and the D3S is xD3Sxxx.BIN.

All these systems are running the ‘Softune REALOS/FR is Realtime OS for FR Framily’, the Axxxx.BIN firmware is for the IO control CPU (metering, focus, buttons) and the larger Bxxxxx.BIN is the main processor firmware, with the main UI and processing (Fujitsu FR CPU).

Some great insight was found from the D70 Hack Project, which was the only remaining information I found. It’s in German, so thank you Google Translate. Also D7000 tear by by Chipworks was very inspiring.