Asgeir S. Nilsen tenker litt

C++0x: A const * const Odyssey

2010-03-10T20:52:00.001+01:00

This is a presentation I and my colleague Anders Schau Knatten held at an internal software seminar this February.

It was meant to cover both some introductory material, best practices we have acquired, new features to look forward to and things that might (or should) scare you.

However not one member of our audience had less experience with C++ than Anders and myself have together, so we quickly jumped down into details.

You should probably view the presentation full screen and open the speaker notes to get the most out of it.

You might find this additional note on slide 43 interesting: A non-virtual destructor once made it to the board of directors..

Why C++ is Male

2010-03-08T21:39:00.011+01:00

Through recent frustrations, and needless to say, core dumps, I have realized that C++ as a language have some distinct features often associated with men: It can only do one thing at a time, and it cannot communicate. In this post I'll elaborate on these.

It Can Only Do One Thing at a Time

In C++ threads or the notion of multiprocessing simply does not exist. The language has neither a thread concept nor any thread synchronization primitives like mutexes, semaphores and locks.

Even relatively simple constructs like static members simply do not work in a multi-threaded context. As described by Scott Meyers and Andrei Alexandrescu in C++ and the Perils of Double-Checked Locking, no matter how many volatile keywords or extra locking you add in the mix, it still won't work consistently. Some compilers try to help you by fixing this for you, but since that is compiler implementation dependent it does more harm than help.

C++ has no thread-safe memory model, and does not provide any means for a memory barrier. Which means that any "clever" compiler or instruction dispatcher is free to reorder your statements in whichever way provides the same result in a single thread of execution.

C++ Cannot Communicate

What is the de facto standard for communicating with the outside world, and has been for the last two decades? The Internet Protocol, or more specifically, the TCP and UDP protocols on top of it. But the C++ language does not support this protocol. The only means of communication provided by the language are streams for input and output, and they can only be used for talking to files or to the console.

You Can Teach C++ New Tricks

In the recent years I've learned the art of walking and chewing gum at the same time. I can even pay attention to a conversation while doing other stuff, or at least I think I can.

Being a married man I have also learned to cope with my lack of communication skills, to some extent. Neither of these feats came easy. They were achieved after failing a lot and suffering many crashes.

I have never tried teaching a man to do these things, but having been taught these capabilities has provided some insight into the requisite effort and amount of frustration involved.

Trying to teach C++ to do several things at the same time while communicating with others can also be accomplished. It is however probably equally frustrating as teaching your significant other something he obviously wasn't designed to do. Both capabilities require that you deal with the more primitive foundation the individual has evolved from (Neanderthals and C), and relying on cultural constructs (operating system interfaces) that have been added on top of evolution and behave differently from individual to individual.

In many circumstances the resulting behavior is undefined, and will certainly cause some surprises. You also have to meticulously and in great detail instruct your friend on every step required to perform the task, in a language he understands.

The Modern Man: C++0x

Men have evolved, and so has C++ as well. In C++0x the language has grown the capability of doing several things at once, but it still prefers to do so alone in a dark corner.

Links for 2010-02-24 [del.icio.us]

2010-02-25T00:00:00-08:00

The Agile Train to Trondheim
A museum train that will leave from Oslo in the morning and arrive in Trondheim by the evening, just in time for the conference. Onboard there will be a mini-conference with lightning talks and open space sessions, plus good food and lots of opportunities to get to know other conference goers. This is the perfect way to get the most out of your journey to Trondheim.

UML in a new way

2010-02-24T22:42:00.007+01:00

Once again I found myself in the need of writing some UML. My last attempt nearly got me carpal tunnel syndrome and a broken mouse.

Yet I once more went on the quest of finding a UML editor, bracing myself for a crunch down on the little pointer with a tool that would eat all my RAM and spit it back out.

This time around however, I discovered PlantUML. The process of writing for instance an action diagram became this easy:

@startuml

(*) --> "Download PlantUML"

--> "Run it"

--> "Open Notepad"

--> "Write Some Text"

--> "Hit Save"

--> "Look at the generated diagram"

--> (*)

@enduml

The tool supports Sequence, Use Case, Class, Activity, Component, and State diagrams, and only requires GraphViz.

And if Notepad isn't your tool of choice, you can choose between Word macros, Eclipse plugins, Ant tasks, or 10 other ways of running the tool.

Since it's text you can easily store it in version control, source code comments, pass it along in emails, tweet it, generate it, and parse it.

Batteries included, mouse not required. Highly recommended.

New version of XPath XML editor

2010-01-14T22:51:00.005+01:00

Recently I needed to use my XPath XML editor to tweak a number of POMs. Only catch was that I was required to do this on a Windows machine.

What's the catch you ask? XML and Java work just as fine on Windows. However, my editor requires a list of POM file names on the command line, something I quite easily fixed with

`find . -name pom.xml`

This isn't easy in CMD, but even Windows has evolved recently. Enter Windows PowerShell. The PowerShell replacement is pretty simple once you get to know the PowerShell basics:

$(ls -include pom.xml -recurse -name)

If you'd like to tinker with it, I've also pushed the xmleditor to GitHub.

Links for 2009-11-24 [del.icio.us]

2009-11-25T00:00:00-08:00

InfoQ: Test Driven Development and the Trouble with Legacy Code

Links for 2009-11-14 [del.icio.us]

2009-11-15T00:00:00-08:00

NASA - 2012: Beginning of the End or Why the World Won't End?
Remember the Y2K scare? It came and went without much of a whimper because of adequate planning and analysis of the situation. Impressive movie special effects aside, Dec. 21, 2012, won't be the end of the world as we know. It will, however, be another winter solstice.

Links for 2009-11-10 [del.icio.us]

2009-11-11T00:00:00-08:00

A Git Workflow for Agile Teams
Suggest workflow for git development. Feature branches with rebasing from an always-green master.

Links for 2009-10-21 [del.icio.us]

2009-10-22T00:00:00-07:00

What Mother never told you about SVN Branching and Merging « Design By Gravity
Using Bunny Hopping (rebase really) to keep a feature branch in sync with trunk

Links for 2009-10-14 [del.icio.us]

2009-10-15T00:00:00-07:00

Google Docs Viewer
Easily view or embed PDFs, powerpoints, or TIFFs on your web page

Six questions on TDD

2009-10-13T21:55:00.004+02:00

Read Uncle Bob's TDD Triage:

"Is TDD a replacement for architecture?
Is TDD a replacement for design?
Should TDD be used for every line of code?
Well, if you are going to write some tests afterwards, why not write all tests afterwards?
Given that we accept the need for tests, why the resistance to test-first?
Wouldn’t it be faster without such high test coverage?"

and Echoes from the Stone Age:

"TDD
The Design Pattern Religion
Minimizing Concurrency"

Very good reads indeed. It is of course triggered by all the controversy of Joel Spolsky's blog on Jamie Zawinski being a Duct Tape Programmer.

Here are some of my comments:

Learn concurrency principles. You'll need it. Moore's Law no longer gives you more clock cycles per second, it gives you more processor cores. To effectively utilize a modern CPU you have to keep all cores busy. And if you struggle keeping two quad-cores busy, imagine keeping 1,000 cores busy. It probably requires a fundamental programming paradigm change like the proposed fork-join framework for Java.
Use high level building blocks like java.util.concurrent and java.util.concurrent.atomic. It gives you well tested code that is maintained by Other Smart People, relieving your brain to work on more important things. Two classes to study: ExecutorService and BlockingQueue.
TDD is a good way of writing safe concurrency code. Design each element as a single threaded execution flow, and assert state before and after. Pay attention to locks and shared data.
Architecture is Important. You don't build a skyscraper or a space station in the same way as a dog house or a residential house.
Testing is a good way of exploring unknown APIs or legacy code. You basically assert the expected output or behavior of the code you are exploring.
Duct Tape is good for many things. You can even build a sailboat out of it. But you probably wouldn't enlist for the Volvo Ocean Race with it?

Links for 2009-10-07 [del.icio.us]

2009-10-08T00:00:00-07:00

Just Enough MBA to Be a Programmer

Versions Maven Plugin

2009-08-24T11:19:00.003+02:00

Ever struggled with keeping versions of dependencies or inter-module links up to date? Now the Versions Maven Plugin comes to the rescue! It has readily available goals for tasks like:

The worst thing about this plugin is that it haven't appeared sooner. It also means I can now retire my XML Editor tool's primary usage.

HTTPS connection explained

2009-07-28T10:36:00.001+02:00

If you wonder how HTTPS works, you could read this one:

The First Few Milliseconds of an HTTPS Connection

After that, please come back and read my post TLS: A Broken Trust Model about the flaws of how HTTPS establishes a trust relationship between client and server.

TLS: A Broken Trust Model

2009-06-11T20:45:00.002+02:00

What is TLS

TLS stands for Transport Layer Security. It was formerly known as SSL, or Secure Sockets Layer. In short, it's the protocol behind the s in https, making internet banking, passwords and other data safe from eavesdropping while browsing.

In https, TLS also has another purpose -- trust. Through a chain of digital certificates and certificate authorities, the system tries to ensure you that the web site you are visiting is controlled by the company they claim to be.

How Does TLS Work

To protect communication, an encryption key has to be agreed upon by both ends in the conversation - namely the web browser and the web server. Because of efficiency this key must be identical on both sides. But how do you exchange this key in a secure fashion, ensuring no-one else can access it?

TLS solves this by encoding this key in another key. This other key is asymmetric, meaning different keys are used for encrypting and decrypting data. The key used for encrypting data can thus be openly exchanged. This encryption key, also called the public key, is encapsulated in a digital certificate with other information about the rightful owner of this key.

Among other things, this certificate contains the DNS name of the web server who owns it, and the name of the organization or individual it is registered to.

This certificate is presented to the browser by the web server when they initiate the TLS session.

The Purpose of those Certificates

But how do you know that this certificate is valid? This is where certificate authorities come in. They sign digital certificates, telling us that they vouch for the validity of this digital certificate and have verified that it is issued to the correct organization or person.

And how does the certificate authority identify itself. By another digital certificate. And how do you know this digital certificate is trustworthy? By yet another certificate authority, vouching for this intermediate certificate authority.

This obviously cannot continue forever. This is why TLS relies on root certificates. These certificates are ultimately trusted, meaning you simply accept that they are valid.

In order for this certificate chain to work in practice, these root certificates are installed in every web browser as they are distributed by the web browser vendor.

Who do you trust?

The web browser's ultimate trust in root certificates is distributed among a large number of corporations like Verisign, Thawte, COMODO, Microsoft, Deutsche Telekom, Entrust, Geotrust, Equifax, GlobalSign, Go Daddy, GTE CyberTrust, Starfield Technologies, and USERTrust. These comprise the list that my Internet Explorer 8 contains.

Firefox 3 contains the following staggering list: TÜRKTRUST, ABA.ECOM, AC Camerfirma SA, AddTrust AB, America Online, AOL Time Warner, Autodidad de Certificacion Firmaprofesional, Baltimore, beTRUSTed, Certplus, COMODO, DigiCert, DigiNotar, Digital Signature Trust, Entrust, Equifax, GeoTrust, GlobalSign, Taiwan Goverment Root Certification Authority, GTE Corporation, IPS Internet Publishing Services, IPS Seguridad, NetLock Halozatbiztonsagi, Network Solutions, QuoVadis, RSA Data Security, SECOM Trust, SecureTrust, Sonera, Staat der Nederlanden, Starfield Technologies, StartCom, Swisscom, SwissSign, TC TrustCenter for Security in Data Networks, TDC, Thawte, Go Daddy, USERTRUST, Unizeto, ValiCert, VeriSign, VISA, Wells Fargo, and XRamp Security Services.

So: Who decides which companies are admitted into this inner circle of ultimately trusted organizations? What means do they take to ensure that they are as trustworthy as they claim. And why is there such a large difference between Microsoft's and Mozilla's list?

A Snake Oil Market: Issuing Digital Certificates

These certificate authorities have created an artificial market, selling their trust at a high price. Furthermore, since digital certificates always have an expiration date, after which the certificate is no longer valid, they have ensured return customers as well.

The organizations provide little added value, apart from the aforementioned trust. But are these certificate authorities really more trustworthy than the organizations forced to purchase digital certificates from them?

What Is Wrong With TLS

It boils down to trust. If you were to make a top ten list of the organizations you trust most -- what would these organizations be? And are any of the organizations mentioned above on that list?

Why can't you choose for yourself who to trust?

What is TLS supposed to protect?

TLS protects the data transmitted between the browser and the web server. Often this is your personal or confidential data, like user identification, financial information or other data which you would like to be kept secret, and that the organization you exchange data with often is mandated to protect.

What are the risks involved?

The most severe risk is that a certificate authority issues a certificate to the wrong people. These people could then pose as the organization the certificate is issued to, and steal your personal data without you being aware of it. The browser would claim the communication is secure with a trustworthy party, and it's impossible to detect that something wrong is going on.

A Better TLS Key Exchange

Remember, the primary purpose of the digital certificate is to exchange and protect the key used to encrypt the data in transit between the browser and the web site.

This key could be protected in other ways, two of which are suggested here:

1. The browser issues a digital certificate

The web browser can generate a self-signed certificate and present it to the web server when initiating a TLS session. The web server doesn't know whether this certificate is trustworthy, but that doesn't matter. It's only a matter of exchanging an encryption key in a secure manner, not trust. Most web sites require you to log on to identify yourself and establish this trusted relationship.

This certificate can be generated when the browser starts, and be destroyed when the browser exits.

2. The encryption key is based on shared secrets

The encryption key, or a scheme to generate an encryption key, can be shared between the browser and the web site. This can be based on the user's authentication data like user name and password. The browser can then start sending encrypted data, expecting the web site to know how to decrypt them.

A Better Trust Model

Since the data exchanged is yours -- shouldn't you decide whether you trust the organization you exchange data with?

This could be solved by removing the intermediate steps in the trust chain. Either you trust the party you are exchanging data with, or you do not. The decision is yours and yours only.

For example, when you establish a customer relationship with an organization, this organization could send you a self-signed digital certificate identifying itself. You would then choose whether you accept or reject this certificate.

Conclusion

TLS in itself is not broken, but the trust model employed is. Furthermore, the TLS protocol does not need to change, at least not significantly, to implement the improvements suggested. The key exchange part needs to be revised, and the way browsers handle digital certificates must change.

But it would create an overall more trustworthy solution, removing an unnecessary middle man.

Why submitting clear text passwords is bad

2009-05-07T15:29:00.002+02:00

Jeff Atwood reminds me why even submitting passwords in the clear to a web site can be dangerous: You basically have to trust that they are storing the password in a secure fashion.

Which they more often than not don't.

Coding Horror: I Just Logged In As You: How It Happened:

"I guess I can tell you, so you don't fall into this trap again. There's a site I help out with that doesn't salt their passwords. They're MD5 encrypted, but if you've got a dictionary password, it's very easy to use a reverse-MD5 site to get the original. I was able to figure out you were a user on the site some time back, and realized I could do this, if only I knew your openid provider..."

There are basically two ways of solving this:

Use different passwords for different web sites. SuperGenPass takes the hassle out of this by generating a password based on a master password and the site's domain name.
Hash the password in the browser before submitting. My blog entry Password Authentication Without Revealing Your Password demonstrates a solution to this one.

I really wish client side hashing was more prominent than it is. I have seen it in action only two or three times, one of those implemented by myself.

Hemmelig kjennelse i Stavanger Tingrett om piratkopiering

2009-05-06T16:42:00.005+02:00

tu.no - Slik forklares hemmelig pirat-dom - Teknisk Ukeblad:

"Dagbladet skrev tirsdag at Stavanger tingrett har avsagt hemmelig kjennelse i spørsmålet om Lyse Tele må utlevere identiteten til en av sine bredbåndskunder som er mistenkt for piratkopiering av «Max Manus»-filmen.

Spørsmålet er svært betent, fordi piratjegerne som vil ha identiteten er advokater som jobber på vegne av en privat kunde.

Idag er det bare politiet som kan kreve dette, men i et byråkratvedtak av Post- og teletilsynet er det åpnet for at advokatfirmaet kan kreve utlevering på egenhånd, uten å måtte gå via politiet.

Reaksjonene på dette har kommet på nettet, men politikerne har så langt holdt seg unna på tross av mange oppfordringer om å komme på banen."

Hvor er politikerne? Er det slik vi vil ha det?

Dommeren som har avsagt kjennelsen har tatt ferie. Tilfeldig?

Se kommentarene på Twitter: #krevsvar

Saksøk drittungene!

2009-04-22T20:50:00.005+02:00

Piratene er de beste kundene - Kultur - Musikk - Aftenposten.no:

"Samtidig som Pirate Bay-gründerne på fredag ble dømt til både fengsel og svimlende bøter, slapp BI de oppsiktsvekkende resultatene fra sin store undersøkelse Kulturkonsum i krisetider.

Den viser at de som laster ned musikk gratis også er de desidert største forbrukerne av betalt nedlastet musikk fra iTunes og andre nettbutikker – og at de betaler for ti ganger så mange nedlastinger som de som aldri laster ned ulovlig.

–Det mest overraskende er at andelen betalt nedlasting er såpass høy, sier Audun Molde, høyskolelektor ved BI og fagansvarlig for musikk ved NISS.

–Resultatene kan tyde på at det lastes ned dobbelt så mye betalt musikk som gratis, og det er altså de som oppgir at de laster ned gratis som faktisk er de desidert største kundene av betalt musikk på nettet."

Når begynte de å lære bort idéen om at den beste måten å behandle innflytelsesrike kunder på er å kriminalisere dem og kalle dem for drittunger?

Se kommentarene på Twitter: #drittunge

Oppdatering: Se artikkelen i DinSide. Den havnet også på førstesiden på Digg! Even Sandvold Roland er på vei til Norsk Redaktørforenings landsmøte for å snakke om denne saken.

The Pirate Bay lost the court case

2009-04-17T12:56:00.002+02:00

Delte meninger : Pirate Bay tapte saken:

"Klokka 11 i dag avsa Stockholm Tingrät dom (PDF, 107 sider) i saken mot Fredrik Neij (30), Gottfrid Svartholm Warg (24), Peter Sunde Kolmisoppi (30), og Carl Lundström (49). Alle de fire ble dømt for medvirkning til brudd på den svenske opphavsrettlovgivingen. Dommen lyder på et års fengsel for alle de fire tiltalte, i tillegg skal de betale en erstatning til flere filmselskap. Til sammen er erstatningsbeløpene på 30 millioner svenske kroner."

All four were sentenced to one years imprisonment, and to pay damages amounting to 30 million SEK.

This is most probably not the final say in this matter, as it is highly likely that this ruling will be appealed.

InfoQ: Sun's Garbage First Collector Largely Eliminates Low Latency/High Throughput Tradeoff

2009-04-17T12:31:00.005+02:00

InfoQ: Sun's Garbage First Collector Largely Eliminates Low Latency/High Throughput Tradeoff:

"Sun's Garbage First garbage collector (hereafter referred to by its nickname G1) is the new low latency garbage collector planned to replace CMS in the Hotspot JVM. It is a server-style collector, targeted at multi-processor machines with large amounts of memory. There are two major differences between CMS and G1. The first is that G1 is a compacting collector. (...) As well as compacting, G1 offers more predictable garbage collection pauses than can be obtained with the CMS collector and allows users to set their desired pause targets."

It is part of JDK 6u14 early access, so you can try it out already now. Use the following options to enable it:

-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -Xloggc:gc.log

The last time I tested this jconsole and jvisualvm were not updated to handle this garbage collector, and the gc log format cannot directly be handled by current gc log parsers.

Delte meninger er lansert

2009-04-16T14:02:00.003+02:00

Delte meninger:

"Velkommen til den opne nettversjonen av boka mi “Delte meninger”! Dei tre delane av boka vil verte lagt ut til fri nedlasting og debatt fram mot lansering av papirversjonen den 19. mai. Artiklane har ulik tilnærming til korleis den sosiale delingskulturen på nettet kan påvirke demokratiet vårt. Eg håpar flest mogeleg vil gi sine synspunkt på desse temaene. – Heidi Grande Røys, Fornyingsminister"

Boken tar opp mange interessante aspekter ved deling av digitalt innhold, og skal bli interessant å følge med på.

De tre delene er:

Åpen, fri og tilgjengelig
Demokratisk delingskultur
Barn og unge i nettverksamfunnet

Furua faller

2009-04-15T19:53:00.004+02:00

Furua som har stått på tomten i femti år måtte ned. Trefellerne.no ble satt på saken:

A man of no principles

2009-04-02T21:37:00.003+02:00

The few of you that might have followed my blog for a while, might have read some of my rather bombastic statements on principles.

In 2005, I said I would never buy an iPod or music from iTunes. Wrong. I both have an iPod Nano (it was a gift though) and have bought a lot of music from iTunes, first buy only three months after this bold statement. In total I have spent more than 3,000 NOK on music from iTunes.

And I've bought a MacBook.

Last year I queued for an iPhone 3G, failing to obtain one and then claiming I would never get one. I ordered one from Telenor two weeks ago, and have spent 200 NOK on apps already.

So, in conclusion, I now have only one principle: to not stick to my principles, except this one.

Maybe.

Why adhering to established standards is always a Good Thing

2009-03-30T21:48:00.002+02:00

The memory in my work laptop failed the other day, rendering my Windows XP setup completely unbootable. It even killed the hard drive in the process. How? I don't know. But if you buy Dell, make sure you add the extended warranty.

As impractical as a complete crash and burn is, it was also an excellent opportunity to do an upgrade of the operating system. I opted to go for broke, and asked for Vista 64-bit.

Compatiblity wise this is a doubly complicated upgrade. Firstly the changed Vista security model, and secondly a 64-bit kernel.

Xerox has not released neither a Vista nor a 64-bit driver for my printer. Impediment? Not so.

My printer talks PostScript, an industry standard for computers and printers to exchange print ready data.

Any decent operating system is fully able to produce postscript data, and provides the necessary software and drivers out of the box. All that is needed is the PPD file describing the capabilities of the printer in question, like duplex printing, input trays, paper sizes and print quality.

So, as long as Xerox is willing to sell supplies and the printer doesn't burn, I'll always be able to upgrade. Driver or no driver.

As for Vista in general? Visual improvement, and many things work better than on XP. And being 64 bit I finally get to use the last 512MB of the memory...

Her lekker det

2009-03-21T10:46:00.002+01:00

Vanntrykket falt, og etter en del leting ble lekkasjen påvist. Nå er det bare å vente til snøen forsvinner før vi kan grave opp hagen..

The Positive Legacy of C++ and Java

2009-03-19T14:02:00.001+01:00

Java might have a limited lifespan as a language, but the JVM ensures that it will have a role even after that.

The Positive Legacy of C++ and Java:

"Java itself will diminish, just as C++ did, to be used in special cases (or perhaps just to support legacy code, since it doesn't have the same connection to hardware as C++ does). But the unintentional benefit, the true accidental brilliance of Java is that it has created a very smooth path for its own replacements, even if Java itself has reached the point where it can no longer evolve. All future languages should learn from this: either create a culture where you can be refactored (as Python and Ruby have done) or allow competitive species to thrive."

Java Specialist Master Course

2009-03-16T09:02:00.001+01:00

Last week I attended Dr. Heinz M. Kabutz' Java Specialist Master Course, which my employer held as an on-site course. These are my field notes from this training.

I strongly recommend this course if you're fluent in Java, but want to take it one step further or remind yourself of some of the advanced features or intricacies.

Day One

Multi-Threading

TreadGroup: Stay away! Not really usable.

A thread can also wake up without being notified, interrupted, or timing out, a so-called spurious wakeup. While this will rarely occur in practice, applications must guard against it by testing for the condition that should have caused the thread to be awakened, and continuing to wait if the condition is not satisfied. In other words, waits should always occur in loops, like this one:

     synchronized (obj) {
         while (<condition does not hold>)
             obj.wait(timeout);
         ... // Perform action appropriate to condition
     }

You cannot break out of a synchronized block in a safe way. If you do, the JVM will be in an unsafe state. The Java 5 locks are much safer.

Synchronizing on this is very dangerous: There's always someone who holds a reference to you (or you would be garbage collected). If they use you as a lock, they can effectively lock you out. They can also notify() you, confusing the internal work flow.

Locking on specific lock instances is always easier to get correct.

Java 5 locks must be wrapped in a try / finally block -- always.

Lock with WAITING state instead for BLOCKING state: lock.lockinterruptibly().

Deadlock: lock.tryLock(long,TimeUnit) can give false deadlock warnings in JConsole.

Thread priorities cannot be relied upon. They can cause starvation, and not all operating systems have a direct mapping between Java thread priorities and OS priorities.

volatile makes race conditions worse.

The value of a ThreadLocal is stored with the thread.

Usually not recommended to introduce fairness in Java. Starvation is usually not a problem, and the performance impact is significant. The OS scheduler does a good job at ensuring fairness.

Day Two

Java IO

Java Input and Output Streams have been designed using Decorator, where additional classes add functionality to underlying streams. Without this we would have extremely many permutations of these classes.

It is only necessary to close() the instance on top of this decorated chain. Closing the input or output stream on a socket will also close the underlying socket.

ObjectOutputStream has two caveats:

The caching for previously written objects is not bounded, and can result in an OutOfMemoryError
If you change an object after it has been written, this change will not be picked up and written.

ObjectInputStream also cache objects in an unbounded Hashtable, and can get an OutOfMemoryError.

SerialVersionUID is a hash of both fields and methods. Adding a method will also change it!

readResolve() and writeReplace() overrides the default serialization input or output form.

Externalizable classes are very easy to hack. The public read / write methods could be called from anyone to either extract its private contents or inject fabricated data into an instance.

Versioned serialization: Write a version number to the head of the stream, and switch on this version number when you read the object back.

writeObject() / readObject() can have roughly the same performance benefit as Externalizable.

Roedy Green: http://mindprod.com/jgloss/serialization.html

Java NIO

You read or write as much data from / to a channel as you can without blocking. So a thread busy transfering data would spend less time waiting for data. But there is a risk of spin looping if either the source or target is unable to accept data, but isn't closed.

Direct allocation and MappedByteBuffer uses the C heap, not the Java heap. Overallocating would therefore not cause a OutOfMemoryError, but virtual memory thrashing.

Java Memory

Premature tenuring can be a real performance problem. If the survivor spaces are slightly too small, and you hold on to your objects slightly too long, the survivor spaces get contended, and objects get tenured (moved to the old gen tenured space) too soon.

This can be fixed by tuning the survivor space sizes.

When tuning GC logs: Chop off the startup phase of the JVM -- it can confuse the steady state behavior.

Problems with JConsole:

It takes snapshots. A lot can happen between snapshots.
It cannot connect to a JVM which is having problems.

Tuning JVM

Don't confuse memory leaks with loitering objects (objects that just stick around longer than necessary).

Recommended heap size: 10% larger than steady state.

Day Three

References

What is the size of a Boolean (64-bit)

16 bytes for an object
1 byte for the boolean value. This is rounded up to the nearest 8 bytes
Total: 24 bytes

Finalizers: 80 bytes overhead per instance.

References makes object stay around longer than necessary, thereby leaking into old space.

Phantom references must be cleared, and must be constructed with a reference queue.

Object Pooling

Object Pooling is like ceramic mugs compared to disposable cups. It has a higher initial cost, and also a maintenance cost.

Generally used only in two cases: Thread pools and database connection pools.

Reflection API

Class.getDeclaredFields() uses a different SecurityManager permission than Class.getFields(). It is thus possible that the former is not allowed; only the latter.

Class.newInstance() has dangerous side effects. It can throw checked exceptions that aren't declared! The culprit is this:

// Run constructor

try {

return tmpConstructor.newInstance((Object[])null);

} catch (InvocationTargetException e) {

Unsafe.getUnsafe().throwException(e.getTargetException());

// Not reached

return null;

}

It is generally safer to use Class.getConstructor().newInstance() and catch the InvocationTargetException wrapping this checked exception.

Changing a final field through setAccessible(true) is currently possible, but this might change. Also, changing a final field might not be visible everywhere, since the Hotspot compiler can cache or inline the value of this final field.

java.util.Arrays.copyOfRange(T[], int, int) -- return a subarray. There's also a method where you can declare the return type.

java.lang.Class.getComponentType() returns the type of the array members, and null if the class is not an array.

Dynamic Proxies

The Method instance is linked to the declaring class (or interface), not to the implementing class or instance. This means this can be passed from the InvocationHandler to the actual implementation you are facading.

A generic factory for dynamic proxies:

import java.lang.reflect.InvocationHandler;

import java.lang.reflect.Proxy;

public class ProxyFactory {

public static <T> T proxy(Class<T> type, final T obj, final InvocationHandler handler,

Class<?>... otherInterfaces) {

return type.cast(Proxy.newProxyInstance(type.getClassLoader(),

merge(type, otherInterfaces), handler));

}

private static Class<?>[] merge(Class<?> first, Class<?>[] rest) {

Class<?>[] result = new Class<?>[rest.length + 1];

result[0] = first;

System.arraycopy(rest, 0, result, 1, rest.length);

return result;

}

Data Structures

CopyOnWriteArrayList is faster for iterating than the regular ArrayList, but much more expensive for modifications.

Sorting

Java uses merge sort because of overall good performance. Quick sort is often faster, but can be much slower in the worst cases, one being if the list is already sorted.

1.4+ HashMap bit masking was originally introduced for performance reasons, but over time the bit mixing has become more complex, thus closing this performance gap. The end result is that the hashing is completely unpredictable and much harder to understand.

equals() should cover all attributes of an object, but hashCode() doesn't need to. Find the attributes that most uniquely identify your instance.

Generics

<N extends Number> means that you now can call methods on Number using N: N n; n.doubleValue();

Generics sometimes produce very confusing compiler errors.

Other Structures

A LinkedHashMap normally orders elements by insertion order, but can also be set up to order elements by access order, placing most accessed elements first. This can be used to create a LRU cache.

Exceptions

RuntimeException's original definition was "exceptions thrown by the Runtime to protect itself from your code."

Checked exceptions were designated for errors where there would be some corrective action.

It is always a bad idea to use exceptions for flow control.

An uncaught exception handler can be used to restart a thread replacing the one that just died, continuing its work.

Day Four

Critical Errors Inside the JVM

What to do when the JVM dies:

Upgrade to a later version of the JRE
Examine the hs_err file, find out what was causing the problem, and change your code so that it doesn't provoke this behaviour

To analyze the hs_err file, take a look at http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/TSG-VM.pdf

Best Practices for Exceptions

Checked vs unchecked
Custom exceptions -- avoid when possible
Wrap (chain) low level exceptions
Don't swallow exceptions!
Cleanup after an exception -- in the finally block
Never abuse exceptions for flow control
Exceptions management strategy is a vital element of code reviews and post mortem analysis

Exceptions thrown in a finally block will hide the original exception.

Assertions

Ensure you avoid side-effects : assertions could always be disabled, effectively disabling this statement.

You could assert that you hold the lock as expected: assert Thread.currentThread().holdsLock(lock)

When using assertions, the entire system must be tested both with assertions enabled and disabled.

Useful to verify preconditions, postconditions, and class invariants.

Java Optimizations

Order of importance for performance improvements:

Design and architecture
Algorithm selection
Code implementation
System configuration
System infrastructure

Good design should win over performance. The Hotspot is very good at optimizing "normal" code, but could struggle if the code is cryptic. By coding in a complex way, you can make the performance worse than the clean and simple way.

Measure, don't guess.

Performance improvements <15% are typically not noticeable.

Specify your target, so you know when to stop.

Top five bottlenecks, find the easiest one, and fix this. Rinse and repeat.

Record everything. Use paper or some other means independent of the test setup.

Start by looking at hardware -- CPU, memory usage, I/O. Locate any bottlenecks on the hw level. Most performance problems appear as CPU bottlenecks.
Then look at JVM level: threads, memory usage, garbage collection.
Application level: Basically lock contention.
People level: study usage pattern, arrival rates.

Beware of Hotspot effects. Discard the first few results of a run. Look at the variance to determine whether the test is deterministic.

Work around GC interference: Larger initial heap size, forciby run System.gc() periodically. Sleep a while after the gc, at least 100ms, ideally 2 seconds.

Compare -client vs -server vs -Xint

The HotSpot compiler

Two levels of compilation: Just-In-Time compilation replaces bytecodes with machine instructions. The HotSpot compiler profiles the running application, directing the JIT on how to compile based on an execution profile.

-Xcomp compiles everything to native. This is generally slower, since it compiles too much, like methods invoked only once.

Bi-morphic: special case of polymorphism if only two implementations of a certian function is available. The method dispatch is simplified into an if / else statement.

The cost of a method call: 0 if only one implementation exists, little if two implementations exist, more if three or more implementations exist.

The thread must stop while the On Stack Replacement is performed.

Problem Areas

The two major problem areas are excessive looping and excessive object creation.

Special byte codes to handle the first four arguments and local variables. Long and double counts as two. Move most-used variables into these slots.

Object Pooling usable only for expensive resources (files, sockets, connections, threads)

You don't save memory by using substring(), since this new String instance would share the char[]. You'd need to create a new String.

Profiling Tools

hprof is bundled with the JVM. Can do cpu and heap profaling. Its output can be read by many tools.

Logging

Don't write your own logging framework....

Mentions Commons Logging, but fails to mention that Commons Logging is hated by many developers: http://www.qos.ch/logging/thinkAgain.jsp. Also it is claimed to have memory leaks: http://www.szegedi.org/articles/memleak.html

Also describes setting up logger statically per class, with some tricks involving a stack trace to get the class name. It is probably better to have a transient instance field.

SLF4J is also a wrapper API, enabling you to select logging implementation at runtime through the classpath. But SLF4J is more modern.

For Log4J he mentions only the properties format, and not the XML format, which is more structured.

Logging guards: use log.ifXXXEnabled() before your log statement. One neat trick is to create an aspect, inserting this code for you.

Varargs has a performance side effect. It will create an Object[] containing object instances of all parameters.

With asynchronous logging you would loose the most interesting messages if the JVM crashes.

Bruker du IE6? Slutt!

2009-02-19T15:43:00.005+01:00

Dersom du fortsatt bruker Internet Explorer 6: Nå er det på tide å slutte med det. Dersom du er forhindret fra å oppgradere Internet Explorer -- bytt til en annen nettleser!

tu.no - Microsoft støtter IE6-kampanjen - Teknisk Ukeblad:

"Det pågår en omfattende kampanje for å fåbrukere som fortsatt bruker Internet Explorer 6 til å oppgradere til en nyere versjon, eventuelt bytte. Den gamle nettleseren skaper store problemer for alle nettsteder fordi utviklerne må bruke mye tid på testing og spesialtilpasning til en nettleser som ble lansert helt tilbake i 2001. Microsoft erklærte i fjor at de nå satser for fullt på åpenhet og på å følge åpne standarder, og sier til Teknisk Ukeblad at de støtter den pågående IE6-kampanjen fullt ut"

For mer informasjon, se ie6.forteller.net

Her er noen av nettleserne du kan velge imellom:

An automatic way to figure out which commit broke the build

2009-01-23T11:17:00.002+01:00

Atlassian Developer Blog - git bisect...:

"JIRA's Clover Code Coverage build is scheduled to run once a day. Yesterday someone checked in something that broke 194 tests. The problem is there were 18 commits by 7 unique authors and none of the commits come with a helpful message like 'Warning, this commit is the one that will break the coverage build!' To try to narrow down which of the commits was the problem, I decided to use an automated bisection of commits. git comes with a built-in utility called bisect. This isn't unique to git. You can even do it with subversion via svn-bisect. The idea is that you tell it a good revision and a bad revision and then git helps you perform a binary search to narrow down exactly which commit broke things."

Change has come

2009-01-23T10:40:00.001+01:00

Front pages all over the world after Barack Obama's Inauguration Day in january 2009, viewed as a zoomable portait of Obama.

read more | digg story

Linnea og Elvira

2009-01-16T11:20:00.001+01:00

Aketur i masse snø

Skøyter på Veslevann

2009-01-16T11:12:00.002+01:00

Vi var ute og skøytet på Veslevann i romjulen. Isen var kjempeflott!