Jamie's shared items in Google Reader

Virtual method interception

2010-09-01T11:00:00Z

Delphi XE has a new type in Rtti.pas called TVirtualMethodInterceptor. It was originally designed for use in DataSnap authentication scenarios (though I don't think it's currently being used there), but of course, making it only work for that would have been quite a limitation.

What does it do? Essentially, it creates a derived metaclass dynamically at runtime that overrides every virtual method in the ancestor, by creating a new virtual method table and populating it with stubs that intercepts calls and arguments. When the metaclass reference for any instance of the "ancestor" is replaced with this new metaclass, the user can then intercept virtual function calls, change arguments on the fly, change the return value, intercept and suppress exceptions or raise new exceptions, or entirely replace calling the underlying method. In concept, it's somewhat similar to dynamic proxies from .NET and Java. It's like being able to derive from a class at runtime, override methods (but not add new instance fields), and then change the runtime type of an instance to this new derived class.

Why would you want to do this? Two obvious purposes spring to mind: testing and remoting. Mock objects have been in vogue in the testing space in other languages for some time. By intercepting method calls, one may more easily verify that a particular subsystem is calling all the right methods, with the correct arguments, in the expected order; similarly, the subsystem can proceed with the return values from these method calls, without necessarily having to hit the database, the network, etc. for what should be a unit test. Remoting on the basis of method calls is somewhat less useful, especially when an unreliable and latency-prone network gets into the stack, but that's not the only usage point. The virtual method interceptor logic was originally implemented to be used as part of DataSnap authentication, so that a call that comes in from the network can still be checked as its code flow spreads throughout the graph of objects.

Anyhow, here's a simple example to get started:

uses SysUtils, Rtti;
{$apptype console}
type
  TFoo = class
    // Frob doubles x and returns the new x + 10
    function Frob(var x: Integer): Integer; virtual;
  end;

function TFoo.Frob(var x: Integer): Integer;
begin
  x := x * 2;
  Result := x + 10;
end;

procedure WorkWithFoo(Foo: TFoo);
var
  a, b: Integer;
begin
  a := 10;
  Writeln('  before: a = ', a);
  try
    b := Foo.Frob(a);
    Writeln('  Result = ', b);
    Writeln('  after:  a = ', a);
  except
    on e: Exception do
      Writeln('  Exception: ', e.ClassName);
  end;
end;

procedure P;
var
  foo: TFoo;
  vmi: TVirtualMethodInterceptor;
begin
  vmi := nil;
  foo := TFoo.Create;
  try
    Writeln('Before hackery:');
    WorkWithFoo(foo);
    
    vmi := TVirtualMethodInterceptor.Create(foo.ClassType);
    
    vmi.OnBefore := procedure(Instance: TObject; Method: TRttiMethod;
      const Args: TArray<TValue>; out DoInvoke: Boolean; out Result: TValue)
    var
      i: Integer;
    begin
      Write('[before] Calling ', Method.Name, ' with args: ');
      for i := 0 to Length(Args) - 1 do
        Write(Args[i].ToString, ' ');
      Writeln;
    end;
    
    // Change foo's metaclass pointer to our new dynamically derived
    // and intercepted descendant
    vmi.Proxify(foo);
    
    Writeln('After interception:');
    WorkWithFoo(foo);
  finally
    foo.Free;
    vmi.Free;
  end;
end;

begin
  P;
end.

Here's what it outputs:

Before hackery:
  before: a = 10
  Result = 30
  after:  a = 20
After interception:
  before: a = 10
[before] Calling Frob with args: 10 
  Result = 30
  after:  a = 20
[before] Calling BeforeDestruction with args: 
[before] Calling FreeInstance with args:

You'll notice that it intercepts all the virtual methods, including those called during destruction, not just the one I declared. (The destructor itself is not included.)

We can get more ambitious with what it does. I can change the implementation entirely, and skip calling the underlying (i.e. inherited) method body:

procedure P;
var
  foo: TFoo;
  vmi: TVirtualMethodInterceptor;
  ctx: TRttiContext;
  m: TRttiMethod;
begin
  vmi := nil;
  foo := TFoo.Create;
  try
    Writeln('Before hackery:');
    WorkWithFoo(foo);
    
    vmi := TVirtualMethodInterceptor.Create(foo.ClassType);
    
    m := ctx.GetType(TFoo).GetMethod('Frob');
    vmi.OnBefore := procedure(Instance: TObject; Method: TRttiMethod;
      const Args: TArray; out DoInvoke: Boolean; out Result: TValue)
    begin
      if Method = m then
      begin
        DoInvoke := False;
        Result := 42;
        Args[0] := -Args[0].AsInteger;
      end;
    end;

Here, I inhibit the invocation and hard-code the result to 42, while negating the first argument. The proof is in the output:

Before hackery:
  before: a = 10
  Result = 30
  after:  a = 20
After interception:
  before: a = 10
  Result = 42
  after:  a = -10

I could have inhibited the call by raising an exception instead:

    vmi.OnBefore := procedure(Instance: TObject; Method: TRttiMethod;
      const Args: TArray; out DoInvoke: Boolean; out Result: TValue)
    begin
      if Method = m then
        raise Exception.Create('Aborting');
    end;

And output:

Before hackery:
  before: a = 10
  Result = 30
  after:  a = 20
After interception:
  before: a = 10
  Exception: Exception

It's not limited to interception before the logically inherited call, but also interception after the call, again with the opportunity to fiddle with arguments and return value:

    m := ctx.GetType(TFoo).GetMethod('Frob');
    vmi.OnAfter := procedure(Instance: TObject; Method: TRttiMethod;
      const Args: TArray; var Result: TValue)
    begin
      if Method = m then
        Result := Result.AsInteger + 1000000;
    end;

And output:

Before hackery:
  before: a = 10
  Result = 30
  after:  a = 20
After interception:
  before: a = 10
  Result = 1000030
  after:  a = 20

And if the inherited implementation raises an exception, that can even be squashed:

function TFoo.Frob(var x: Integer): Integer;
begin
  raise Exception.Create('Abort');
end;

// ...
    m := ctx.GetType(TFoo).GetMethod('Frob');
    vmi.OnException := procedure(Instance: TObject; Method: TRttiMethod;
      const Args: TArray; out RaiseException: Boolean;
      TheException: Exception; out Result: TValue)
    begin
      if Method = m then
      begin
        RaiseException := False;
        Args[0] := Args[0].AsInteger * 2;
        Result := Args[0].AsInteger + 10;
      end;
    end;

Output:

Before hackery:
  before: a = 10
  Exception: Exception
After interception:
  before: a = 10
  Result = 30
  after:  a = 20

One thing the TVirtualMethodInterceptor class doesn't have, however, is a way to unhook (unproxify) the object. If the object is never unhooked, it's important that the object doesn't outlive the interceptor, because the interceptor needs to allocate executable memory in order to create the little stubs with which it redirects method calls to the events. Fortunately, it's pretty trivial to do:

    PPointer(foo)^ := vmi.OriginalClass;

Another point: the class inheritance chain is changed by the hooking process. This can be shown easily:

//...
    Writeln('After interception:');
    WorkWithFoo(foo);
    
    Writeln('Inheritance chain while intercepted:');
    cls := foo.ClassType;
    while cls <> nil do
    begin
      Writeln(Format('  %s (%p)', [cls.ClassName, Pointer(cls)]));
      cls := cls.ClassParent;
    end;
    
    PPointer(foo)^ := vmi.OriginalClass;
    
    Writeln('After unhooking:');
    WorkWithFoo(foo);
    
    Writeln('Inheritance chain after unhooking:');
    cls := foo.ClassType;
    while cls <> nil do
    begin
      Writeln(Format('  %s (%p)', [cls.ClassName, Pointer(cls)]));
      cls := cls.ClassParent;
    end;
// ...

And output:

Before hackery:
  before: a = 10
  Exception: Exception
After interception:
  before: a = 10
  Result = 30
  after:  a = 20
Inheritance chain while intercepted:
  TFoo (01F34DA8)
  TFoo (0048BD84)
  TObject (004014F0)
After unhooking:
  before: a = 10
  Exception: Exception
Inheritance chain after unhooking:
  TFoo (0048BD84)
  TObject (004014F0)

The feature is primarily an infrastructure piece for advanced libraries, but hopefully you can see that it's not too difficult to get into.

Getting Started with Delphi XE's Regular Expressions

2010-09-02T14:00:00Z

Delphi XE includes Regular Expression support, something that has been requested many times over the years. In this blog post I'll show some basic usage of regular expressions in delphi.

More...

How to tune Delphi XE’s start up

2010-08-31T17:41:29Z

When I started Delphi XE today the first time, I recognized a 6 second delay when it shows “IDE Packages loaded” in the splash screen. On the second start it still shows “IDE Packages loaded” for 5 seconds. Because I wanted to know what the IDE does in those 5 seconds with 100% CPU load (25% on my 4 CPU machine), I debugged the IDE with an empty IDEFixPack for Delphi XE project (sorry no bug fixes yet ). I pressed the Pause-Button when it reached the delay.

The function that I saw in the call stack was the TStrings.IndexOfName method that was called by TStrings.SetValue. The string list contained about 3000 items. All of them were files in $(BDS)\source. That means that something enumerates all source file names and put them in the string list in the format “filename=path”. Returning from TStrings.SetValue showed me the IDE plugin that is responsible for the startup delay: Beyond Compare. The IDE plugin seems to cache file locations. I don’t know why it needs the location of all source files, but not on my computer. I have removed BC from the “Experts” registry key and now my IDE starts much faster.

BTW: Don’t forget to disable the Component-Toolbar package if you want to switch faster from Code to Design view.

Sneak preview about DORM, The Delphi ORM

2010-09-01T08:17:04Z

My yesterday post about this busy time, have raised some interest about DORM, the Delphi ORM.

So, also if I still haven’t released any files, wish to expose some internals about DORM.

DORM is an implementation of the DataMapper design pattern written having Hibernate in mind.

It’s completely unit tested and have the following features:

External file mapping. (JSON format)
Persistence ignorance (every TObject can be persisted)
Support for One-One and One-Many relations (still no many-many)
Support for LazyLoading (you can enable§/disable lazyloading by file or by code by per-basis needs)
Support for IdentityMap
Support for custom “finder” (you can still use complex SQL if you want)
Complete support for CRUD
Transactions
Built in logging system to log *EVERY* sql or action performed by the framework
Opened to multiple data access strategies (interfaced based, not inheritance based) for use with different database (now I’ve developed the firebird one using DBX)
Caching for RTTI (the TSession object have a single TRttiContext holding ALL metadata)

Code is still under heavely development.

Those are 2 test-method to show the use of DORM:

TPerson = class(TObject)
…
property Phones: TdormCollection…. //implements IList
end;
TPhone = classs(TObject)
…
end;
//and now the unit tests
procedure TTestDORMHasMany.Setup;
begin
Session := TSession.Create;
Session.Configure(TStreamReader.Create('dorm.conf'));
end;
procedure TTestDORMHasMany.TearDown;
begin
Session.Free;
end;
procedure TTestDORMHasMany.TestHasManyLazyLoad;
var
p: TPerson;
t: TPhone;
guid: string;
begin
p := TPerson.NewPersona; //static method. Return a fully populated TPerson object
try
t := TPhone.Create;
p.Phones.Add(t);
Session.Save(p);
guid := p.guid; //GUIDs, or other PK types, are generated automagically by DORM. Obviously there is a specific class loaded to do this specified in the dorm.conf file)
finally
Session.Commit;
end;
Session.StartTransaction;
// Test with lazy load ON
Session.SetLazyLoadFor(TypeInfo(TPerson), 'Phones', true);
p := Session.Load(TypeInfo(TPerson), guid) as TPerson;
try
CheckEquals(0, p.Phones.Count);
finally
Session.Commit;
end;
Session.StartTransaction;
// Test with lazy load OFF
Session.SetLazyLoadFor(TypeInfo(TPerson), 'Phones', false);
p := Session.Load(TypeInfo(TPerson), guid) as TPerson; // Without commit, AV becouse IdentityMap doesn't work properly
try
CheckEquals(1, p.Phones.Count); // Child objects are loaded
finally
Session.Commit;
end;
end;
procedure TTestDORMHasMany.TestLoadHasMany;
var
list: IList;
t, t1: TPhone;
p: TPerson;
guid: string;
begin
p := TPerson.NewPersona; //static method. Return a fully populated TPerson object
try
t := TPhone.Create;
t.Numero := '555-7765123';
t.Kind := 'Casa';
p.Phones.Add(t);
t1 := TPhone.Create;
t1.Number := '555-7765123';
t1.Kind := 'Casa';
p.Phones.Add(t1);
Session.Save(p); // save Person and Phones
guid := p.guid;
finally
Session.Commit;
end;
Session.StartTransaction;
p := Session.Load(TypeInfo(TPerson), guid) as TPerson;
try
CheckEquals(2, p.Phones.Count);
finally
Session.Commit;
end;
end;

Mapping, contained in a file called “dorm.conf”, is similar to the following:

{
"persistence": {
"database_adapter": "dorm.adapter.Firebird.TFirebirdPersistStrategy",
"database_connection_string":"127.0.0.1:C:\\MyProjects\\DORM\\experiments\\dorm.fdb",
"username": "sysdba",
"password":"masterkey"
},
"config": {
"package": "dorm.bo.Person",
"logger_class_name": "dorm.loggers.FileLog.TdormFileLog"
},
"mapping":
{
"TPerson":
{
"table": "people",
"id": {"name":"guid", "field":"guid", "field_type":"string", "size": 100, "default_value": ""},
"fields":[
{"name":"firstname", "field":"first_name", "field_type":"string", "size": 100},
{"name":"lastname", "field":"last_name", "field_type":"string", "size": 100},
{"name":"age", "field":"age", "field_type":"integer"},
{"name":"borndate", "field":"born_date", "field_type":"date"}
],
"has_many":[{
"name": "Phones",
"class_name":"TPhone",
"child_field_name":"guid_person",
"lazy_load": false
}],
"has_one": {
"name": "car",
"class_name":"TCar",
"child_field_name":"guid_person"
}
},
"TPhone":
{
"table": "phones",
"id": {"name":"guid", "field":"guid", "field_type":"string", "size": 100, "default_value": ""},
"fields":[
{"name":"number", "field":"number", "field_type":"string", "size": 100},
{"name":"kind", "field":"kind", "field_type":"string", "size": 100},
{"name":"guid_person", "field":"guid_person", "field_type":"string", "size": 100}
]
}
}
}

The PODO (Plain Old Delphi Objects) can be binded to the VCL controls with a set of MediatingView (Model-GUI-Mediator Pattern) with an Observer mechanism to mantain things in synch.

Any comments? Someone interested?

Vigenere Cipher Algorithm - Delphi Implementation (Comments by Alan Lloyd)

2010-08-30T09:36:28Z

in Vigenere Cipher :: A long time Delphi developer, Alan Lloyd had a few comments on the recently published Vigenere Cipher - Delphi Implementation - Fancy Delphi Application Contest Entry #49.

Alan suggests:

While we have loads of computation power & memory available today, I think as a professional it is better to code quicker than slower, unless it makes code unclear.

In the posted Vigenere code . . .

String memory allocation/re-allocation is slow so avoid ConCat(), String + String (even though its quicker than ConCat), and Delete(). Better to SetLength for strings and use index accessing for characters.
Use 'for' loops rather than 'while' loops.
Reduce the code for every character. Check for bEncrypt before entering repeated similar code in a 'for' loop for each character action.
IMO sTable should be initialised before use by := '' or SetLength(). AIUI variables in functions are not zeroed, and you might be using garbage.
Crypt by re-referencing the key as given rather that using multiple keys up to the source length.

Here's the Vigenere crypto algorithm as suggested by Allan:


function Vigenere(Src, Key : string; Encrypt : boolean) : string;
const
  OrdMinChar : integer = Ord('A');
  OrdMaxChar : integer = Ord('Z');
  IncludeChars : set of char = ['A'..'Z'];
var
  CharRangeCount, i, j, KeyLen, KeyInc, SrcOrd, CryptOrd : integer;
  SrcA : string;
begin
  CharRangeCount := OrdMaxChar - OrdMinChar + 1;
  KeyLen := Length(Key);
  SetLength(SrcA, Length(Src));
  If Encrypt then 
  begin
    // transfer only included characters to SrcA for encryption
    j := 1;
    for i := 1 to Length(Src) do
      begin
      if (Src[i] in IncludeChars) then 
      begin
        SrcA[j] := Src[i];
        inc(j);
      end;
    end;
    SetLength(SrcA, j - 1);
  end;
  SetLength(Result, Length(SrcA));
  if Encrypt then
    begin
    // Encrypt to Result
    for i := 1 to Length(SrcA) do 
    begin
      SrcOrd := Ord(Src[i]) - OrdMinChar;
      KeyInc := Ord(Key[((i - 1 ) mod KeyLen)+ 1]) - OrdMinChar;
      CryptOrd := ((SrcOrd + KeyInc) mod CharRangeCount) + OrdMinChar;
      Result[i] := Char(CryptOrd);
    end;
  end;
  else
  begin
    // Decrypt to Result
    for i := 1 to Length(SrcA) do 
    begin
      SrcOrd := Ord(Src[i]) - OrdMinChar;
      KeyInc := Ord(Key[((i - 1 ) mod KeyLen)+ 1]) - OrdMinChar;
      CryptOrd := ((SrcOrd - KeyInc + CharRangeCount) mod CharRangeCount) + OrdMinChar;
      // KeyInc may be larger than SrcOrd
      Result[i] := Char(CryptOrd);
    end;
  end;
end;

Vigenere Cipher Algorithm - Delphi Implementation (Comments by Alan Lloyd) originally appeared on About.com Delphi Programming on Monday, August 30th, 2010 at 09:36:28.

Permalink | Comment | Email this

Delphi XE Beta Blogging: New in the RTL

2010-08-20T10:25:10Z

In this third "beta blog" I'll just provide a (partial) list of new RTL classes and functions. I've picked just half a dozen of new things in the RTL I find very useful, but the complete list accounts for several dozens of significant changes.

In case you don't know, "Delphi beta blogs" are authorized blog posts by independent bloggers covering features of a beta version of Delphi. This means I was given a specific permission to post about this topic and I'm discussing a feature of a pre-release version. Notice also that the official RAD Studio XE Preview page is at http://www.embarcadero.com/rad-studio-xe-preview . Since this is a pre-release, there is no guarantee it will work the same in the final version (or will actually be in the product).

Here is a partial list of new classes in Delphi XE RTL (compared to Delphi 2010):

TBinaryReader and TBinaryWriter somewhat counterparts of the TTextReader and TTextWriter introduced in Delphi 2009
TTimeZone is a little helper for managing time zone and computing time differences
TInterlocked, TSpinWait, TSpinLock and many other new core synchronization classes help you push your multi-threaded code further and make it more cross-platform (as these classes have a native implementation, based on only core OS support). The TThread class has been also extended with direct execution of anonymous methods inside a temporary thread and several other features.
TRegEx is the core record implementing native regular expressions support. This is a big step forward, despite many developers already used third party libraries for regular expressions.

Other notable changes, beside new classes, include:

The addition of a DefaultEncoding to TStrings, which lets you pick for Unicode format when a list of strings to a text file. There is also support for converting string lists to and from arrays of strings.
The TPath record in IOUtils has a new GetHomePath class method, which on Windows returns the applications data folder, “AppData\Roaming”). The method calls a corresponding new function in SysUtils. There are actually a lot of improvements in file management, for example with the inclusion on TSymLink, and abstraction... all due to the coming cross platform support.
Finally, I'd like to mention one of the new functions, SplitStrings. This is quite fast, compared with other techniques I used in the past for splitting strings. Another small nice addition.

Again, this is a very partial and limited list of features, nothing revolutionary but many welcome additions.

Teaser – MonoDroid with Oxygene

2010-08-19T22:02:47Z

Now that the MonoDroid preview is out, I thought I’d experiment a little with Oxygene (aka Delphi Prism), so I ported the “Button” sample to Oxygene. Here’s a taste of what I came up with:

method Activity1.OnCreate(bundle : Bundle);
begin
    inherited OnCreate(bundle);
 
    // Create your application here
    var button := new Button(self);
 
    button.Text := 'Oxygene Says: Click Me!';
    button.Click += method begin
      button.Text := string.Format('{0} clicks!', count);
      inc(count);
    end;
 
    SetContentView(button);
end;

As you can see, there’s not a lot of difference from the C# version.

Obviously, this won’t be in the build that we RTM’ed last week, and will soon be available from our friends at Embarcadero, but it will work with said release. So if you’re a Delphi Prism customer, or just have a fondness for Pascal syntax, keep an eye out here – more info will be available soon, including templates & other support files.

method Activity1.OnCreate(bundle : Bundle);
begin
inherited OnCreate(bundle);

// Create your application here
var button := new Button(self);

button.Text := ‘Oxygene Says: Click Me!’;
button.Click += method begin
button.Text := string.Format (‘{0} clicks!’, count);
inc(count);
end;

SetContentView (button);
end;

The Visitor Pattern – Part 4

2010-08-17T13:17:43Z

The interface approach introduced in Part 2 of this series and extended in Part 3 did cover most of our needs. But there is nothing that can’t be done better with the right tools.

I’ve been using this approach for a couple of years and it made my life a lot easier. You need an export as DXF? No problem! Displaying in OpenGL? I’ll write a new visitor!

Writing the additional interfaces was easy done with the help of a customized code template, so that has never been a pain. The only thing that bothered me over time was that I could only apply the Visitor Pattern to classes I have control of. I just wasn’t able to visit, say, all the components of a TForm in a descent fashion (i.e. not queueing if component is something then <typecast>).

While digging through the new features of Delphi 2010 I discovered a solution which is as clean, lean, flexible and elegant as it could be. A little RTTI and some decent naming convention did the trick. Here is the code:

unit uVisitor;

interface

type
  TVisitor = class
  public
    procedure Visit(Instance: TObject); virtual;
  end;

implementation

uses
  Rtti;

procedure TVisitor.Visit(Instance: TObject);
var
  context: TRttiContext;
  currentClass: TClass;
  params: TArray
;
  paramType: TRttiType;
  selfMethod: TRttiMethod;
  S: string;
begin
  context := TRttiContext.Create;
  currentClass := Instance.ClassType;
  repeat
    S := currentClass.ClassName;
    Delete(S, 1, 1); // remove "T"
    for selfMethod in context.GetType(self.ClassType).GetMethods('Visit' + S) do begin
      params := selfMethod.GetParameters;
      if (Length(params) = 1) then begin
        paramType := params[0].ParamType;
        if ParamType.IsInstance and (ParamType.AsInstance.MetaclassType = currentClass)then begin
          selfMethod.Invoke(Self, [Instance]);
          Exit;
        end;
      end;
    end;
    currentClass := currentClass.ClassParent;
  until currentClass = nil;
end;

end.

And here is a visitor capable of handling some VCL controls on a form together with an example of its usage:

type
  TMyVisitor = class(TVisitor)
  public
    procedure VisitButton(Instance: TButton);
    procedure VisitLabel(Instance: TLabel);
    procedure VisitEdit(Instance: TEdit);
    procedure VisitMemo(Instance: TMemo);
  end;

procedure TMyVisitor.VisitButton(Instance: TButton);
begin
  Instance.Caption := 'a button';
end;

procedure TMyVisitor.VisitEdit(Instance: TEdit);
begin
  Instance.Text := 'an edit';
end;

procedure TMyVisitor.VisitLabel(Instance: TLabel);
begin
  Instance.Caption := 'a label';
end;

procedure TMyVisitor.VisitMemo(Instance: TMemo);
begin
  Instance.Lines.Clear;
  Instance.Lines.Add('a memo');
end;

procedure TForm40.Button1Click(Sender: TObject);
var
  I: Integer;
  visitor: TMyVisitor;
begin
  visitor := TMyVisitor.Create;
  try
    for I := 0 to ComponentCount - 1 do
      visitor.Visit(Components[I]);
  finally
    visitor.Free;
  end;
end;

If you need to visit some class, just introduce a new method with a suitable name and signature and you are done. No new interfaces, no fiddling around with classes. It just works right out of the box.

I’m still visualizing some possible scenarios where this approach can be useful, too. Stay tuned.

The Visitor Pattern – Part 3

2010-08-17T08:13:37Z

In Part 2 of this series we learned about a more flexible implementation of the Visitor Pattern than the traditional approach. But we can do better.

When we have a usual class inheritance within our shape or whatever classes, it might be desirable to handle groups of classes with a common ancestor all the same. For now we have to implement each interface individually. A slight change does the trick. We introduce a visitor interface for TAbstractShape, make TAbstractShape.Accept not abstract. Then we change the Accept implementations of the derived shapes to first check the visitor implementing the interface and call inherited otherwise.

unit uShapeBase;

interface

uses
  Classes, SysUtils;

type
  ENotSupported = class(Exception);

type
  TAbstractShape = class
  public
    procedure Accept(Visitor: IInterface); virtual;
  end;

  IAbstractShapeVisitor = interface
    ['{7ED85A22-564A-4F23-AF32-37D29FD3C623}']
    procedure VisitAbstractShape(Instance: TAbstractShape);
  end;

  { we avoid ref-counting here }
  TShapeVisitor = class(TInterfacedPersistent)
  public
    procedure Visit(Instance: TAbstractShape);
  end;

type
  TShapeRectangle = class(TAbstractShape)
  private
    FX0: Double;
    FX1: Double;
    FY0: Double;
    FY1: Double;
  public
    procedure Accept(Visitor: IInterface); override;
    property X0: Double read FX0 write FX0;
    property X1: Double read FX1 write FX1;
    property Y0: Double read FY0 write FY0;
    property Y1: Double read FY1 write FY1;
  end;

  IShapeRectangleVisitor = interface
    ['{6F521E71-5DB8-4ECE-B453-60F9A3B8BE22}']
    procedure VisitShapeRectangle(Instance: TShapeRectangle);
  end;

type
  TShapeCircle = class(TAbstractShape)
  private
    FCenterX: Double;
    FCenterY: Double;
    FRadius: Double;
  public
    procedure Accept(Visitor: IInterface); override;
    property CenterX: Double read FCenterX write FCenterX;
    property CenterY: Double read FCenterY write FCenterY;
    property Radius: Double read FRadius write FRadius;
  end;

  IShapeCircleVisitor = interface
    ['{27D7DCD3-952A-4879-8667-E86023612BFB}']
    procedure VisitShapeCircle(Instance: TShapeCircle);
  end;

implementation

procedure TShapeVisitor.Visit(Instance: TAbstractShape);
begin
  Instance.Accept(Self);
end;

procedure TShapeCircle.Accept(Visitor: TShapeVisitor);
var
  myVisitor: IShapeCircleVisitor;
begin
  if Supports(Visitor, IShapeCircleVisitor, myVisitor) then
    myVisitor.VisitShapeCircle(Self)
  else
    inherited;
end;

procedure TShapeRectangle.Accept(Visitor: IInterface);
var
  myVisitor: IShapeRectangleVisitor;
begin
  if Supports(Visitor, IShapeRectangleVisitor, myVisitor) then
    myVisitor.VisitShapeRectangle(Self)
  else
    inherited;
end;

procedure TAbstractShape.Accept(Visitor: IInterface);
var
  myVisitor: IAbstractShapeVisitor;
begin
  if Supports(Visitor, IAbstractShapeVisitor, myVisitor) then
    myVisitor.VisitAbstractShape(Self)
  else
    raise ENotSupported.Create('Visitor doesn''t support ' + ClassName);
end;

end.

The Visitor Pattern – Part 2

2010-08-16T21:33:28Z

In Part 1 of this series we learned about the benefits of the Visitor Pattern and saw a basic approach according to the GoF description.

As clean and elegant the visitor pattern might look in the first place, it nonetheless has its disadvantages. Say, we want to add a new shape TShapeTriangle. We have to do it inside the same unit the other shapes are declared. As the shapes reference the visitor and the visitor references the shapes we have to put all shape classes together with the visitor into the same unit. Although this might work in some smaller scenarios, it definitely doesn’t in most of the real ones. Probably this is one of the reasons why the Visitor Pattern is not as famous as it should be.

Can we find a solution? Yes, we can!

Lets examine why there is such a close coupling between all the shapes and the visitor: The shapes use the visitor’s method corresponding to their own class and the visitor has to know about all the shapes it has to handle. How can we break this up?

The answer are interfaces! Instead of declaring a visitor capable of handling every defined shape, we declare an interface for each shape that only handles that specific shape.

unit uShapeBase;

interface

uses
  Classes;

type
  TAbstractShape = class
  public
    procedure Accept(Visitor: IInterface); virtual; abstract;
  end;

  { we avoid ref-counting here }
  TShapeVisitor = class(TInterfacedPersistent)
  public
    procedure Visit(Instance: TAbstractShape);
  end;

type
  TShapeRectangle = class(TAbstractShape)
  private
    FX0: Double;
    FX1: Double;
    FY0: Double;
    FY1: Double;
  public
    procedure Accept(Visitor: IInterface); override;
    property X0: Double read FX0 write FX0;
    property X1: Double read FX1 write FX1;
    property Y0: Double read FY0 write FY0;
    property Y1: Double read FY1 write FY1;
  end;

  IShapeRectangleVisitor = interface
    ['{6F521E71-5DB8-4ECE-B453-60F9A3B8BE22}']
    procedure VisitShapeRectangle(Instance: TShapeRectangle);
  end;

type
  TShapeCircle = class(TAbstractShape)
  private
    FCenterX: Double;
    FCenterY: Double;
    FRadius: Double;
  public
    procedure Accept(Visitor: IInterface); override;
    property CenterX: Double read FCenterX write FCenterX;
    property CenterY: Double read FCenterY write FCenterY;
    property Radius: Double read FRadius write FRadius;
  end;

  IShapeCircleVisitor = interface
    ['{27D7DCD3-952A-4879-8667-E86023612BFB}']
    procedure VisitShapeCircle(Instance: TShapeCircle);
  end;

implementation

uses
  SysUtils;

procedure TShapeVisitor.Visit(Instance: TAbstractShape);
begin
  Instance.Accept(Self);
end;

procedure TShapeCircle.Accept(Visitor: TShapeVisitor);
begin
  (Visitor as IShapeCircleVisitor).VisitShapeCircle(Self);
end;

procedure TShapeRectangle.Accept(Visitor: TShapeVisitor);
begin
  (Visitor as IShapeRectangleVisitor).VisitShapeRectangle(Self);
end;

end.

The Visitor Pattern – Part 1

2010-08-16T14:12:01Z

Some of you might have already heard of the Visitor Pattern, especially if you have read the GoF book Design Patterns, but do you really use it? Do you use it often?

The common answer to this question is No. Once I was even confrontated with the question “Does anybody use it in real code at all?”.

Well, I do – and here I’m going to explain the why and the how. Actually there are different hows and at the end I will introduce a real elegant how…

Some basics: the Visitor Pattern is used to decouple an operation from the class it operates on and thus allows to create new operations without changing the class.

Let me give you an example: Assume we have a shape class providing the necessary properties to define the shape completely. To draw the shape onto a TCanvas we can introduce a Draw method with a Canvas as parameter. Easy done and easily extended to other shapes. We normally would introduce a TAbstractShape with an abstract Draw method ond override this in the derived classes.

Next task will be to store the shapes somehow into an inifile and read it back. The classical approach would be to introduce two methods LoadFromFile and SaveToFile each with a TCustomInifile parameter probably accompanied by a section name. I’m sure we’ve all done that before.

Requirements change: the drawing should be done with OpenGL and instead of inifiles we use XML now – but the other behaviour has to remain for compatibility reasons. I know, there are OpenGL wrappers that emulate a TCanvas and so are XML wrappers emulating a TCustomInifile, but honestly – such an approach is just crap, isn’t it? We have to introduce some overloaded methods with an OpenGL context for drawing and an XMLDocument for reading and writing to get this done properly. Oh, I just heard someone mentioning Direct3D…

We are actually following a pattern here: introduce an abstract method in the abstract ancestor and implement the specific behaviour in the derived classes. As a side effect we have to use a bunch of units in the abstract class that normally wouldn’t belong there. This can be really awkward when you need the classes, but don’t use the additional functionality, like drawing to OpenGL or writing to XML. Although a perfect valid object-oriented approach, it doesn’t contribute to the modularity of the application.

Here the Visitor Pattern steps in. Its goal is to move the operations from the class (in our case TAbstractShape and descendants) to different visitor classes. So there will be one for drawing to a TCanvas, one for the OpenGL output, two classes for reading and writing inifiles and two for reading and writing XML files.

Improving Your Audio: Hardware Edition

2010-08-07T18:22:00Z

[ASIDE: Community Server (our current blogging engine here at CodeBetter.com) is swallowing the HTML5 <audio> tags in this post along with the Flash Player fallback. To hear HTML5 audio in your browser, jump to here with a browser that supports HTML5 audio such as FireFox 3.5+, Chrome, Safari, Opera, or IE9. I've successfully tested it in FireFox 4.0b2 and Chrome 6. I've also successfully tested it in IE8 where it falls back to the Flash Player. Sorry for the hassle.]

[ASIDE Part Deux: Brendan has tweaked the Community Server settings to allow CodeBetter to serve up <audio> tags. So the audio below should work now. Thanks, Brendan!]

Over the years, I’ve done a lot of audio work between podcasts, screencasts, and webcasts. So I know a thing or two about computer audio. I don’t claim to be an expert like my friends Carl Franklin or Richard Campbell, but I’ve done enough to be able to offer some helpful tips. We’re going to start with the hardware.

The quality of your computer audio can only be as good as the raw captured product. Use a bad microphone and no amount of software cleanup is going to magically produce good audio. You might be wondering how much difference the hardware can make? I’ve recorded the same audio track using four (4) different microphones on the same computer. (I didn’t record them simultaneously as multi-track recording is notoriously difficult, but I did say the same phrase into each microphone on after the other.) Let’s start with the LifeChat ZX-6000.

No audio playback capabilities, please download the audio below
Download Audio: Closed Format: MP3 Open Format: OGG

My voice sounds like I’m on a telephone. The sound is hollow and lacks depth. If we plot a frequency analysis using Audacity, we can easily see the problems.

OK, maybe not easily if you’re not familiar with audio. Let me explain some basic ideas and then you should be able to see the problems.

Normal human hearing discerns frequencies between 20 Hz and 20 kHz. The standard tuning note for musicians is the 440 Hz, which is an A above middle C on the piano. The lowest note on the piano (A0) is 27.5 Hz and the highest note (C8) is 4186 Hz. (I’m using the example of a piano since many people, even non-musicians, have at least played with a piano at one time or another.) Lower frequencies correspond to lower notes and higher frequencies to higher notes. The frequencies mentioned are the fundamental frequencies. When you play an A4 on the piano (or any other instrument including the human voice), the major frequency is 440 Hz, but there are many harmonics or overtones that occur. These harmonics give a colour and depth to the sound. This is one of the reasons why different instruments sound vastly different when playing the same note – the harmonics produced by each instrument are quite different. This is how we perceive different ranges of audio frequencies. (Taken from Wikipedia Audio Frequency.)

Frequency (Hz)	Octave	Description
16 to 32	1st	The human threshold of feeling, and the lowest pedal notes of a pipe organ.
32 to 512	2nd to 5th	Rhythm frequencies, where the lower and upper bass notes lie.
512 to 2048	6th to 7th	Defines human speech intelligibility, gives a horn-like or tinny quality to sound.
2048 to 8192	8th to 9th	Gives presence to speech, where labial and fricative sounds lie.
8192 to 16384	10th	Brilliance, the sounds of bells and the ringing of cymbals. In speech, the sound of the letter "S" (8000-11000 Hz)

Note how 2048 to 8192 Hz gives a presence to speech, whereas 8192 to 16384 give a brilliance. Without these frequencies present, speech will sound hollow.

With this in mind, let’s take another look at the frequency spectrum from the LifeChat ZX-6000. We see virtually no frequencies above 4000 Hz, which is making my voice sound hollow. Old analog telephones transmitted 200 Hz to 3000 Hz, which is why it sounds like I’m talking on an old phone. You’ll also note that the lower frequencies (below 400 Hz) are attenuated (e.g. not as pronounced), which is why the sound is lacking some of the bass timbre of my voice.

Let’s try a different microphone and see how it performs… Next up the LifeChat LX-3000.

No audio playback capabilities, please download the audio below
Download Audio: Closed Format: MP3 Open Format: OGG

The audio quality is vastly improved. Let’s take a look at the frequency spectrum.

You can visibly see the difference. We have frequency response all the way up to 20 kHz with the majority of the response in the lower frequencies, which is expected due to the timbre of my voice. The lower frequencies are also not as attenuated. The quality of the sound is much warmer and vibrant with the LX-3000 than the ZX-6000.

As our last point of comparison, let’s listen to a semi-pro microphone – the one I use for my recording work – the audio-technica AT2020.

No audio playback capabilities, please download the audio below
Download Audio: Closed Format: MP3 Open Format: OGG

The differences are subtler this time, but still noticeable. The audio has more depth and presence than with the LX-3000. Let’s take a look at the frequency spectrum.

Notice the better bass response below 400 Hz giving a truer rendering of my low voice. We also have better harmonics in the 10 to 20 kHz range, providing a more life-like sound. We can also take a look at the frequency response of the microphone, which can be found on the manufacturer’s website here.

Note the flat response curve across the entire range of frequencies. This means that the microphone records all frequencies with equal efficiency, which results in little distortion of the raw sound. For comparison, I would expect the response curve for the ZX-6000 to drop to virtually zero above 4 kHz and show attenuation below 400 Hz. You want a flat response curve for your microphone as it will not colour or distort the recorded audio.

I should note that both the LifeChat LX-3000 and ZX-6000 have hardware noise cancellation. (Noise cancellation will remove an annoying background hum originating from fans, pumps, and other sources of low background noise. It can’t do anything to clean up dogs barking, children screaming, or other sudden noises that disrupt your recording sessions.) Applying software noise cancellation on either of these microphones has little additional benefit. The audio-technica AT2020 does not have hardware noise cancellation and benefits from applying software noise cancellation. Assuming you are working in a quiet environment the audio quality of the AT2020 without noise cancellation is still better than the LX-3000 and far superior with noise cancellation. Software noise cancellation usually involves little more than selecting a checkbox in programs like TechSmith Camtasia Studio or similar recording packages. You can perform noise removal using Audacity too, though it’s a bit more work as you have to manually select a quiet region with just the background noise that you want to subtract.

The LX-3000 is a great microphone for conference calls and gaming. It is a good, though not great, microphone for recording podcasts/screencasts/webcasts. It is inexpensive ($30 to $50), easy to use, and can be bought at most computer stores. If you’re just getting started, this is a good microphone to buy.

If you’re looking to take your audio to the next level, the audio-technica AT2020 is a great semi-pro microphone that you can pick up at reasonable cost. You’ll have to go to an audio specialty store as you won’t find these in your regular computer stores. I purchased mine at Long & McQuade, which is a chain of well-respected Canadian musical instrument stores. Now what is a reasonable cost? You’ll need more than just a microphone. You’ll also need a pre-amp to power the microphone as semi-pro and pro microphones don’t have high enough output to jack directly into your computer microphone port. You’ll need a pop filter (which prevents “p” and “t” sounds from making “popping” sounds in your audio), a mic stand, an XLR cable for mic to preamp, and a 1/4” to 1/4” male cable for preamp to computer (or 1/4” to 1/8” male if you are using a normal mic-in on your computer).

Component	Price*
audio-technica AT2020	$120
ART TubeMP Tube Mic Preamp	$49
Pop Filter	$20
Mic Stand	$20
2 Cables (XLR & 1/4”-1/4”)	$20
Total	$229

* Prices are in Canadian dollars.

You can get the same microphone (AT2020) with a USB option, but at a higher cost of $170, which is basically the cost of the preamp and cables. The TubeMP preamp has an actual vacuum tube that gives a warmth to the sound that is hard to achieve otherwise. Given the similar costs, I would personally err on the side of using a tube preamp over USB.

You might want to invest in a decent sound card, such as a Creative Labs X-Fi Platinum or similar card, which has better audio recording qualities than the audio-in that comes on your motherboard. It’s hard to find the X-Fi cards anymore. So you’ll have to look around to find a good quality audio card, but expect to spend $100 to $200 on the audio card alone. Remember your audio is going to be no better than the weakest link in the chain.

Is $229 of the audio-technica AT2020 worth the improved audio over the $30 to $50 LifeChat LX-3000? That’s up to you to decide.

Creating multiple objects using try-finally

2010-07-06T14:37:43Z

One of the most used object-oriented patterns is probably this one:

foo := TFoo.create;
try
  //do something with foo
finally
  foo.free;
end;

It basically means, that if you create an instance of some class and if there is no “owner” who takes care of the new instance’s life cycle, then it is your responsibility to destroy the instance again, as soon as you are done with it.

If you don’t care about freeing the objects you created, then you will get “memory leaks”, i.e. you app will use (possibly a lot) more memory as it should.

The technique above is clear and simple, but what if you have more than one object that is needed in a routine? Line them up?

foo := TFoo.create;
bar := TBar.create;
try
  //do something with foo and bar
finally
  foo.free;
  bar.free;
end;

Looks good, and I see this pattern frequently – but hold on! What happens if there is an exception when TBar is created? Right, foo won’t be destroyed in that case. So maybe handle it like this then:

foo := TFoo.create;
try
  bar := TBar.create;
  try
    //do something with foo and bar
  finally
    bar.free;
  end;
finally
  foo.free;
end;

In theory this would be the correct way, and there are a lot of examples that work like that. But look at the code – right: looks really ugly, esp. if we extend that to even more objects being created. Apart from readability there is also some performance hit: every try-frame needs some extra CPU cycles, which may sum up if your routine is time critical already.

My suggestion is this one:

foo := nil;
bar := nil;
try
  foo := TFoo.create;
  bar := TBar.create;
  //do something with foo and bar
finally
  bar.free;
  foo.free;
end;

Now lets discuss some questions about this pattern:

Didn’t we learn, that calls to “Create” have to be made outside the try-finally block?
No, not really. In fact it doesn’t matter where you create an instance of a class. The first example (which is mentioned in many articles) only suggests this, because it’s easy to do so with just one object.

If TBar.create fails, what happens with its bar.free counter part? Wouldn’t it actually be executed on some not “really created” object?
No, in fact bar.free is special method, which can safely be called even on variables pointing to NIL. Free checks if self<>nil and only then the actual destructor is called.

Why did you initialize fo and bar with NIL?
Because otherwise they might have random values. Esp. if they are declared as local variables. And we need NIL initialized variables so that “free” does not call random code in the case of an exception in one of the create calls.

If TBar.create fails with an exception, wouldn’t be there the chance that some address value would have been written to bar already – so that bar.free would be executed on some half initialized instance?
No, if TBar.Create fails with an exception, then the class function “create” would (like any other function) return NO value at all. bar would keep its original value that is.

Get Shareaholic

Delphi 2009/2010 Post-Build events

2010-06-30T18:51:13Z

Have you ever wondered if it is possible to execute the “Post-Build events” only when the project was actually updated/compiled? With Delphi 2009/2010 you can do that easily by changing the “Execute when” combobox in the “Post-Build events” edit dialog to “Target is out-of-date”. At least you thought that would do the trick. Instead it disables the Post-Build event what means that it is never executed.
I did some research on this because I needed this functionality. Without it the “Execute when” = “Always” setting tells msbuild to execute the post-build script even if the compiler didn’t do anything. And that causes msbuild to build targets that don’t need to be built.

But what is the problem with “Target is out-of-date”. Let’s have a look into the msbuild script that executes the Post-Build events. The “PostBuildEvent” target in $(BDS)\bin\CodeGear.Common.Targets is declared as:

<Target Name="PostBuildEvent"
        Condition=" '$(PostBuildEvent)'!=''  And '$(KibitzCompile)'==''
        and ( '$(RunPostBuildEvent)'=='Always' or '$(RunPostBuildEvent)'=='' or '$(PreOutputTimeStamp)'!='$(PostOutputTimeStamp)' )"
        DependsOnTargets="$(PostBuildEventDependsOn)">
    <Exec Command="$(PostBuildEvent)"  IgnoreExitCode="$(PostBuildEventIgnoreExitCode)" WorkingDirectory="$(OutDir)"/>
</Target>

Now let’s see what msbuild with “/v:diag” tells us about this target.

'$(RunPostBuildEvent)'=='Always' or '$(RunPostBuildEvent)'=='' or
  '$(PreOutputTimeStamp)'!='$(PostOutputTimeStamp)'

is evaluated to

'OutOfDate'=='Always' or 'OutOfDate'=='' or ''!=''

The first two conditions are ok and mean that we have to look at the third condition for our out-of-date check. But what is with the third condition? Why is it an empty string comparison that is always false? And where do PreOutputTimeStamp and PostOutputTimeStamp come from? They are declared some lines about the “PostBuildEvent” target.

<Target Name="_PreOutputTimeStamp">
    <CreateItem Include="%(OutputFile.ModifiedTime)">
        <Output TaskParameter="Include" PropertyName="PreOutputTimeStamp"/>
    </CreateItem>
</Target>
 
<Target Name="_PostOutputTimeStamp">
    <CreateItem Include="%(OutputFile.ModifiedTime)">
        <Output TaskParameter="Include" PropertyName="PostOutputTimeStamp"/>
    </CreateItem>
</Target>

So the PreOutputTimeStamp is the modified time of the output file (EXE/DLL/BPL) before it was compiled and the PostOutputTimeStamp is the modified time of the same file after msbuild has done its work. But why are they empty?
The item “OutputFile” is not declared or set anywhere. Neither in CodeGear.Common.Targets nor in CodeGear.Delphi.Targets nor in MyProject.dproj. And that is why both msbuild properties are evaluated to empty strings.

With this knowledge I was now able to fix the this bug by adding the missing “OutputFile” item to the CodeGear.Delphi.Targets file.

 <ItemGroup>
    <_MSBuildProjectFullPath Include="$(MSBuildProjectFullPath)"/>
    <_DependencyCheckOutputName Include="$(DCC_DependencyCheckOutputName)"/>
+   <OutputFile Include="$(DCC_DependencyCheckOutputName)"/>
 </ItemGroup>

After this change the “Target is out-of-date” combobox item in the “Post-Build events” dialog started to work as expected.

One issue solved. But that wasn’t all that I needed for my build script. In order to reduce the file size of our massive EXE files we strip the PE format’s relocation section from the EXEs and we create and attach *.jdbg files to our EXEs. But both post-build events aren’t necessary when we debug our projects in the IDE, especially not the relocation section removal. In the IDE we want a fast edit-compile-debug turn-around. So I had to find a way to only execute those post-build events when the Release-Configuration was active.
Unfortunately Delphi doesn’t support this feature. But after browsing through the CodeGear.*.Targets files and reading msbuild /v:diag log files, I had the solution right there sitting in front of me. The $(OUTPUTPATH) from the post-build events are replaced by msbuild properties when msbuild processes the build script. So I had the idea of just trying to use other properties from msbuild in addition to the macros that are listed in the Post-Build events edit dialog. And with $(Config) I had the active configuration right at hand.

if "$(Config)" == "Release" (
  stripreloc.exe ...
  linkmapfile.exe ...
)

And with the active configuration being set to “Debug”, msbuild executes this as:

if "Debug" == "Release" (
  stripreloc.exe ...
  linkmapfile.exe ...
)

Now that I have solved the build script issues, I can turn to all the other “things” that I have to solve, think about and develop before all my co-workers can start working with the new Delphi IDE.

Those “things” are things like keeping the components as part of the project to make it possible to update to a new component version without breaking older project versions. And I’m not speaking of using the “-r” IDE command line option. That’s what we used in the older Delphi version. I’m speaking of opening a project from the 1.0 branch, closing it and opening the same project from the 7.0 branch that requires newer versions of the components. Furthermore the components are in the project’s svn folder, so they have the same revision number. (Our old svn “layout” was to have a separate svn repository for the components) And if you svn-update and somebody has changed the components, the IDE automatically compiles them and loads them when you open the project. And today this IDE plugin passed it’s final tests.

DeHL 0.8.3

2010-06-17T21:27:59Z

I will be brief as usual — version 0.8.3 of DeHL is out. The downloads can be found on this page and changelog on this page. This release “fixes” some of the things I wanted fixed for a long time, so it seemed this is the perfect moment for this to happen. A new unit is introduced — DeHL.Tuples — which brings seven generic Tuple<…> types. I have also finished moving away from Integer and Cardinal to NativeInt and NativeUInt through all DeHL.

Breaking changes are:

DeHL.Converter was renamed to DeHL.Conversion. This name change was done mostly to reflect the nature of the unit. It does not contain one simple class anymore. Now there is a fully featured conversion system.
TBinaryTree<T> (in DeHL.Collections.BinaryTree) is no more. It was written way in the beginnings of the project and was buggy, incomplete and utterly useless.
And the most visible and breaking of all changes is the removal of TKeyValuePair<TKey, TValue> (in DeHL.KeyValuePair). It was replaced with KVPair<TKey, TValue> (in DeHL.Tuples). The easiet way to get over this change is to find and replace all TKeyValuePair instances with KVPair and all DeHL.KeyValuePair uses with DeHL.Tuples.

Now, obviously an example using Tuples:

uses
  SysUtils, DeHL.Tuples;

{ If you are lazy and do not wish to declare a new record type
  to be used as result. Use Tuple<..> do to that. }
function GiveMeSomeData(): Tuple<String, Integer, Integer>;
begin
  { ... Do some processing ... }
  Result := Tuple.Create('Some data', 100, -99);
end;

var
  LResult: Tuple<String, Integer, Integer>;
begin
  { Obtain the result }
  LResult := GiveMeSomeData();

  { And write the results to the console }
  with LResult do
    WriteLn('Something was done with result: ', Value1, ', ',
      Value2, ', ', Value3, '!');
end.

The new conversion engine handles most of the possible conversions and also allows registering custom ones:

uses
  SysUtils,
  DeHL.Conversion,
  DeHL.Collections.List;

type
  { Declare some type that cannot be converted into integer directly.
    We're making up an "int". }
  TRecordInt = packed record
    FValueAsString: string;
    FSign: Boolean;

    constructor Create(const AInt: Integer);
  end;

{ TRecordInt }

constructor TRecordInt.Create(const AInt: Integer);
begin
  { Decompose an int }
  FValueAsString := IntToStr(Abs(AInt));
  FSign := (AInt < 0);
end;

var
  LInputList: TList<TRecordInt>;
  S: String;
  I: Integer;
begin
  { Create a list of TRecordInt }
  LInputList := TList<TRecordInt>.Create();

  { Fill it with some random values (positive and negative) }
  for I := 0 to 10 do
    LInputList.Add(TRecordInt.Create(Random(MaxInt) - (MaxInt div 2)));

  { Now comes the interesting part ... register a custom converter
    from TRecordInt to Integer }
  TConverter<TRecordInt, Integer>.Method :=
    function(const AIn: TRecordInt; out AOut: Integer): Boolean
    begin
      { Convert the TRecordInt back to an integer }
      Result := TryStrToInt(AIn.FValueAsString, AOut);
      if Result and AIn.FSign then
        AOut := -AOut;
    end;

  { Now print the values to the console. Convert them from TRecordInt to
    Integer then to String }
  for S in LInputList.Op.Cast<Integer>.Op.Cast<String> do
    WriteLn(S);
end.

TConverter is also smart enough to figure out that “type MyInt = type Integer” is actually equivalent to Integer. If there is no explicit custom conversion method registered for it the converter for the standard type will be selected is possible. In the worst case, when TConverter cannot convert directly between the given types, it falls back to Variant conversion (using TType.TryConvertToVariant and TType.TryConvertFromVariant) which all types registered with DeHL’s type system, if possible, should implement.

Well, that’s all for today,
Have Fun!

The Performance of Arrays

2010-06-11T21:01:26Z

Stop me if you’ve heard this one, but here’s some information about how arrays perform, and a neat trick you can do to (possibly) get some performance back.

Some background

In .NET, arrays of reference types are covariant in their element type, but not safely. Eric, as always, has a post that goes into this more deeply if you want to refresh your memory. The upshot is that if you have a Derived[], you can convert it to a Base[] and use it that way. For instance,

class Base { }
class Derived : Base { }
 
class Program
{
    static void Main()
    {
        Derived[] derivedArray = new Derived[10];
        // This is the covariant conversion
        Base[] baseArray = derivedArray;
 
        for (int i = 0; i < baseArray.Length; ++i)
        {
            // Putting a Derived into our Base[] is ordinary polymorphism
            baseArray[i] = new Derived();
        }
    }
}

Allowing this conversion from Derived[] to Base[] is not safe because now the compiler can’t help you know if your types match up when you set array elements. In the preceding example, I can put the Derived in the array, but imagine if I had another unrelated derived class, OtherDerived. Putting those things in the array must fail, since the actual array can contain only Deriveds, and OtherDeriveds are not Deriveds. In Eric’s perhaps more folksy example, “you can’t put a Turtle into an array of Giraffes.”

class Base { }
class Derived : Base { }
class OtherDerived : Base { }
 
class Program
{
    static void Main()
    {
        Derived[] derivedArray = new Derived[10];
        // This is the covariant conversion
        Base[] baseArray = derivedArray;
 
        for (int i = 0; i < baseArray.Length; ++i)
        {
            // Putting a OtherDerived into our Base[] is ordinary
            // polymorphism. However, this is going to cause a runtime
            // exception because the actual array won't be able to
            // store an OtherDerived.
            baseArray[i] = new OtherDerived();
            // Unhandled Exception: System.ArrayTypeMismatchException: 
            //   Attempted to access an element as a type incompatible
            // with the array.
        }
    }
}

Well now to the point I want to address: where did that exception come from? It came from the runtime, which was in a unique position to know what the real type of the array object was, and the real type of the element. It needed to determine if the assignment was allowable and throw if not. Now if you think about this for a minute, you can see that it’s going to have to perform that check for every assignment to an array element, right? The compiler just can’t know if any assignment to a covariant array is ever going to work, and the runtime is always going to have to be there to manage the potential for failure. Can we turn that off? Does it make sense to? What does the check cost you?

A workaround

Well, if it’s doing that check for every assignment to an array, that’s probably going to cost you something. Let’s see if we can get rid of the check to determine what the cost is. Remember I said that arrays are covariant only for reference types? We’re going to use that information. If we have an array of value types, the runtime isn’t going to have to perform that check anymore, since it knows that the IL the compiler emitted wasn’t going to allow assignment of anything but that value type to that array.

So if I want to have an array of reference types, but get the runtime to treat it like an array of value types, what have I got to do? Just wrap the reference type in a value type. There you go. That’s the trick. Let’s see what that does.

I’m going to create a simple value type that merely holds a reference. I am even going to make it generic. And to make this as dead-simple as I can, that’s all I’ll do. You could imagine a potentially easier-to-use version of this code that has implicit conversions and nice constructors, but let’s not do any of that.

struct Reference<T> where T : class
{
    public T Value;
}

Ok, so, wow, what is that thing? It’s no bigger than the reference we started with. And if I want to make an array of them in order to get rid of covariance, I can. And here’s how I’d use it.

class Base { }
class Derived : Base { }
 
class Program
{
    static void Main()
    {
        Reference<Base>[] baseArray = new Reference<Base>[10];
 
        for (int i = 0; i < baseArray.Length; ++i)
        {
            baseArray[i] = new Reference<Base> { Value = new Derived() };
        }
    }
}

Now I have an array assignment that doesn’t need the runtime to perform a check anymore, and the reason is because it’s impossible for there to be anything in “baseArray” except for an array of the exact type I said. The struct-ness of Reference<T> prevents the possibility of anyone having performed a covariant conversion as existed in our previous examples. So I got rid of that check, right? But I added the initialization of a value type. Did I win or lose? Let’s do some timing to figure it out.

The experiment

Here’s my code. I’m going to time two different things: First, I’ll put a lot of Deriveds into a Base[], and then I’ll put a lot of Reference<Base>s that hold Deriveds into a Reference<Base>. I’m using huge arrays just because I want to prove to myself that the jitter isn’t going to lift some of these operations out of the loop. And I’m producing Release binaries that will run on the 32-bit framework on my 64-bit machine. I’ll time each example a few times to be sure. Here’s the code.

class Base { }
class Derived : Base { }
 
struct Reference<T> where T : class
{
    public T Value;
}
 
class Program
{
    const int Count = 1 << 27;
 
    static void Main()
    {
        Stopwatch watch = new Stopwatch();
 
        TestReferenceArray(watch);
        TestReferenceArray(watch);
        TestReferenceArray(watch);
 
        TestValueArray(watch);
        TestValueArray(watch);
        TestValueArray(watch);
    }
 
    static Base[] TestReferenceArray(Stopwatch watch)
    {
        Base[] array = new Base[Count];
        var element = new Derived();
 
        watch.Restart();
        for (int i = 0; i < Count; ++i)
        {
            array[i] = element;
        }
        watch.Stop();
        Console.WriteLine(watch.ElapsedMilliseconds);
 
        return array;
    }
 
    static Reference<Base>[] TestValueArray(Stopwatch watch)
    {
        Reference<Base>[] array = new Reference<Base>[Count];
        var element = new Derived();
 
        watch.Restart();
        for (int i = 0; i < Count; ++i)
        {
            array[i] = new Reference<Base> { Value = element };
        }
        watch.Stop();
        Console.WriteLine(watch.ElapsedMilliseconds);
 
        return array;
    }
}

Have a seat. Here’s what it prints out.

2507
2471
2474
595
607
593

Wow! That’s a huge time difference, and a big win. It’s FOUR TIME FASTER! I wouldn’t blame you for thinking that you have to go to your code base right now and eliminate arrays of reference types from it; but hold on a second. That’s not the whole story.

Issue #1: Are the types exact?

In the example that I timed, I was putting a Derived into arrays with element type Base. The types weren’t the same, and therefore the check that the runtime performed had to be complicated by doing something like walking an inheritance hierarchy. But what if I were putting Bases into the Base array. What do the times look like then? (I won’t replicate the code here; just replace “new Derived()” with “new Base()” everywhere).

805
775
771
593
593
599

Oh, that’s much more reasonable. The lesson here is that for my scenario, if I’m not making use of polymorphism, I have much less to be worried about.

Issue #2: Which runtime exactly?

I said that I was doing tests on the 32-bit framework on my 64-bit machine. I ensured that by building my binaries “x86,” which is the default for VS2010. What if I instead build them “Any CPU” so that I can run the test on the 64-bit framework? Under those conditions, I am swapping out the whole stack that my tests run on for a whole different runtime. Let’s see what the numbers are then.

2102
2057
2055
789
784
784

Ok, still faster, but not by as big a factor. What about for the exact type case?

861
848
846
792
790
817

Nearly the same! So my example seems to be not as big a deal for 64-bit processes. I am not sure why. There is another oddity here that I can’t explain: when I build Debug binaries, the Reference<T> trick makes my example about three times SLOWER. I have no idea why.

Issue #3: My example is not your scenario

I tried to make the thing I was timing as simple as possible. But I’m using relatively a lot of array assignments. Do you do that much? Probably not. I’m not doing any I/O and not waiting on any user interaction, are you? I also have a giant array and I’m clearly blowing out the CPU cache in my for loops. Does your code do that? I’m assigning arrays in a tight loop with a known upper bound. Are you? Does the jitter lift the bounds checking out of the loop, and how would that check compare with the assignment compatibility check?

My point is that as with any performance timing, there are a lot of variables and a lot of considerations. If you want to make your program perform better using this trick, you need to try it and then time it yourself. I am not saying that .NET arrays are slower than they need to be or unsuited for your application. But it’s a fun experiment to try; how many milliseconds faster can you make your scenario, and is it worth the cost? Were you using the covariance of arrays in your code, and therefore ineligible to even consider this trick?

Why?

Why does C# have these unsafe covariant arrays to begin with? I was not here at the time, but I can confidently say that the CLR needed them because they needed to support languages that have array covariance. One example of such a language (in fact, probably the interesting example) is Java. C# has them both because the CLR has them and because it was a clear goal to that if users were porting existing code to C#, they were met with as few hurdles as possible. Also, there would have been complications with supporting both covariant arrays and invariant arrays in the same runtime; imagine the confusion users and compiler-writers would face trying to interop between the two. The language design notes from 1999 bear out these claims.

Hundred Thousand Lines in a Unit

2010-06-14T10:12:48Z

At a recent event, a person mentioned he had problems with the debugger with a very large unit (as you can see in the questions).

Although I'm not sure why someone would ever want to have over 64,000 lines of Delphi code in a single unit (a single file, for the non-Delphi developers out there), I was puzzled about the request. So I wrote a very large unit like that. Well, that's not entirely true, I wrote a program I used to write a large unit, with 20,000 functions for adding numbers (all with the same signature). Than I added the unit to the same program, called a function, and palced a breakpoint. In fact the developer asking the question was mentioning debugging problems. Here is the large unit in the debugger, stopped at line 81,732 (out of 120,011 lines):

I have not tried to compile and run the same application in Delphi 2007 (the version against which the problems was reported), but 2010 seems to handle it. Yes, opening the unit is not terribly fast, as the IDE has to parse it. As an experiment I tried to create and compile a unit with 200,000 similar functions, but the compiler started using the system memory and CPU at a very high rate and the compiled lines count after a while was moving versy slowly... so eventually I gave up compiling a unit with over a million lines of source code.

My original question remains, though. Why would one developer have such a large unit? Which are the largest units (in terms of lines of source code) you have?

Delphi Help Insight XSL

2010-06-07T16:54:42Z

Over two past articles I focused on the role of Help Insight and discussed possible customizations. While researching for this last article, I tried changing the XSL file in the ObjRepos (object repository) folder of Delphi, with no effect. I had missed the fact the actual "live" files are in the language subfolder, in my case "en" for English. The right file to edit, on my computer, is:

C:\Program Files\Embarcadero\RAD Studio\7.0\ObjRepos\en\HelpInsight.xsl

Here are some ideas to change the predefined XSL for Help Insight and get an extended output. In terms of actual information, the only that's not displayed is the full path of the unit containing the symbol definition. Notice also that the size of the output window depends on the content of the top div called main.

My extensions to the XSL are mostly at the end of the file, displayed below:

The effect of the templates with mode xmltohtml and attrotohml can be used to debug the data behind Help Insight, as they display a tree of the XML nodes in HTML, as for the TButton symbol below:

For local symbol you get much more information, including the full unit file name:

This is helpful to let you figure out the information behind Help Insight as you add comments. Finally the last template of the XSL file above makes a full copy of any comment portion enclosed in the HTML tag, letting you format the output freely, including any HTML tag including links (better to open them in a new page rather than in the Help Insight page). Below you can see the HTML in the comment, including a link to this blog, and the output below:

This is just the starting point of customizations you can do, and you can also change the CSS and the background image. The Help Insight pane opens up the ability to create fully customized viewers, driven by your source code and the comments you add to it. Demo XSL file download is here.

May 2010: Fun with the Windows 7 Taskbar

2010-05-29T15:42:22Z

In this article, I will show how we can manipulate the Windows 7 taskbar, and also how to solve an issue when running an application with the "run as Administrator" option while needing the taskbar to be able to send messages to it.

Google Chrome Frame in your Delphi application « progdigy.com

2010-05-24T16:19:44Z