http://www.slideshare.net/rhymes/erlang-and-python

I’ll be back on the 6th of May.

PyCon Tre is coming, in fact I’ll be in Florence from the 7th of May to the 10th to attend it. This year I’m going to speak for the first time. I submitted a talk about Erlang and Python but then I pretty soon realized that it should have been about Erlang and the outside world because there’s no direct connection between the two

Anyway, I know that few people of the non-italian community will be there (Guido, Fredrik, Raymond, Anna, and many more) so I’ll see you all in Florence on the 8th!

It will be held in Florence as last year, on the 8th, 9th and 10th of May!

We’ve received several talk proposal, we let the community vote to create a chart and we created the schedule including regular talks, sponsored ones and talks by invited guests. Talking about the invited guests: this year we have the pleasure to be joined by Guido Van Rossum (with two keynotes and a QA session), Alex Martelli (talking about software abstractions), Raymond Hettinger (AI with Python and descriptors), Fredrik Lundh (with the first talk ever about Unladen Swallow!) and Antonio Cangiano (talking about how to get rich with Python !

Remember that as last year all the talks will have simultaneous translation between english and italian and viceversa so you’ll be able to attend the conference and understand everything.

If you are interested in sponsoring, donating or any kind of help for the conference please contact me or write to the board of organizers at info at pycon.it

Hope to see you there!

I’m thrilled about this because it also means that PyCon Italy is really growing. Last year we had around 300 people and Alex Martelli, Brad Fitzpatrick and Raymond Hettinger among others. This year Guido agreed to come so I hope I’ll see at least all the Italian Python community there!

See you on the 8th, 9th and 10th of May, 2009.

Recently I’ve had the opportunity to read Michael Foord’s upcoming book: IronPython in Action and let me say that I found it extremely interesting, and I suggest it to everybody needing (or wanting) to work with .NET from a dynamic perspective or to .NET developers interested in the world out there.

The book covers a lot of ground, sometimes deeply, sometimes just in the surface but it seems to talk about everything there’s to know to make that ground solid enough to build something lasting.

From the Python introduction to IronPython extension, from IronPython embedding to XML, Visual Studio, testing, metaprogramming, Windows Presentation Foundation (the .NET new UI), system administration, ASP.NET, databases, Silverlight and much more.

Let’s go through the chapters themselves.

Chapter 1: A New Language for .NET

This chapter is an introduction to IronPython, .NET and our beloved interactive interpreter. One of the things I enjoyed discovering throughout the book is the extent of the language support in the .NET framework. Who knew Microsoft could have promoted it as a full citizen!

You can use IronPython in Visual Studio, ASP.NET, Silverlight, XNA (Microsoft’s new gaming platform), the Robotics Kit and many more places. Since .NET libraries are all written with the same bytecode (called IL) pretty much every door is open.

IP’s license is opensource and approved by the OSI, the development is done in public so there’s no need to worry about the future of the language.

There are two versions of IP: 1 and 2. I’d stick with version 2 because support Python 2.5 features. Recently there’s been some new development about the next version and support for Python 2.6. The second version also ships the Dynamic Language Runtime (DLR), a specific runtime dedicated to dynamic languages. It seems Jim Hugunin was definitely right after all.

IP 2 should work in Mono too (by default requires .NET 2) but I didn’t succeed to do so and soon gave up anyway because I planned to use my Windows virtual machine on Mac OS X.

The book as a nice series of reasons why a Python developer should try IP:

• IronPython has no GIL for multithreaded applications
• Extending IP through C# is easier than extending Python through C
• Applications can be sandboxed
• You can leverage two Window’s GUI toolkits for desktop apps: Windows Forms and WPF
• .NET has a huge number of libraries available
• Silverlight allows the developer to code inside the browser in IP

IP is a full citizen of the platform, and the interactive interpreter can be used to explore at runtime the whole framework while developing. That’s a huge advantage over C# developers

Chapter 2: Introduction to Python

The second chapter is an overview of Python. I posted a detailed summary in Italian on my old .NET blog just to help .NET developers (at least those who read the blogs of the Italian .NET user group) to understand what’s Python all about.

Chapter 3: .NET Objects & IronPython

How to use .NET from IP? That’s what this chapter is all about. After the intro there’s a whole Windows Forms example showing the important parts. Notice that creating a desktop application take something like this:

    >>> import clr
    >>> clr.AddReference('System.Windows.Forms')
    >>> from System.Windows.Forms import Application, Form
    >>> form = Form()
    >>> form.Text = 'Hello World' # set the title
    >>> Application.Run(form) # main loop

You can either choose to write the interface manually or use Visual Studio to design it and load it from the application.

Note that .NET objects are inheritable from IP, so that’s a go!

I remember the old days of C# where I had to declare each and every type up front and it feels nice to be able to use the same libraries and classes without all the verbosity.

Chapter 4: Writing An Application and Design Patterns with IronPython

Finally with the fourth chapter I can get my hands dirty and start delving into the process of writing a real application in IP.

The application is a Windows Forms based multiple document editor, MultiDoc. You can find all the book source code online.

The chapter starts talking about Python protocols, duck typing, magic methods to help understand what’s going on underneath and how to leverage Python features for the application at hand.

The choice is perfect because in a few line of code Michael Foord can throw in a lot of features, functionalities and patterns (like MVC, command, observer and others).

It’s shown that there’s some overlap sometimes between Python features and .NET’s and Michael Foord wisely suggests to choose considering needs, portability and pure taste.

I personally found amusing the fact that you can add event handlers for GUI applications like this:

  button.Click += lambda sender, event: command.execute()

Chapter 5: First Class Functions in Action with XML

MultiDoc in this chapter gains XML supports. There’s an introduction to first class functions, higher order functions and decorator. Functions are used as event handlers.

The command pattern is used to implement some functionalities for the application.

Chapter 6: Properties, Dialogs and Visual Studio

The chapter starts with an introduction to Python properties used to augment a partial implementation of the observer pattern to synchronize parts of the application. Cool.

After that Visual Studio is fired up to design a dialog to embed in the application.

Chapter 7: Agile Testing – Where Dynamic Typing Shines

This chapter is all about testing. We all know how testing is fundamental. Michael Foord uses the standard unittest module and his Mock library.

I really liked the monkey patching explanation combined to the method lookup rules in Python. It refreshed my memory and I guess it’s useful for newcomers.

The author also shows functional testing and user stories applied to the same GUI application.

I once developed a wxPython app in TDD (with Twisted in place too) and that thought me a lot of stuff about mocking and GUI testing.

Chapter 8: Getting deeper in IronPython: Metaprogramming, Protocols and More

I think this chapter is mandatory for a non-Python developer, or a beginner anyway.

It’s all about protocols in common Python code and in the standard library: basic procotols, generators, descriptors and so on.

I honestly didn’t know there was some kind of duck typing in C# too (see IEnumerable and foreach)

A good book on Python cannot ignore metaprogramming through metaclasses and I think Michael Foord does a good job explaning the how, what and why without giving an headache to the reader about the gory details. There’s plenty of documentation about the dark side of Python out there already.

The “more” part of the chapter’s title is: profiling through metaclasses, differences between .NET types and Python’s and more.

Chapter 9: WPF and IronPython

From now on the book goes in details about the .NET world seen from IronPython side. From chapter 1 to chapter 8 you build the solid ground, now it’s time to make it flourish.

One of the big parts of .NET is his UI support and Windows Presentation Foundation (WPF) is the king in the castle.

Michael Foord develops an example showing lots of stuff. It’s good to know that you can still leverage the tools to design the UI like mainstream .NET developers do.

Chapter 10: Windows System Administration with IronPython

We all know how versatile Python is. It can scale up and down easily. The activity of system administration can be very simplified with a dynamic language. Yes I’m talking to you Windows developers!

Please stop writing scripts with the DOS batch language, (Iron)Python is there to help you.

It really makes system administration on Windows shiny. The book teaches how to interface with WMI and PowerShell. Who new you could call IronPython from another shell language!

Chapter 11: IronPython and ASP.NET

ASP.NET is the web framework of .NET. IronPython can be used but there are some extensions to install to make it so.

I still find extremely odd the concept of viewstate: an encrypted serialization of the state of all the controls of an ASP.NET page that gets embedded in a hidden HTML element and passed back and forth each time. What!?

The book shows a way to serialize Python objects inside there (beware, it can potentially make the HTML pages bigger and bigger).

Anyway I still find a lot of ASP.NET stuff very odd, like the limit of one web form per page.

I stated my opinion about ASP.NET (which I find one of the worst parts of .NET) a lot of time ago. Nice to see that almost nothing has changed

Looking forward to see how ASP.NET MVC will be.

Chapter 12: Databases and Web Services

This chapter starts with the .NET data model and its interface to database systems. Michael Foord Christian Muirhead (the co-author) uses PostgreSQL to support the examples.

The second part is about the SOAP and REST capabilities of the framework. Consuming a SOAP webservice in .NET it’s really easy, altough the example parses a web service written in .NET itself.

I wonder if it’s all easy with a SOAP service exposed by other stacks.

Chapter 13: Silverlight: IronPython in the Browser

Microsoft some time ago released a plugin almost cross platform and almost cross browser called Silverlight.

Silverlight is useful if you want to develop applications inside the browser in a sandboxed environment.

Silverlight features: a slimmed version of WPF, full access to the DOM, access to JavaScript, cross domain requests, threading, asynchronous programming, client side storage and even media streaming with video brushing.

The chapter develops a Twitter client to show these features.

I’m still not really up to the idea that the future of web development should be inside these plugins, but it’s there if you want to use it.

Chapter 14: Extending IronPython with C# / VB.NET

This and the next chapter explain how to reach out the usual domain of IronPython and either extend it or embed it.

IronPython is theoretically extensible with any language of the .NET platform but for obvious reasons the book treats particularities of C# and VB.NET.

It deals with Python idioms and how to implement them, it deals with how to create mutable classes, how to implement keyword arguments and so on.

The really interesting part is the last one where it’s shown an example of a function to generate and compile at runtime a .NET assembly using code generation techniques.

Very funky!

Chapter 15: Embedding the IronPython Engine

With this last, advanced, chapter Michael Foord explains how to embed IronPython and so augment a .NET application. It can be used as a scripting engine, for plugins or to mix and integrate other languages (you can even interact with snippets of codes written in different languages!).

Conclusions

I personally found the book very enlightening and there’s much of interesting stuff in there for both Python and .NET developers. I guess it’s the first book on IronPython and anyway it’s a must have for everyone who needs to work on .NET.

It shows there are some unpolished parts with the integration and some differences between CPython and IronPython behavior but it can only become better in the future.

So I decided to divide my twitter account into two. @rhymes will still be my personal rambling page in Italian, @lawrenceoluyede is going to be my window to the rest of the twitter world.

This way my english speaking followers don’t have to understand italian too

I tried almost them all in various degrees of depth: Python + CGI, Zope 2, Webware for Python, Subway (who does remember it?), Twisted Web, Nevow, Python + WSGI, Paste, Django, Pylons and maybe something else I can’t remember right now.

Nowadays most of my attention is caught by Pylons and Django. The latter has recently come to its 1.0 milestone and altough some books about it out there need update there’s a ton of documentation to start on the right track. Pylons has a different approach (but not that different in my opinion, because Django is not tight coupled as some people think it is) on how to develop applications but I’m not here to judge which one does things in the best way although I have to admit that for personal preference I’d vote for Django.

I’ve been using Pylons at work (the web part of one of our flagship product is going to be Pylons in and out) for a year and I used Django for personal and professional applications and both of them have pro and cons.

What really concerns me is the different feeling the everyday use of both give me. When I’m focused on Django I feel part of an increasing and participating community with tons of blog posts,  job offerings, and niceties such as the amazing Pinax project, django people and so on. It does also feel like I’m using something rock solid despite its limitations (every piece of software has them).

Pylons, au contraire, is just a nice piece of software with which you can develop your applications but I don’t have around me the same community, the same vibe.

I hope the upcoming Pylons book is going to change that. Don’t underestimate the power of communities, there’s not only Facebook out there

The reactor runs a giant loop in which events are processed in a non-blocking fashion. Sometimes, though, everything a man needs it’s just to make it stop. At least for a while, at least for the sake of getting data from the user.

The most prominent example of a command line client program that needs to stop and wait is a REPL. The following kind of REPL is a bit unorthodox, you’ll see.

The secret sauce (at least the one I found) to write such kind of interactive program in Twisted is twisted.internet.stdio.StandardIO. It connects your protocol to standard input and output:

class Repl(basic.LineReceiver):
    delimiter = '\n'
    prompt_string = 'cmd> '
 
    def prompt(self):
        self.transport.write(self.prompt_string)
 
    def connectionMade(self):
        self.sendLine('Welcome to Console')
        self.prompt()
 
    def lineReceived(self, line):
        # blank line
        if not line:
            self.prompt()
            return
 
        self.issueCommand(line)
 
    def issueCommand(self, command):
        # send the command to the server
        d = sendCmd("%s%s" % (command, self.delimiter))
        d.addCallback(self._checkResponse)
 
    def _checkResponse(self, args):
        success, num_lines, data = args
        if num_lines > 20:
            # use less to display the response
            self.lessify(data)
        else:
            self.sendLine(data)
        self.prompt()
 
    def lessify(self, data):
        p = subprocess.Popen(["less"], stdin=subprocess.PIPE)
        p.communicate(data)
 
    def connectionLost(self, reason):
        reactor.stop()

When the protocol it is connected to the standard output with

stdio.StandardIO(TaskConsole())
reactor.run()

the program displays the prompt, hence when a line is received from the standard input it is sent to the other protocol attached on the network and a callback is registered for the response. I also decided to use less to display the response if it’s more than some lines but that’s a detail.

The sendCmd function instantiate the networked protocol and its factory:

def sendCmd(cmd):
    factory = CFactory(cmd)
    reactor.connectTCP('127.0.0.1', 1234, factory)
    return factory.deferred

Then, when the server replies with some content we check to see if everything went ok and the reconstruct the whole response sending it back the REPL:

class Client(basic.LineReceiver):
    delimiter = '\n'
 
    def connectionMade(self):
        # send the command received by the cmdline to the server
        self.sendLine(self.factory.cmd)
        self.buffer = []
        self.cmd_success = True
 
    def lineReceived(self, line):
        # basic check error/success
        if line.startswith('OK'):
            return
        if line.startswith('ERR'):
            self.cmd_success = False
            return
 
        if line == 'END':
            # join the response at the end of it
            self.responseFinished(
                len(self.buffer), "\n".join(self.buffer))
            self.buffer = []
        else:
            self.buffer.append(line)
 
    def responseFinished(self, num_lines, data):
        # disconnect
        self.sendLine('quit')
        self.transport.loseConnection()
 
        # send back the response to the REPL
        self.factory.deferred.callback((
            self.cmd_success, num_lines, data))
 
 
class CFactory(protocol.ClientFactory):
    protocol = Client
 
    def __init__(self, cmd):
        self.cmd = cmd
        self.deferred = defer.Deferred()

How cool is that? Not really to be honest. It has a big gigantic fault: every time you input a line a connection to the server is opened and closed. That’s bad, really bad.

It didn’t take too long to create a version that use just one connection:

def connectionMade(self):
    self.sendLine('Welcome to Console')
    self.factory = CFactory()
    self.connector = reactor.connectTCP(
        '127.0.0.1', 1234, self.factory)
    self.prompt()

We store the connector and the factory. issueCommand does not open a connection anymore, just:

def issueCommand(self, command):
    self.connector.transport.write("%s%s" % (command, self.delimiter))
    self.factory.deferred.addCallback(self._checkResponse)

We write directly to the transport of the connector (and not the one connected to the stdout) and register the callback on the factory’s deferred.

That’s better in my opinion and a nice start. I know that within twisted.conch.stdio there’s something more evolved. I’ll try to look into it when I have more time.

You can find the first version (bad) and the second version (better) online.

What do you think about this try?