Technology Temple

Duetto: A faster and smarter alternative to Emscripten. And more.

2013-04-15T23:04:23Z

We have seen a lot of hype on the Web recently after the announcement of Asm.js by Mozilla: a new “extraordinarily optimizable, low-level subset of JavaScript”. The system builds on the work that has been done on Emscripten: a LLVM-based solution which compiles C++ to JavaScript, allowing for an easy port of applications and games to the Web. The excitement for Asm.js stems from the fact that, by using a special virtual machine integrated in Firefox, it can improve the performance of emscripten-generated code and get it even closer to native performance.

We (Leaning Technologies Ltd) would like to introduce Duetto, our own LLVM-based solution for programming the Web using C++. And by the Web, we mean both the client and server side of it, but let’s talk about the client side first.

Emscripten handles C++ code by emulating a full byte-addressable address space. This is definitely a good solution, but suboptimal. Javascript is not based on a byte-addressable address model, but on an object-addressable model: all the accessible memory is contained in some object. But when you think about it, C++ is not that different.

Our solution integrates with clang and the LLVM toolchain and is able to map C++ object-oriented constructs to native JavaScript objects. It turns out that accessing objects on modern JavaScript engines is faster than accessing arrays. By using this (and a few more) tricks we managed to get the following, preliminary, results on micro benchmarks.

For each benchmark the best time in 10 runs has been selected. The V8 and Spidermonky JavaScript shells has been used. The respective commits are b13921fa78ce7d7a94ce74f6198db79e075a2e03 and b9d56a1e0a61. *The fasta benchmark has been modified by removing a memory

We managed to do this by realizing that by disallowing some unsafe C++ capabilities (such as type unsafe pointer casting and pointer arithmetics inside structures) it’s actually possible to create more efficient, smaller and faster JavaScript code from C++. Interestingly enough, we discovered that, in most cases, the needed limitations on the language are actually specified as undefined behaviour!

So, yes, Duetto does need some minimal porting to bring C++ code to the Web while emscripten makes it mostly free. What you get in exchange for that is faster performance with no need of a special VM and deep integration with the browser. Duetto creates a really seamless C++ programming experience for the Web:

Seamless integration with the browser environment, complete access to the DOM and HTML5 technologies including WebGL. You can even access and use your favourite JavaScript library or existing JavaScript from C++ by declaring the available interfaces in the C++ code using a simple convention.
Seamless client/server programming, using transparent RPCs in single codebase. The compiler will split the code automatically in the client part (compiled to JavaScript) and server part (compiled to native code).

The Duetto backend is already in a very advanced state, and we believe it’s already suitable to bring the first applications to the Web. Especially games, which are our primary target. Unfortunately our front end is not yet as polished as we would like, as we want to improve the error reporting to make the porting experience as smooth as possible.

We are not yet ready to release Duetto, but we are eager to start opening collaborations, so if you are interested in bringing your C++ application or game to the web, feel free to contact me (alessandro@leaningtech.com). We believe that in six months or less from now we will be able to release a robust product, most probably capable of generating even faster code. And we want to release it as open source.

For more information please visit our site: http://www.leaningtech.com

A tale of hidden symbols, weakness and gold

2013-03-08T00:04:49Z

A few days ago I was profiling the startup time of the clang compiler. The callgrind report highlighted that a relatively high amount of time was being spent in the _dl_lookop_symbol_x. A quick gdb inspection of the call stack quickly pointed out that the dynamic linker was spending the time doing dynamic relocations. Tons of them.

Using objdump -R clang I found out that a whopping 42% of them actually contained the word “clang” in the mangled symbol names. Demangling the names made it clear that they were mostly definitions of the C++ virtual tables for clang internal classes.

Basically 42% of the relocations happening at run time were actually looking for symbols which are obviously defined inside the clang executable itself. So what’s happening here?

Turns out the problem is a poor interaction between the traditional linker (ld, from the binutils package) and C++‘s ODR rule.

ODR stands for One Definition Rule and says that any entity in C++ code must be defined only once. This is obviously trivial for methods, since if they are defined twice, even in different translation units, there will be an error at link time. There are, though, a few things that are implicity defined by the compiler, for example the virtual tables, which are defined whenever a class using virtual methods is declared. (Here I’m speaking informally, I suspect the terminology is not 100% correct) Since the vtable is potentially defined in more than a translation unit, the symbols for the vtable are flagged as weak. Weak symbols will not conflict with each other and they will be all discarded but one (any one is fine). The surviving one will be used by the end product of the linker.

Unfortunately, the compiler does not know if the compiled object file will be used as part of a dynamic library or as part of a main executable. This means that it has to treat the vtable symbols like any library method, since (if the target is a dynamic library) potentially such symbols may be overridden by another definition in a library or in the main executable. This has to happen since the ODR rule must also apply across the library borders for proper support of a few things, especially exception handling.

So at compile time there is no way around using dynamic relocation on the vtable symbols. A possible workaroud would be to compile all the code with the -fvisibility=hidden flag. Unfortunately this is actually wrong since it may break the ODR rule!

At link time the linker has the chance of eliminating the dynamic relocation, since it does know if the target is a main executable and symbols defined in an executable cannot be overridden by anything (not even by LD_PRELOADed libraries). Unfortunately the traditional ld linker does not apply this optimization.

The new gold linker, originally developed at Google, does tough! It is able to completely eradicate the dynamic relocations to internally defined symbols, effectively reducing the load time.

Moral of the tale: use the gold linker. It should work in most cases (I think kernel code is a notable exception) and generate faster executables while consuming less memory and cpu time during linking.

And please, dear debian/ubuntu maintainers, link clang using gold.

Linux support for Asus Xonar U1 USB audio device

2013-01-15T16:10:17Z

I got as a Christmas gift an external USB audio device: an Asus Xonar U1. It’s a nice device with a decent audio quality. The audio itself, both in and out works perfectly with the standard snd-usb-audio kernel module. Unfortunately the audio controls on the device are not standard, so I developed a simple daemon and accompanying udev and pm-utils script to get the device to work. I’ve published everything under the GPL hoping that they might be useful for someone else. Feel free to clone and fork my repository on github

https://github.com/alexp-sssup/asus-xonar-u1-utils

Lightspark 0.7.0 released

2012-10-28T18:43:39Z

Lightspark 0.7.0 has been released, including several months of improvements and bug fixes.

Version 0.7.0:

* Support LZMA compressed SWFs [Requires liblzma]
* Improved BitmapData support
* Improved ActionScript compatibility
* Improved virtual machine performance and memory consumption
* Improved XML support
* Experimental support for bytecode optimization at runtime
* Improved ExternalInterface (browser integration) support
* Improved performance of JPEG loading
* Support for XMLSocket
* Completely redesigned and improved masking support

You can grab a copy of the sources from launchpad as usual.

I would also like to ask for information on a specific problem the project is facing. We are encountering more sites that tries to load library code in the form of SWZ files, which seems to be undocumented (probably obfuscated/encrypted) flash files. Any information on how to parse/load them is appreciated.

Lightspark 0.6.0.1 released

2012-06-11T09:29:44Z

I’m very happy to annouce a new major release for Lightspark, the open source flash player implementation. This release includes quite a lot of fixes, both internal and visible to users. The most important ones from a user point of view are probably the improved support for PulseAudio flat volumes, which makes it impossible for lightspark to manipulate the system volume and the newly added support for the BBC video player. To use BBC site you might need to use AdBlock, which is generally recommended since more often than not flash advertisements are not supported by Lightspark and causes failures in otherwise supported sites.

From the ChangeLog:
* Enable RTMP support by default, requires librtmp
* Fixed support for IEventDispatcher implementation pattern
* Improved serialization robustness
* Improved matrix handling
* Implement string and namespace pooling to reduce memory consumption
* Proper support for private namespaces
* Improved support for fonts
* Support LLVM 3.1
* Fix full volume issue when PulseAudio flat volumes are enabled
* Initial support for AIR desktop applications
* Support for www.bbc.co.uk video player

Source tarball is, as usual available from Launchpad.

Lightspark 100% volume issue is now fixed

2012-05-27T16:26:12Z

For some time I’ve received a few reports about lightspark raising the system volume to 100% (and “killing ears”) whenever a YouTube clip was started. It took some time for me to figure out what was going on, since I was not able to reproduce the issue on my system and of course the code was not willingly touching the system volume.

It turns out that the issue was caused by the recently introduced “flat-volume” support in PulseAudio. When flat volume is enabled the volume of a stream is absolute and not relative to the system volume. Since the default volume in flash is 100% by spec then Lightspark was unwillingly setting the system volume to an extremely high value, and I’m really sorry for that.

I’ve just committed a fix in git master that should fix this problem for good by properly virtualizing the volume seen by flash and scaling it by the system volume. I hope this helps people with flat volume enabled.

Lightspark 0.5.7 released

2012-05-16T11:19:13Z

A new version of Lightspark has been released yesterday. You can give it a try by getting the source code from launchpad. Ubuntu packages should be available shortly from our PPA

Beside a lot of small improvements this new release improves the graphics capabilities, with a focus on Flash features used by games.

Here is the changelog:

* Fixed a few memory leaks
* Improved support for BitmapData::draw
* Support for BitmapData::copyPixels
* Support for soft masking
* Support for memory usage profiling (massif compatible)

Moreover, there has been some work on implementing fully accelerated Stage3D support, it’s still in early design phase... but stay tuned.

Lightspark 0.5.6 released

2012-04-12T17:49:02Z

After a few months of great progresses under the stewardship of Jani Monoses Lightspark, the modern and open source flash player implementation is now again being maintained by me. And I’m very proud to announce the release of lightspark 0.5.6, that brings quite a few new features and a usual load of stability fixes. Source code is available of course from Launchpad. Packages should be shortly available for various distros. I will also try to get an updated Windows build in the next few days.

Here are the new features, from both 0.5.5 and 0.5.6, since I was too lazy to publish the previous release. My bad, it will not happen again.

Fix YouTube support for several videos
Support for custom serialization/deserialization
Support RPC (NetConnection::call)
Support for PNG images
Experimental support for Google Street View
Support for Firefox 10

As you can see experimental support for Google Street View is now available (kudos to Antti Ajanki for this). Moreover, the RPC and serialization support has been written in an ongoing effort by me to support FarmVille and similar highly interactive and complex flash games. We are definitely expanding the scope of what is considered supported, but there are still some unfinished corners in YouTube support, and many other video players do not work yet.

This is way I would like to encourage young hackers, especially students, that are interested in contributing to a complex project like lightspark to choose their favorite site using flash to show videos and drop by the #lightspark IRC channel on Freenode. Me and the other lightspark developers will provide the guidance you may need in the first steps. Most often it just a matter of implementing or improving a few Flash APIs to see a player start working. And you favorite site (and your name) may be posted here in the next Lightspark release!

Gnome shell extension: GTG in the calendar panel

2011-12-24T16:34:37Z

Hello Gnome Shell lovers,
from today, you can keep an eye on your tasks with style.
Here's the pic:

Here you can install it, here you can get the source code and report bugs.
It works only with GTG 0.3 and above, that will be released in March. You can also use Trunk, which is quite stable (all you have to do is bzr branch lp:gtg && cd gtg && ./gtg).

One final thing: this is the first time I develop a gnome-shell extension, and I have to say I was impressed by how nice and easy is to code. Kudos Shell people!

Happy holidays!

Lightspark 0.5.0 — Bacchus released

2011-07-26T10:42:00Z

After the last round of bug fixing and improvements over the release candidate I’m happy to announce that version 0.5.0 of Lightspark, a FOSS flash player implementation aimed at supporting newer flash files (SWF 9+), is finally released.

Beside stability improvements there is no new great new feature since the last released candidate (see here for more info). You can grab your copy of the sources on Launchpad as usual. Packages for both Ubuntu natty and oneiric are available from our PPA. Packages for your favorite distro will be probably available in a short time. I’d also like to say “thank you” to all downstream packagers for the critical work they have always done.

As always bug reporting and testing is very appreciated. If you need any help you can drop into the #lightspark channel on Freenode.

Lightspark 0.5.0 RC1 is out!

2011-07-05T19:50:59Z

This is a great day for lightspark: the modern, open source, flash player implementation. I’m very happy to announce that the first release candidate for the 0.5.0 release (codenamed Bacchus) has been just released. There are many new features in this release (see below for a detailed changelog). For the end user the most visible changes are:

For YouTube: Play/Pause/Enlarge buttons are now working correctly
Initial support for Grooveshark (currently the first song of the playlist works)

And under the hood:

Improved XML support
Improved FFMpeg based media playback
Improved robustness
Improved ExternalInterface (browser communication)
Improved masking support
Improved alpha support
Improved shader performance
Support capture phase of the event flow
Support SimpleButton
Support audio volume
Support for introspection of ActionScript objects (describeType)
Support for AMF3 serialization
Support plugin resize
Support for dynamic text
Removed SDL, FontConfig and FTGL dependency from the core
Added SDL based audio backend
More GLES compliant

Source tarball is, as usual, available from Launchpad. Packages should be available for the major distributions in the next few days.

Please test this release candidate and report any bug on the Launchpad bug tracker.

Just a couple of warnings:

there is an issue with FFMpeg 0.7 that causes a crash. We are aware of the issue and we are currently discussing a fix that should be available for the next release candidate. In the mean time please use FFMpeg 0.6.x
The newly added support for dynamic text uses pango as the backend. Unfortunately pango is currently not thread safe. Lightspark itself correctly serialize the pango calls, but there is no sane way to synchronize with the calls made by the browser thread. This means that the only safe way to use lightspark is to employ out of process plugins. Luckily OOPP is now used by default on the major browser.

Lightspark 0.4.8 released!

2011-05-27T12:07:13Z

Just a quick note to announce that Lightspark 0.4.8 has been released! The main difference from the RC1 is that a nasty memory allocation bug has been fixed, greatly reducing the amount of memory used when playing video. This release of the free, high performance, flash player implementation officially supports YouTube (as usual) and Vimeo (new entry!) You can find the code on Launchpad. Testing and bug reporting is extremely welcome. You can also usually find (friendly) support on the #lightspark IRC channel on FreeNode. As a last note, a regression of the NVIDIA proprietary driver forced me to start using nouveau and, although I was fairly skeptic, I’m pretty happy so far. This of course means lightspark will be finally developed and tested throughly on a completely free stack!

Update: I forgot to cherry pick one commit before the release and so 0.4.8 don’t actually build. 0.4.8.1 fixes the issue. Sorry for the silly mistake.

Lightspark 0.4.8 RC1 is out!

2011-05-22T19:26:33Z

A new release of lightspark, the free, high performance, flash player has been released! This is the first candidate for the 0.4.8 release. Beside the usual ton of fixes and improvements this release includes an initial support for Vimeo.

As you can see form the screenshot the support is still experimental and the video is way smaller than it should be.

Nevertheless this is a really important milestone for our project and proves what I’ve been saying for a long time: Lightspark is _not_ a YouTube specific hack, we are working hard to improve the amount of supported flash features and over time expect more and more sites to work.

Please note that this is a release candidate as many changes has been done, so it’s definitely possible that some issues will come up. Please report bugs on Launchpad as usual. You can find the source for this release on Launchpad as well.

Lightspark 0.4.7.1 released

2011-05-06T21:56:11Z

Hi everyone! After quite some time I’m proud to announce the release of a brand new version of the Lightspark open source flash player. The most visible feature for users is that YouTube support is now restored. Under the hood also the code base has been cleaned up a bit. You can get the updated tarball on Launchpad as usual

The work to support Hulu.com is currently on hold as rtmpdump/librtmp seems to be no more able to access Hulu’s contents and we’re waiting for an upstream fix. In the mean time I’m experimenting a bit to add Vimeo.com support and the results are fairly promising

Getting things GNOME! is participating in GNOME Outreach Program for Women

2011-04-05T18:12:37Z

If you are eligible and you are looking for a great summer internship, apply!

The program is here, projects ideas and motivation to participate are in my previous post.

Deadline is April 8 (I know, I’m terribly late).

Get paid to FLOSS! Apply for the Google Summer of Code with GNOME and GTG!

2011-04-05T18:02:30Z

Hello fellow students!

If you like to code and you have no plans for this summer, consider applying for the Google Summer of Code with GNOME!

I did that last year, and it is a fantastic experience for lots of reasons. Here’s a few:

You work from wherever you like, whenever you like (as long as you Get Things ~~GNOME~~ Done, it’s great)
You get paid to contribute in a FLOSS project! In three months, you have time to make an important contribution too!
It looks great on you CV, and your works gets great visibility
You get Google’s swag
You get to go to GUADEC (the Gnome conference), with is an awesome experience itself!
You can fully enjoy almost all August
You get to blog on planet Gnome, and show your contribution to the world
You become a better coder!

Now that I got your attention, have a look at the project ideas for GNOME. These are just ideas, you are free (and encouraged too) to come up with your idea: if the idea is awesome, you are more likely to get chosen!

Personally, I am rooting for you to come by and help us improve Getting Things Gnome. New ideas and great people are very welcome!

Don’t postpone, the deadline is April 8, and a great proposal can be written in just a few hours. You don’t have to have previous experience in FLOSS development (although it helps). If you are not chosen, consider contributing anyway: your next year proposal will be great!

Ps: if you have ideas for gtg, talk to me on IRC (GIMPNet/#gtg is our channel, I’m also available on freenode as invernizzi)

A C++ feature I would love to have: custom type modifiers

2011-03-18T16:09:58Z

In C++ the const type modifiers when applied to methods has the nice property of making it impossible by default (e.g. without a const_cast) to call non-const methods. A similar feature is provided by C++0x keyword constexpr.

It would be very nice if this property could be applied to custom classes of functions. The idea would be to tag method definitions with a custom keyword, or a type. Tagged methods would then only be able to call method tagged with the same type.

A nice usage I could find for this would be to define a non-blocking class of methods. Non blocking methods are such if they never blocks for a potentially unbounded amount of time waiting for an event. Using the tags the compiler will be able to detect if a method supposed to be non-blocking is trying to call some potentially blocking code and this would be very useful to detect possible deadlocks at compile time. Another usage would be to tag all the methods that are supposed to be used by a specific thread with the same tag. With such setup methods that access shared data would be untagged and using them would require a cast or some special syntax, hopefully helping the programmer remember that some synchronization is needed.

A possible approach to (ab)use the volatile keyword to detect race conditions was described here in 2001. But maybe this approach could be more general. Any comment?

Lightspark 0.4.6.1 released

2011-03-18T04:17:01Z

Version 0.4.6 was released no more than a free days ago, but unfortunately a change in the YouTube player exposed a bug that could cause browser crashes. Version 0.4.6.1 fixes the issue and updating is recommended.

I’d also like to share some though and explain why we need to make so many point releases to keep things working. Lightspark shares a lot of similarity with the Wine project in my opinion, as both projects aims at reimplementing from scratch a large (huge for Wine, of course) platform for third party applications.

The “advantage” for Wine is that executables for installed programs (e.g. Microsoft Word) change not so often, and usually updating is under the control of the user. A user that is interested in keeping Wine compatible with a certain application is, most often, free to delay the update until Wine catches up.

Lightspark, on the other hand, needs to execute contents (e.g. YouTube video player) that are mostly out of the user control and new, incompatible, versions of the contents may pop out at anytime and there is very little we can do about it.

The only way we can be faithful to the advertised “YouTube[*] support” is to improve our code whenever a breakage pops up. I understand this is also challenging for distros and packagers because the life span of our stable releases is linked to quite unpredictable events and I’d like to say “thank you” to all the people packaging lightspark for distros in the wild. Your work is exceptional.

[*] As a side note, although currently only YouYube support is stable enough to be used daily, Lightspark is not an YT specific hack. Support for other sites is coming along slowly given our limited workforce. I’ve done some work to have hulu working and I think we’re now 80% done in supporting it. If anyone is interested in contributing to lightspark to have Hulu (or any other popular site) supported, please drop by #lightspark IRC channel on Freenode

Lightspark 0.4.6 released

2011-03-15T14:47:27Z

I’m happy to announce that a new version of Lightspark flash player has been released. The release features improved stability and graphics:

Gradient support has been vastly improved
Graphical glitches are greatly reduced
Added support for the upcoming gnash 0.8.9 release
Added support for profiling ActionScript code
Added support for muting all sound by pressing ‘m’
Added support for copying error messages to the clipboard by pressing ”c’

The tarball is, as usual, available on LaunchPad. Binary packages for Ubuntu maverick and natty are available from the Sparkers Team PPA.

Packages for Debian and Fedora will be available soon.

FOSDEM!

2011-01-10T18:24:22Z

Having bought the flights I’m now sure I’ll be in Bruxelles, Belgium the 5 and 6 February to visit FOSDEM. As I always do I’ve organized this too late, well after the dealines to apply for lightning talks.

I’ve asked anyway if there are still 10 free minutes, so there is a small chance that I’ll be able to give a short talk about lightspark while I’m there.

Lightspark 0.4.5.1, with experimental PowerPC support

2010-12-16T22:24:19Z

Lightspark standalone executing YouTube on a ppc64 Debian machine

Only a few days have passed since the 0.4.5 and a new Lightspark release is already out! With big news for PowerPC users. The main new feature of this release is experimental support for the PPC platform!

Currently the performance of the video playback is far from ideal as some very inefficient code is used on a critical path, and will be rewritten in an efficient manner. Nonetheless tests and bug reports from PowerPC users are very welcome.

Beside this feature the release include:

A fix to restore YouTube support
A new AMF3 parser that will be used to provide Local Shared Object (flash cookies) support in a following release.

Packages will be as usually available for Ubuntu Maverick from the PPA for the i386 and amd64 architecture. Packages for PPC are not currently available as launchpad does not support ppc packages.

Lightspark 0.4.5 Release

2010-12-14T01:27:45Z

Lightspark 0.4.5 is out! No big changes from the last Release Candidate, beside a –version switch that will make it easier to assess if the last and best version is being used.

In the mean time work has not stopped. A brand new parser for the AMF3 is already upstream and will be used to implement ByteArray::readObject and support for Local shared Objects. Moreover work is under way to build a supported application database where users will be able to report successes/failures and vote for applications to be supported. This system is being customized having Wine’s AppDB as a base, so thanks the the Wine team for their work!

iCTF hacking competition 2010: solution of some challenges

2010-12-10T22:36:18Z

This year, having joined the computer security group at UCSB for my phd, I’ve helped in the organization of the 2010rh edition of the iCTF, the biggest international online hacking competition. It has been plenty of fun, with more than 70 teams participating from all over the world. The CMU team “Plaid parliament of pwning” won it, getting the 1000$ price (thanks Adobe and IEEE Security & Privacy magazine for the sponsorships!).

Since people have been asking for the solutions of the two challenges I wrote, so here they are.

challenge 5:

This challenge is easy, I encourage you to give it a try, it’s fun!

Question: “Who’re you’re gonna call?”

File: call
Points: 300
Teams that have completed it: 44 (congrats!)

Solution: in an html comment following this line

challenge 4:

This challenge is a little more difficult, but if you know python you have all the skills necessary to beat it.
Question:

Hello Anonymous,

it has come to our knowledge that the Lityvan Secret Service (note: this was part of a more complex story that ran across all the CTF)

is installing a remote control system on their submarines. This would

allow a handful of man to launch a massive attack from a secure

location.

This threat is not acceptable: your mission is to break into one

submarine remote control system and launch a missile against Navy Control Center,

so to shut down the system for good.

The details of this systems are unknown, but we have recovered from a

stolen usb device two files that might give some insights to you: they

are attached. We discovered what we suspect to be one of the entry point

of this system: it is located at 10.15.42.42, on TCP port 5000.

Should you complete this mission, you will be lavishly remunerated.

Good luck!

To make it run, unpack this file and run server.py (it’s all python, you can check it for backdoors). To start the challenge, you should read only the content of the “pub” directory: that was the material that was given in the ctf.

Points: 500
Teams that have completed it: 0, as most of the difficult challenges — because of the structure of the iCTF 2010, it turned out that it was more convenient to focus only on the easy challenges. We’ll have to fix it next year!

Solution: in an html comment following this line. The given file also contains a script that can solve the challenge (the test_* files)

For the solution of another difficult challenge, head over to Bryce’s blog.

Setting IP options in Scapy

2010-12-06T10:06:36Z

During this year iCTF (the world largest online hacking competition — very cool) my team faced the problem of setting an IP header option in Scapy. That was needed to break a service that was relying on IP addresses for authentication.

Now, Scapy is an awesome project, but it lacks a bit on the documentation side, as most of it is composed by a bunch of slides. Since IP options are overlooked in the docs and there is nothing findable on the web that shows how to set them, I’ll post it here hoping to help some future fellow googler — that might as well be future me.

Option to set: Loose Route/Record Route (the others work just the same):

>>> ip=IP(src="1.1.1.1", dst="8.8.8.8", options=IPOption('\x83\x03\x10'))

>>> ip.show2()

###[ IP ]###
version= 4L
ihl= 6L
tos= 0x0
len= 24
id= 1
flags=
frag= 0L
ttl= 64
proto= ip
chksum= 0xd4d0
src= 1.1.1.1
dst= 8.8.8.8
\options\
|###[ IP Option Loose Source and Record Route ]###
|  copy_flag= 1L
|  optclass= control
|  option= loose_source_route
|  length= 3
|  pointer= 16
|  routers= []
|###[ IPOption_EOL ]###
|  copy_flag= 0L
|  optclass= contro
|  option= end_of_list

The ‘\x83\x03\x10’ has been created following the linked specs (\x83 is the type of the option, \x03 is the length, \x10 is the pointer).

Note that if you forget the IPOption() bit, your packet will be printed correctly with show()/show2() but will refuse to be sent on the network. Instead, it will show this error (here for the sake of search engine indexing):

>>> send(IP(src="1.1.1.1", dst="8.8.8.8", options='\x83\x03\x10'))

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/pymodules/python2.6/scapy/sendrecv.py", line 247, in send
    __gen_send(conf.L3socket(*args, **kargs), x, inter=inter, loop=loop, count=count,verbose=verbose, realtime=realtime)
  File "/usr/lib/pymodules/python2.6/scapy/sendrecv.py", line 230, in __gen_send
    s.send(p)
  File "/usr/lib/pymodules/python2.6/scapy/arch/linux.py", line 384, in send
    sx = str(ll(x))
  File "/usr/lib/pymodules/python2.6/scapy/arch/linux.py", line 382, in <lambda>
    ll = lambda x:conf.l2types[sn[3]]()/x
  File "/usr/lib/pymodules/python2.6/scapy/packet.py", line 260, in __div__
    cloneB = other.copy()
  File "/usr/lib/pymodules/python2.6/scapy/packet.py", line 140, in copy
    clone.fields[k]=self.get_field(k).do_copy(clone.fields[k])
  File "/usr/lib/pymodules/python2.6/scapy/fields.py", line 401, in do_copy
    return map(lambda p:p.copy(), x)
  File "/usr/lib/pymodules/python2.6/scapy/fields.py", line 401, in <lambda>
    return map(lambda p:p.copy(), x)
 
AttributeError: 'str' object has no attribute 'copy'

About Sant’Anna School of Advanced Studies

2010-11-29T16:59:29Z

As a departure from the usual technical articles and release announcements I’d like to talk a bit about my university: the Sant’Anna School of Advanced Studies (Scuola Superiore Sant’Anna in Italian). This place is quite unique in Italy, the only similar institution being the Scuola Normale Superiore. I’ve never heard about similar institutions in other countries. I’d love to hear if anyone knows about something like this somewhere else.

At a first glance the Sant’Anna of Advanced Studies could be descibed as a very small, state owned (public) university. By “very small” I mean that the grand total of the students attending bachelors and master classes is no more than 250. Each year around 45 students are selected in a very strict nationwide selection from around 1000 applicants in the fields of Engineering, Economics, Medicine, Agriculture, Political Science and Law. Selected applicants have access to the school services completely free of charge, including lodging and canteen.

The high level of selectivity (the peak is 1 admitted each 20 applicants) makes it possible to gather some of the most talented people graduated in the nation’s high schools. As the school provides all needed infrastructure to the students people are able to leave their home and come to Pisa independently of the economic and social status of their families. There is no way to access the School other than the official selection, not even paying a fee.

As students we live in collegiate structure. The rooms are assigned following our own internal regulation, this is an example of the large amount of management that is delegated to students. Several commissions are elected each year to manage a variety of things, from network infrastructure to public relations. Living in common structures and sharing the responsibility of their maintenance generate a very peculiar and often long term bonding between students. During the five years that most people spend in the school a net of relations forms that connects people that are very different for interests and experiences. Living in a interdisciplinary environment is an extraordinary experience. It’s very common to see people discussing about anything from philosophy and politics to trappist beers with the same commitment, most often while eating together at the canteen.

From the academic point of view the School is also pretty unique. Although it’s a full state recognized university it does not offers full courses. We are at the same time Pupils of the Sant’Anna School and students of the University of Pisa. We attends regular classes at the University of Pisa and classes on more advanced modern research topics. Anyway most people feel that the most important part of what we learn comes from living with older students and research experiences abroad. I’ve myself worked in California and Switzerland during the last couple of summers.

Feel free to comment and share your opinions about such particular kind of university.

Lightspark 0.4.5 RC is out!

2010-11-27T16:52:04Z

Finally, after a long delay caused by several issues Lightspark 0.4.5 is out. Here it is a brief Changelog.

Include the new Advanced Graphics Engine, that should provide smoother and faster graphics and support for clipping
More robust input support (makes it possible to use Play/Pause on YouTube)

The work on this released has been especially slowed down by an issue found in libxml++ and another one found in mesa. Packagers should pay attention to fulfill the following dependency requirements:

Libxml++ version 2.33.1 or better. If an older version on libxml++ must be supported some commits must be backported. More info about this in the README file
Mesa should include the fix for this bug. The issue affects some radeon and maybe intel cards.

Packages will be available for Fedora Rawhide and are already uploaded on the PPA for Ubuntu Maverick

See you in a week!

2010-11-16T22:58:41Z

Me and most of my engineering colleagues here at Sant’Anna School for Advanced Studies are leaving tomorrow morning for a week long tour of universities and labs in Belgium and Netherlands. This means I’ll have no time to work on Lightspark, sorry

Anyway the 0.4.5 release will be probably out in the last days of the month. The critical bug in libxml++ is now fixed as my patch was accepted upstream. If (as I believe) a new version of the library will be released and packaged before i get back there will be no reason to hold the release anymore.

On the road to Lightspark 0.4.5

2010-11-07T23:09:52Z

The last week have been pretty busy for lightspark development. As the Advanced Graphics Engine is now merged in master the upcoming 0.4.5 release will feature better looking graphics and reduced CPU consumptions.

Moreover, input support has been completely redesigned. The new approach is very robust, even of the very complex and crowded YouTube interface and with lightspark 0.4.5 it will be finally possible to click the pause button.

Currently the release is on hold because of a bug in libxml++, the library used in lightspark to handle XML files. I’ve submitted some patches that correct the bug (that caused random crashes inside firefox code) but they’re still waiting for a review. I’m considering and discussing with packagers if bundling a patched libxml++ inside lightspark could be a good temporary solution, until the library gets patched upstream. More news about this in the next few days.

As a last note, I’ve received some feedback from the developer working on the PPC port and I’m happy to show a first image of Lightspark running on a PPC machine. There are still issues with graphics and some remaining endianess swapping to be done, but the port is taking a reasonable shape finally.

RFC: is OpenGL not the Right Thing (TM)?

2010-10-02T13:33:38Z

It’s late at night and I’m back in Pisa for my last year of master. And, as often happens, a weird idea struck my mind. What if OpenGL is not the right thing for Lightspark? No, I’m not talking about dropping hardware accelerated rendering as that’s surely the right way to go, but using OpenGL really looks unnatural. In the design of the advanced graphics engine OpenGL is being basically used only to upload images rendered with cairo to the VRAM, and to blit and composite all the rendered chunks together... do we really need all the OpenGL complexity to do this??

Well... OpenGL is basically the only thing we have, the only way to talk with the graphics hardware. But, here it comes the gallium project! As gallium splits the API from the driver it could be possible, in theory, to write a specialized gallium state tracker to do only the work we need... and maybe do it better.

I’m writing here first because I’m not (yet) experienced enough with the gallium platform to know if the idea is sane, and second because I somehow feel the same approach could be useful for other apps... for example Lightspark and compositing window managers have similar needs. So I’d like to have some feedback about writing a small API and a gallium state tracker to do:

DMA accelerated transfers of rendered image data
Blitting and compositing of such data on screen
Notify the application when asynchronous work (such as DMA transfers) has ended (BTW: what’s the right way of doing this in OpenGL?)
Enqueue to-be-uploaded-to-vram images and have them sequentially transfered
Apply simple but programmable (shader-like) transformation to pixel data

Big disclaimer: I’ve not yet started working on this idea. I’ve not even seriously though about it’s feasibility. I’d just like to have some feedback on this.

Lightspark new release and Advanced Graphics Engine progresses

2010-09-25T18:06:50Z

A new point release in the 0.4.4 series has been released yesterday (0.4.4.3). There are not many effects visible to end users, beside a small fix to restore YouTube compatibility. Under the hood much work has been done to support policy files one of the security oriented features of the flash platform.

Although we still miss some stuff (e.g. Text), the Advanced Graphics Engine is able to render gradients

I’d also like to give some insight about what is going on the advanced graphics engine branch. The roadmap is being walked without major issues. Now the texture real estate for graphics elements is allocated from a single large texture, geometries are drawn asynchronously using cairo on the CPU side and the resulting raster data is uploaded to the right chunks of texture using PBO based transfers that are (hopefully, this is dependent of the OpenGL implementation) handled through DMA by the graphics card.

The new engine is not yet on feature parity with the old one, but to showcase the new possibilities i’ve added support for color gradients. It’s also interesting to note that, after the first rough measurements, the new engine is fairly faster than the previous one.

YouTube video sample

Another advantage of the new design is that OpenGL code will be pretty much condensed in a few locations. Moreover, it will be used only for accelerated blitting, compositing and upload of data to video memory. This opens new interesting possibilities for other graphics backends.

As a closing news on Friday I had the first chance to physically meet another member of the lightspark community, namely our Debian packager Didier Raboud, as we discovered we are both working at EPFL. It has been a nice coffee-based meeting and we had a change to shake hands and discuss about some long term plans for the project.

Lightspark 0.4.4.2 released

2010-09-12T13:53:47Z

Another week, another bugfix release for Lightspark! Apart from restoring the support for YouTube this release features the new plugin based audio framework that makes it possible to support other backends beside PulseAudio. At the moment both an ALSA and OpenAL plugins are being worked on.

As always you can grab the release here on Launchpad

GTG for old fashioned people

2010-09-12T13:12:51Z

This insomniac night I’ve added a little nice thing to Getting Things GNOME, which is the possibility to export and print a to-do list in a PocketMod format. This is nothing more than a little foldable booklet, so that you can carry around your grocery list without needing anything more technological than a pen (you can also use strawberry juice to mark your tasks, which is way more recyclable and a little gory).

This all comes from a bug reported by Jan Girlich, so kudos to him!

Obligatory screenshot:

By the way, GTG export plugin supports templates, so creating your own webpage/pdf/t-shirt with your own tasks is easy. I’ll be glad to help anyone interested in that.

Lightspark 0.4.4.1 released

2010-09-02T17:56:06Z

Hi,

version 0.4.4.1 of the lightspark player has been just released. It’s mainly a bug fix release, the most relevant news are:

Fixed a crash when using flashblock
Restore support for YouTube

Moreover, from this release large downloads are cached to disk to reduce memory pressure.

As you may have noticed lightspark is now on Flattr and the last few posts about lightspark included the “Flattr this” button. Moreover, the Flattr-foss project (that suggest free software to be supported using flattr) has recommended Lightspark for september. Thanks a lot to the Flattr-foss team for their interest and support! Many people flattered lightspark the last month and that is really appreciated. I’m not of course talking about the (little) money, what is awesome is the large support received from the community and even the smallest donation is greatly motivating. Thanks to everyone

Lightspark 0.4.4 released

2010-08-29T16:32:08Z

Architecture of the proposed upcoming Advanced Graphics Engine

Lightspark 0.4.4 has been released today. Thanks a lot to all the people that made this release possible. Beside the usual amount of bug fixes several new features have been included

Localization support (using gettext)
ActionScript exception handling support
More robust network handling
Streams controls (Play/Pause/Stop)

It should be noted that, although now video streams controls are supported they’ll be not usable in most YouTube videos as mouse event dispatching to controls is still clobbered by missing masking support.

Lightspark now supports localized error messages, but we miss translations! So I’d like to invite any user (non developers included) willing to help Lightspark to contribute the translation for his/her native language.

I’d also like to give some insight what is being worked on for the next release (0.4.5). First of all the pluginized audio backend is now mature enough to be merged upstream, this is the first step toward support for multiple audio backends. That said anyway Lightspark will always focus on functionality and not on the amount of backends offered. We’ll work to offer a very small number of fully working backends.

In the mean time we’re also discussing a new faster and more powerful graphics architecture. My proposal is a mixed software/hardware rendering pipeline, somehow inspired by modern compositing window mangers. Static (defined in the SWF file) and dynamic (generated using ActionScript code) geometries will be rendered in software using cairo and exploiting the thread pool to be scalable on multi core architectures. The resulting surfaces and decoded video frames (if any) will be uploaded using Pixel Buffer Objects to offload the work to the video card (this usually involves a DMA transfer). OpenGL will then be used to blit the various rendered components on screen, while applying filters, effects and blending.

That’s all folks. As always testing from as many people as possible is critical for the success of the project, so please try out this release and report any crashes/weird issues and anything you don’t like. I’d like to put an emphasis about this: never assume a bug is already known. If you hit a crash take a look at launchpad bug tracker. If your issue is not already reported, please do it!

Getting things GNOME and Remember the milk: call for testers!

2010-09-06T07:26:43Z

Aloha planet,

as you may know, Getting Things GNOME is a todo-manager software that, among other things, can synchronize the things-you-really-should-be-doing-instead-of-reading-this-post with various sources.

One of its most popular features is the ability to synchronize with Remember the Milk.

For the next release of GTG, I’ve written a new version of that synchronization, with features a variety of nice things. Among those:

background sync (set up once and forget about it)
support for RTM recurrent tasks
rollbacking of partial syncs
on-the-fly sync from GTG to RTM

Before releasing that, however, I’d like to do have it well tested, to ensure that no harm is done to your precious todo items, so.. I’m looking for testers!

If you have the 5 minutes necessary and you’re interested in trying new stuff even before it hits a PPA, execute

tar czf gtg_backup.tgz .local/share/gtg/  #just in case, backup of your gtg install (if any)
sudo aptitude install bzr
bzr branch lp:~gtg-user/gtg/all_the_backends_merge_requests  gtg_backends
cd gtg_backends
./scripts/debug.sh -d #this lauches a debug version of GTG that doesn't touch your own GTG tasks

And go to Edit->Backends to start (your existing GTG tasks won’t be touched, so you’re safe).

You’re more than welcome to play with the other backends as well (and report bugs, as they should be ready for being released as well). Those are, at the moment:

twitter
identi.ca
tomboy
gnote (needs development version of gnote)
launchpad
local file

If you find a bug, please report it to https://bugs.edge.launchpad.net/gtg (linking this post).

If you don’t fine any, leave a comment here, so we can know that we’re good (^_^).

Thanks!

Lightspark 0.4.3 final release

2010-08-10T18:24:01Z

I’m officially announcing Lightspark 0.4.3. A couple of interesting features as been added since the RC and a good amount of bug fixes thanks to user reports. The aggregate changelog since 0.4.2 is huge! Here it is:

Liquid layout support
Faster rendering of the input layer
Reduced memory consumption
Support for H263/MP3 videos
Smoother playback of audio and video
Fallback on Gnash for older clips

Many users asked why vimeo, hulu and many others video sharing sites don’t work while youtube does. The answer is, although flash based video players looks simple a lot of action script code is actually being executed under the hood. Support for youtube has been a priority given it’s huge popularity but I’d like to clarify that lightspark is not a youtube specific hack and that over time support will come for any site.

As a way to help new developer join my effort I’d like to launch a “site adopting” campaign. Developers that are willing to work on lightspark, but are scared by the complexity of the challenge can choose a site using flash that is important/useful for them and work to implement the needed features. To help them I’m most always online in the #lightspark IRC channel and available to tutor them.

As a closing news, after the last post a radeon user helped to shed some light over the ati related issues. radeon users are urged to upgrade at least the mesa 7.8.2 to try lightspark as the support for the needed GL features got definitely better. Keep reporting bugs if anything is not working even after the upgrade. Moreover, I’ve discovered today that firefox older than version 3.5.11 will not work as it misses a needed plugin interface, so keep your systems updated!

Lightspark 0.4.3 RC1

2010-08-08T22:19:51Z

Today has been published the first release candidate of Lightspark 0.4.3. Source tarballs are available as usual from launchpad. Prebuild packages for Ubuntu Lucid and Maverick are available from the PPA as usual.

The new features in this release are

Faster rendering
Reduced memory consumption
Support for H263/MP3 video (using FFmpeg)
Smoother audio and video playback

Be sure to try this out and report bugs in launchpad and our irc channel (irc://irc.freenode.org/lightspark). Users of radeon cards (and the open source radeon driver) are especially invited to try lightspark. Many radeon users complained about crashes and weird renderings which are often caused by missing features in the drivers. It would be nice to gather as much information as possible on working/non working cards to open a unique bug report upstream.

Getting Things GNOME! — GSoC review (#11)

2010-08-06T18:33:18Z

Hello planet!
I’m back from GUADEC. It was my first conference about open source and it was great.
I’ve found particularly inspiring the talk by Guillaume Desmottes about Telepathy and Epiphany, which can be great to extend GTG possibilities in collaboration.
The talk by Jake Edge about promoting free software projects was also very interesting, in particular for the young Lightspark project (that went completely unnoticed for a few months before showing up on planet GNOME).

Thanks to the exciting talks and people at GUADEC, the GTG team (even the people who were not there!) has been working fervently on a nice rewrite of some parts of GTG core, along with a lot of unit-tests. Hopefully, a lot of bugs will be closed thanks to this, and GTG will be nicer to code.

Some of the GTG people at GUADEC. From left to right: Bertrand Rousseau, Karlo Jez, Lionel Dricot and me.

As for my Google Summer of Code on Getting Things Gnome support for multiple backends, this week has seen:

a port of my Evolution plugin as a backend (that was the last one planned)
refactoring of the Twitter plugin to get authorization through Oauth (using the tweepy library, thanks Tante for the hint)
docs, docs, docs

Next week, I’ll keep documenting and testing. I should also write a guide on how to write new backends. See you next week!

Lightspark 0.4.2.3 bug fix release

2010-08-04T17:42:22Z

Just a brief announcement for release 0.4.2.3 of the Lightspark project, the modern, open source flash player

This release will not show off huge user visible changing but fixes a number of stability problems, give it a try.

The current focus of development is testing lightspark’s VM against the ActionScript testsuite provided by tamarin. In the meanwhile I’ve also found a way to reduce lightspark memory consumption while increasing performances, but the solution is not stable yet for prime time.

Stay tuned.

Getting Things GNOME! — GSoC review (#10)

2010-07-30T08:32:19Z

I’m at GUADEC!

It’s been awesomely great so far. I’ve finally met the guys I’ve been working with and new people from all over GNOME. Thanks to them and to the talks I’ve been following we have new ideas for GTG! I’ll tell you about them as they get a little more polished, but they will involve Telepathy.

As for my GSoC work, the first version of the Remember the milk backend is now complete. A couple of small bugs still need to be fixed

Thanks to the first days GUADEC wifi, I’ve been able to test my backends on an unreliable link (now the network works perfectly).

We’ve also been having a small GTG hackfest to fix the nastiest bugs in trunk.

See you next week, and thanks for all the people organizing GUADEC, giving talks or just being here!

Lightspark 0.4.2.2 released

2010-07-27T03:34:07Z

Trying to keep up with the old rule “Release early, release often” I’ happy to announce release 0.4.2.2 open source flash player.

This apparently small point release actually includes the biggest feature plannend for the upcoming 0.4.3 release, namely Gnash fallback on older SWF clips. Lightspark currently relies no Gnash for any Flash content that does not require AVM2 (ActionScript 3) support.

I would also like to explain an issue that many users and testers reported. Firefox is not able to handle multiple plugin for the same file type! Not even if only one of those plugins is actually enabled. So, if lightspark is installed alongside adobe’s player or Gnash no flash content will be displayed. This is a firefox bug, I’ve reported the bug and proposed a patch that is currently waiting the review.

The source of the release is as always available on launchpad. Binary packages for Ubuntu Lucid and Maverick will be available on the usual PPA (in a couple hours from now, Launchpad seems pretty busy at the moment). Moreover, since the last announcement lightspark has been also included in the debian experimental suite (thanks to Didier Raboud and Luca Falavigna).

Stay tuned, and follow the roadmap

Getting Things GNOME! — GSoC review (#9)

2010-07-23T17:57:29Z

This week in “Getting Things Gnome!” development of multiple backends:

As for yesterday’s post, we can now automatically import (some) Evolution mails in GTG
I’ve been working on the Remember The Milk backend to speed it up. Since RTM allows only one API call per second, anything cacheable must be cached.
I’ve been testing and fixing the couchdb backend. Now it syncs on ubuntu one without error on my machine. It would be great if someone is willing to test it. Keep in mind that ubuntu one is syncing couchdb databases every ten minutes or so. Instructions on how to test are here.

Next week, I’ll be at GUADEC! I really don’t know if I’ll have the time to work on this at all, but if I do, it will be all about the RTM backend.

See you in Den Haag (or L’Aia, in Italian. I wonder why..)!

Automatically convert a new mail in Evolution to a GTG task

2010-07-22T14:52:12Z

A GTG user, Damiano Venturin, asked me if it was possible to convert a mail stored in Evolution to a GTG task, as it can be done in mutt.

This is a pretty interesting question. There are two different ways:

automatic conversion of new mails tagged in some way (e.g. “Note to self: go to surf tomorrow #TODO”)
manual conversion of the mail currently being read

Both of them work. I’ll show you how:

Automatic conversion of new mails

Download this script, place it anywhere you like and make sure you give it executing permission (that is, chmod +x <script-name>)
Open Evolution, in the mail tab
From the menu, select Edit->Message Filters
add a new filter
Edit the filter such that it catches only the mails you want to transform in tasks
Edit the filter such that it executes the downloaded script

This screenshot shows all these steps

Here, when a new mail containing the text #task is received, the user is notified via the usual notifier bubbles and a new gtg task is created.

Manual conversion of mails

In GTG, enable the Evolution plugin
In Evolution, activate the Mail-to-Task plugin (via the menu bar: Edit->Plugins)

Now, each time you want to import a mail in GTG, you should:

in Evolution, from the global mail view (that is, not from an opened mail), right click and select “Create a task”
in GTG, synchronize with evolution.

The Evolution plugin for GTG will soon be transformed in a backend, which will make the synchronization completely automatic (removing the need for the second step).

Lightspark 0.4.2 released

2010-07-20T18:47:51Z

Version 0.4.2 of Lightspark, the modern, efficient, open source Flash player is now officially released, with a couple of last moment fixes and improvements.

The main features of this new release are:

Use fontconfig to select fonts
Greater compatibility with youtube video
Sound synchronization
Chrome/Chromium support
Firefox’s OOPP support

If you use a version of firefox newer than 3.6.4 you can ensure a crash free experience by enabling OOPP mode for lightspark, for instructions see the wiki

Package for Ubuntu Lucid and debian testing/unstable are provided as usually by the PPA. Packages for Fedora 13 are also available (currently only for the i686 platoform). See the installation page for more information.

To test lightspark install it from packages or sources and try one of the following pages:

http://www.youtube.com/watch?v=4N2YWRJ-ppo

http://www.youtube.com/watch?v=XITHbsUUlYI

http://www.youtube.com/watch?v=-j7c0u9yZ8M

Lightspark 0.4.2 RC3... almost ready

2010-07-18T14:53:43Z

Having now sound synchronization fully working Lightspark has all the feature programmed for the 0.4.2 release! So no new big changes will be introduced, except for critical bugs reported.

But there are more big news! As I’ve revamped the plugin by switching from the old deprecated Xt interface to GTK, Lightspark has also gained Chrome/Chromium Plugin support and support for the new Out Of Process mode offered by newer versions of Firefox for improved stability and crash free experience.

Moreover, a contributor is currently porting Lightspark for the PowerPC platform and the work is currently in a very advanced stage. So PPC guys cheer up, Flash is coming!

I’m not sure those feature will be stable enough for the final 0.4.2 but will be surely included in the early 0.4.3 release cycle.

Getting Things GNOME! — GSoC review (#8)

2010-07-23T17:33:35Z

Hello and welcome to my GSoC report for my eighth week of work about supporting multiple backends in “Getting Things Gnome!”.

This week, I’ve finished the first complete version of the couchdb backend. This one is supporting all GTG’s features (subtasks, tags, fuzzy dates...), so it can be used as default backend (instead of the xml one).

Its performances are fairly good: for small and medium sets of tasks (I’ve been testing it with ~500 tasks) it’s a tad slower than the xml one, but I suspect that for big tasks sets there are speed advantages. Anyway, being GTG 0.3 fully asyncronous, users shouldn’t even notice it.

I’ve written a unit-test for it, and started to check how it behaves with replicating couchdb databases. It works quite well (a few crashes occur from time to time), but I’ll test it a bit more next week.

I haven’t been able to get automatic replication of the database via Ubuntu One so far. I’m starting to think that something is wrong in my installation.

Next week, I’ll keep on testing the couchdb backend and I’ll work on finishing the RTM one (which should be the last big one, since the Evolution one is very easy, thanks to the nice api).

Ciao!

Getting Things GNOME! — GSoC review (#7)

2010-07-09T21:15:30Z

This week, I’ve started to work on two backends planned for my GSoC on “Getting Things Gnome!”.
One is the Remember The Milk backend, for which I’ve written the authentication system and started to sync task titles (I already have the code for the rest in the old plugin, but it will need a bit of refactoring to make it easier to read and compatible to the new backend system).
The more interesting backend is the one based on couchdb, which should enable to keep two GTG installations in sync. This backend should also work on the Ubuntu, which has a couchdb implementation which differs a little bit from the standard (e.g. deleted records are not really deleted, but are “marked” as deleted — FYI, a good tutorial on Couchdb and Python is the one on Ars by Ryan Paul). Anyway, so far, it’s working nicely, although it’s not yet supporting all the features of a gtg Task.

Next week, I’m planning to expand GTG Dates module to make serializing dates easier. I’ll also have to finish and test threads destruction upon closing GTG, to ensure tasks operations to be atomic.

ps: it seems that an API for Google Tasks should be announced soon. That would be a great backend for GTG!

Lightspark 0.4.2 RC2... it shines!

2010-07-03T23:20:13Z

I’m very proud to announce the the second release candidate of Lightspark 0.4.2: the modern, efficient and open source Flash Player implementation. Thanks to all the people that tested the project and reported feedback on the bug tracker and on the IRC channel, without their help this awesome results would have not been possible.

Although we’re still missing a couple of feature before the real 0.4.2 most of the pieces are already in place. Let’s see what you can expect from this release:

Youtube support for H264 videos. Currently only those are supported as they are played using the Action Script 3 based player. This may seem a huge limitation, but actually a huge part of the YouTube contents are available in H264 format. This limitation will go away when lightspark will be able to fall back to Gnash. This feature is scheduled for 0.4.3
Even faster video presentation after a bit of refinement of the SSE2 based video packer
Sound support using pulseaudio. If you want to try Lightspark without installing the pulse server that’s ok, as Lightspark detects at runtime if the server is available and if not it just politely disables sound.

As usual you can grab the source from Launchpad

Official binary packages for Ubuntu Lucid and Debian testing are available from my PPA http://launchpad.net/~sssup/+archive/sssup-ppa (in launchpad build queue as I’m writing)

Packages for Fedora 13 are also available here

As I mentioned before we’re not yet ready for the final release as the following issues needs to be fixed:

Sound is not synchronized
Sound sample rate is not always correctly detected

Beside those known issue, everything should be pretty ok. So go on, give it a try!

Getting Things GNOME! — GSoC review (#6)

2010-07-02T19:19:57Z

This week has been busy for me, since I have (just a few hours ago) discussed my master thesis. Anyway, now I’m going to be working full time on GTG, which is nice.

So, this week I have finished the first version of the Launchpad backend (importing in a read-only fashion launchpad bugs assigned to somebody in GTG). I’m thinking if it would be interesting to have the possibility of changing something about the bug through its task in GTG, but I haven’t found nice ideas so far. I’ll look into making a bugzilla backend, since many people requested that.

Secondly, I’ve written an export backend to Zeitgeist, so that tasks that have been completed are also visible there. This makes it easy to see what it has been done day by day. Another approach would be informing Zeitgeist when a task gets modified, created and so on (like a regular document). While this second approach is more “zeitgeisty” (since it leaves a trace of the activity of the user), I think that for todo items the important information to keep trace of is when they get done. I’ll see what users prefer when they start using it, or you can leave your opinion in the comments here.

I’ve also written a patch to gnome-activity-journal to support TODO items (which are supported in Zeitgeist 0.4 which has just been released, and GAJ is being updated to use that).
That’s what you should get in September (~ planned GTG release time):

Next week, I’ve a lot of things to do. A few of them are:

I’ll add a “remember the milk” backend, which will have a series of advantages versus my old plugin (automatic syncing is one of them)
I’ll review and ask for merging to trunk the code for my UI and the tomboy backend, so that other developers and brave users can start using my code
I’ll make order among my threads, since a few libraries that I used have a series of synchronous calls which can make closing GTG slower than normal. I’ve discussed this with a friend (the creator of Lightspark), and a few interesting ideas have come out.

Getting Things GNOME! — GSoC review (#5)

2010-06-24T21:50:39Z

Another GSoC week has gone, and my project with “Getting Things Gnome!” has moved forward.

We now have a complete tomboy backend. Tomboy notes matching a particular @tag (you can configure multiple tags, or get all notes) are imported in GTG. The synchronization is both ways: if you change something in Tomboy, GTG will be updated and vice versa (and GTG doesn’t need to be running when the modification is done).

That backend comes with a unit test, which has helped me spot a few bugs. The tomboy dbus interface is synchronous, which coupled with the presence of a Global Interpreter Lock in python, has caused a few weird hangs during execution, as threads were not really indipendent. Anyway, now everything seems to work fine

Secondly, the new backend framework has landed in GTG Trunk, so my work is starting to be tested by a few reckless testers. Thanks!

Lastly, I’ve been working on a Launchpad backend, which imports (read only) the bugs assigned to you on that bug tracker. This one still needs a little work, as I’ve started working on it yesterday, but it will be ready and tested for next week. The synchronization engine, which is common for all backends (and generic, as the only requirements it has is that objects it syncs have unique id and a modification date), really does most of the work.

As usual, a screenshot with the Launchpad backend at work:

Next week I’ll work a little less, as I have my “pre-degree” exam. I’ve already done most of the things I’ve put in my proposal (I’m missing the RTM and Evolution backends, which are just a matter of converting my old plugins, and a couchdb backend –which was optional, but I certantly have the time to do it), so I’m confident I will finish in time.

Have a good weekend!

Lightspark, on the road to release 0.4.2

2010-06-20T17:39:13Z

I’ve not been posting for a while, but development of Lightspark is going on fast and well. I’ve to iron out a couple of issues before we get back support for YouTube AS3 player, which was lost after another site update. Since the last release a lot of work has been done and basic sound support has been added using PulseAudio.

Please follow the project RoadMap to keep track of what’s going on. As you may have noticed I’ve moved the lightspark home page to trac, although only for the wiki. The bug tracker is still on Launchpad. You may find interesting the WorkingSiteList where I’m gathering a few sample flash applications which are known to work and a screenshot to verify that everything is working as expected.

Expect news very soon!

Getting Things GNOME! — GSoC review (#4)

2010-06-24T20:38:44Z

Another week has gone, and new ways to synchronize things to do have landed in my “Getting Things Gnome!” branch.

First of all, there is a new two-way synchronization engine for tasks (complete with test cases), which will be the core of all the backends to come.

The first backend using that is the new TOMBOY–GTG synchronization: the idea behind this one is that some of you are used to use tomboy to take quick notes, but then they realize that they also need that note in their todo list. With this backend, just adding a tag to the note (like “@tomboy-gtg”) will automatically add that note to GTG the next time you start it.

So, basically, it’s a way to superpower some of your notes, giving them a way to get scheduled in your super-busy third-millennium life.

Screenshot is due:

For now, two-way sync is working for all tasks (not filtering them yet), and a few bugs have to be ironed out, the most important of which is that GTG sometime issues the deletion of all tasks when it’s quitting (which is rather unpleasant).

Next week, I’m planning to get this plugin ready for the users. Then, I think the Remember The Milk backend will follow, and shortly after Evolution and CouchDb.

5 questions nearer to GUADEC

2010-06-24T20:39:13Z

1)Who are you and what do you do?

I’m Luca Invernizzi, a quarter of a century old Italian student. In a couple of weeks I’ll be graduating in Control Engineering at the leaning University of Pisa and in the fall I’ll fly to California for a Ph.D. in Computer Science (at UCSB). Right now, I’m a Google Summer of Code student working on “Getting Things GNOME!”.

Whow, lots of Capital Letters, my pinkie is hurting.

2) How did you get into GNOME?

It started as a way to keep my coding skills sharp, but then I got addicted to contributing. It’s great to work together and see our little app growing.

3) Why are you coming to GUADEC?
This will be my first FLOSS conference, and it will be an occasion to meet the people I’ll been collaborating with. AFAIK, they may be all very smart IRC bots

4) In 1 sentence, describe what your most favorite recent GNOME project has been.
Getting Things GN... Jokosher! I just discovered that I can plug in my acustic guitar and have fun with effects.

5) Will this be your first time visiting the Netherlands?
I’ve been to the Netherlands for just a little more than one day so far, so it will be like my first time.

Getting Things GNOME! — GSoC review (#3)

2010-06-10T15:37:24Z

I’m pleased to announce the first two new backends for “Getting Things Gnome!”: a twitter and an identi.ca backend.

Those are import backends, meaning that your dents/tweets will appear in your GTG tasks. In particular, the message will be parsed like this:

“this will be the task title, this will be the task description #sometag @alsoatag”.

There is no support for start/due dates yet.

Obviously you will not want every dent/tweet in your GTG tasks, so you can choose to just import direct messages/@ replies/your own messages, and select only messages with one or more tags matching the a set of tags.

Screenshots time!

Adding a backend:

Configuring it:

If you type your password incorrectly, you will be notified:

If the connection cannot be established, you will be notified in a similar way only if network-manager reports that at least one of your network devices is connected (otherwise, GTG assumes you know you’re offline ). Passwords are stored in the gnome-keyring.

Apart from the new backends, this week has seen improvements in the backend framework and fixes, as always.

Next week will be quite busy for me, as the deadline for my thesis is approaching. I’m planning to work on read/write backend support. I’m currently developing a directory-based backend (each file in the directory is a task) for testing purposes, but I’d like to start working on a Tomboy backend: notes in Tomboy having a particular tag (like @importfromtomboy) will be automatically imported in GTG.

Lightspark 0.4.1 released

2010-06-05T01:57:03Z

Lightspark 0.4.1 has been released today, featuring initial YouTube support, and of course the usual performance improvements. As the plugin is currently very stable everyone is invited to test and measure the CPU consumption of lightspark versus Adobe’s player. During my preliminary tests lightspark resulted up to twice as fast! This is of course not a completely fair comparison, as Lightspark is not feature complete yet.

It should be noted that only the YouTube videos served using the new AS3 player are supported, those are the ones with the new UI. YouTube currently uses a legacy AS2/Flash8 player for older content and that should be fully supported by Gnash.

Moreover, with this release, Lightspark has been relicensed from GPL3 to LGPL3 to avoid licensing issue when distributing the plugin with non GPLed browsers such as Chrome

Getting Things GNOME! — GSoC review (#2)

2010-06-10T14:46:00Z

Aloha again, planet Gnome!
This has been a nice week for my Google Summer of Code project on Getting Things GNOME, featuring:

A great speedup (via the refactoring of how the file containing all the tasks is handled): the time for adding 1000 tasks in GTG has been reduced from 30 to 5 seconds!
A new testing class, which helped me find-n-fix a few bugs (ehi! testing works!)
The first request to merge of part of my code (~3600 lines). I have to thank my mentor Lionel Dricot, who has found the time to go through it.
A new twitter backend. It still misses the UI to configure the username and password, but the basic functionality is there. Currently, it adds to GTG any direct message matching a set of chosen tags (e.g., #todo).

I’m still working on twitter authentication. I’m currently doing it via the userid/password combo, but the correct way to go should be Oauth. Unfortunately, python-twitter does not support this. I’ve found a few libraries around the web, but none seem to work so far. Any hint will be welcome.

Next week I’m planning to finish the twitter backend, expanding the framework as I go. A nice thing is that, thanks to the framework, the twitter backend (which is all a developer should write to add a new backend) is less than 100 lines long.

Lightspark 0.4.0 released

2010-05-31T01:43:25Z

Just a quick update. I’ve released version 0.4.0 of Lightspark, a free flash player implementation. This release was focused on improving stability, so all the crashes found by many testers should be fixed now. Thanks a lot for testing, several issues were related to particular graphics hardware and I would have never found them without your collaboration. Please keep testing and reporting any issue.

Now focus shift on YouTube support, which was lost after one of the last update of YouTube’s infrastructure. And believe me, we’re not far! I’m attaching a screen shot of the current status (in GIT master) as a proof. Full support will be delivered with release 0.5.0

Getting Things GNOME! — GSoC review (#1)

2010-06-10T14:46:18Z

news dispatch:

I’ve just been informed that a Getting Things GNOME! plugin is being added to Kupfer! That’s great.

The real post:

For the Google Summer of Code, the time to start coding has officially arrived.
My work is about adding yet another feature to the personal organizer software Getting Things GNOME!, that will let you synchronize your tasks in a variety of online and offline backends (Remember the milk, Launchpad...). Details are here.

This week, I’ve been working in:

how backends are enabled and disabled
a signaling framework for backends changes
a nice ui for managing backends (which took most of the time)

Here’s how you add a backend:

And here’s how you configure it (yes, a lot of things are missing):

I think the UI is coming up pretty well: you can see which tags are associated with each backend and edit them, rename the backend, add and delete any number of backends...

Next week, I’ll focus on hunting bugs down and writing the ui for a series of backends parameters (filenames, authentication via web pages, passwords stored in the gnome-keyring).

The UI needs a lot of makup still (images, alignments etc..): they will come in due time ^_^

Lightspark News: Progress on stability, Codenames and Logo Poll

2010-05-23T00:35:09Z

One of these will be the official logo of Lightspark

First of all, thanks a lot to all those brave enough to try out this project. I’m sorry about all the (frequent) crashes but, with the help of all the people who filed bugs on launchpad, the stability of Lightspark is improving very fast. Please keep testing and reporting any issues. The next big release, 0.4.0 codenamed “Aeolus”, is planned for the first week of June. The focus for this release is the stability of the platform and no major features are being implemented. The release is also going to include a brand new logo! The call for logos of the previous post generated a lot of very nice works, and it was very hard to choose between them. In the end I managed to keep only two of them, and now it’s your turn! Vote for the one you prefer.

Beside aesthetic things I’m also trying to define a bit the roadmap of the project. If the next release is only focused on stability, for the following one (0.5.0, codenamed “Bacchus”) I’m planning working Youtube support which was lost after one of the updates of the video player.

I’ve also received a lot questions and interest about porting Lightspark to other OSs and architectures. The code is build using standard technologies, such as pthreads and STL and should be quite portable, but some critical code paths has been written in assembly to guarantee atomicity or improve performance. I’ve very little experience with anything beside x86/x86-64, so I prefer not to port such critical code. However I will gladly accept any contributions for other platforms, such as PPC and ARM. The good news is that a contributor managed to compile lightspark on FreeBSD/x86 with minimal changes to the build system and a windows port is also planned. Moreover, beside the Ubuntu PPA I’m maintaining, packages are being created for Arch Linux and Debian, thanks a lot to the community.

Getting Things GNOME! — GSoC review (#0)

2010-05-23T00:24:35Z

Hello again, dear Planet!

This one has been a busy week in both GTG mailing list(s) and my little branch. I have one thing I would like your opinion on, which is clearly marked, so you can skip some parts.

You may not know that in the last month GTG lied on the surgeon table to undergo a huge refactoring, mostly by Lionel Dricot. That made GTG faster and asynchronous in task loading. Just for a measure of the improvements, we’ve lowered the time taken to load 1000 tasks from 40 to less than 30 seconds, and we haven’t even started profiling yet. The feels way faster than before, and we have lowered our startup time. We have also gained a strong division between our core and our UI, which makes it possible to write new UIs for GTG (that what Karlo Jez is doing for his GSoc, writing a GTG web service). KDE UI, anyone?

Last Wednesday has been declared “Getting Bugs Done” day, so we worked in fixing all the regressions introduced in the trunk after the refactoring. We have now a trunk that can be used normally (if you don’t mind the occasional glitch). A few minor bugs are still unresolved, but the only “big” bug remaining is the breakage of the plugin API.

Anyway, my branch, which is about having the support for multiple backends (Remember the milk, CouchDb (ubuntuone), lauchpad, zeitgeist, twitter...), has seen a complete refactoring of backends loading and storing, with the addition of a backend Factory. We can support multiple instances of the same backend, so right now you can save/load your tasks on multiple files at the same time. You can “attach” to each backend some tags, and the backend will sync only those (say, you want on your work computer only the tasks marked @work).

I’ve also worked on laying out a well documented base class that each new backend should derive, in order to make the creation of new backends easier.

The more eye-catching part is that we are currently discussing a mockup for the UI to add and edit backend. The current design is this one (Empathy inspired). I’d like to hear your opinions on that, whether you like it or not.

Going to Guadec!

2010-05-20T23:09:56Z

I’ll soon be attending my first big FLOSS conference! GNOME developers, enthusiasts, users and passer-bies, we’ll meet at GUADEC at the end of June, and we’ll see how gnomish the future is.

A big thank you to the GNOME foundation, that is sponsoring me to stay at GUADEC and created this nice green badge full of happy people (or people pushing up the GNOME logo? Maybe it’s falling down? Hope they’re fine).

Meanwhile, I’ll keep trying making GTG more awesome!

update: Guess I’m still a beginner in blogging, since I’ve messed up the title. This is not my GSoC weekly report, obviously

Lightspark, an innovative FLOSS Flash player, reaches Beta Status

2010-06-18T14:26:51Z

More than a year has passed since the beginning of the Lightspark project. It was a long time indeed, but not wasted. I’m extremely pleased with the current level of support and the robustness of the system. You may be wondering “What the heck is lightspark?”, so let me introduce it.

Lightspark is an open source Adobe Flash implementation, designed from the ground up to be efficient on current and (hopefully) future hardware.

The long term objective of this project is not only to deliver a free as in freedom flash player, but also to give to the Flash technology the dignity of a real open standard. I really believe Flash is a powerful technology, it’s main weakness being it’s closedness. Lightspark has been entirely designed after Adobe released the SWF file format documentation, so no reverse engineering has been done.

Features

I’ll try to review and summarize what as been done so far, and what is planned for the future.

Lightspark showing video. Profiling graphs can be overlayed on the movie.

OpenGL based rendering allows for fast rendering of geometries and video frames. Moreover, although those are not yet supported, blur and other effects can be implemented using shaders for maximum efficency. Using OpenGL textures to display video frames is a bit less efficent than using XVideo, but make it possible to support any overlaying/transformation effect that flash provides.
Mostly complete support for the newer version of the flash scripting language: ActionScript 3.0, introduced with Flash 9. Both an interpreter and a JIT engine based on LLVM are provided. The previous versions of the language (supported by Gnash, which does not support 3.0) run on a completely different, and quite weird, virtual machine. Currently Ligthspark does not support the older engine, but most sites (e.g, YouTube) are migrating to the newer engine, so we will be fine just with 3.0
Beside the Virtual Machine Flash provides a huge runtime libraries that ranges from Video Rendering services to XML parsing. Implementation of the runtime is of course far from complete, but the architecture of the VM makes it possible to easily implement the various objects in plain C++. Moreover, an arbitrary degree of mixing between VM generated and C++ code is possible. This means that functions can be called from whatever context without worrying about their origin.
A mozilla compatible plugin is provided, I’ve spent a lot of time trying to make the code robust to unexpected conditions and it should fail gracefully and display a gentle message. But, as plugins runs inside the browser process, there a not-so-slight possibility of crashes. Please report any crashes using the bug reporting system of launchpad
As an added bonus, during development I’ve also added a couple of interesting performance profiling/debugging features which are not provided by the original Adobe player. Keyboard shortcut can be used to enable a profiling overlay, and a primitive inspector of the objects on screen

Therefore I declare lightspark to be in beta status! Now the most important thing that is still missing is a nice Logo for the project. I’m very bad at art, so any contribution is very welcome. And of course the whole project, in a pure open phylosophy, is open to contribution! To have a quick glance at the current status it is possible to get lightspark packaged from our PPA on launchpad.

Lightspark should fail in a clean way on unsupported Flash movies

Getting Things GNOME, supercharging procrastination since 2008. Now in the Cloud!

2010-05-16T17:35:32Z

Hello there, Planet GNOME! I’m Luca, one of the developers of Getting Things GNOME!. I’m taking this little empty space between posts in your RSS reader of choice to present you to some of the awesome things that will be featured in the next major version of GTG (due next fall). Preorder your copy now!

In the Cloud

In the near future, GTG will free your task from the cage of your hard disk. I know that some of you are already using the plugin for synchronization with Remember The Milk (that’s one of my most popular plugins ), but we are talking about more serious business here.

In short, you will be able to import/export/synchronize automatically all or part of your tasks into a variety of Back-ends. On top of that there will be an easy UI to rule them all.

Back-ends will include:

Remember the milk (full synchronization with notes, tags ...)
Evolution Tasks
Launchpad (automatic import of bugs assigned to you / all bugs in a project / all bugs with a certain tag..)
Zeitgeist (exporting completed tasks — have you seen that Zeitgeist now supports hamster?)
Ubuntu one support (CouchDB)
Twitter/Identi.ca (automatic import of direct messages tagged with #TODO — or something of your choice)
Xml (for backups)
Email
...

You will choose what to export and where, by “attaching” tags to a back-end. That is, if you want all your work tasks to be on your Remember The Milk account, just tag them @Work and add a RTM back-end synchronizing only the @Work tags.

Therefore, be joyful! You will have your TODO list everywhere, even on your Windows machine we all know you’re keeping in your basement.

All of this will be done thanks to the the “Google Summer of Code”, which kindly changes my “I should be working instead” thoughts to “hell, I’m working!”.

Not a big fun of commercial online services, even if they are (beer) free? We have the FLOSS!

Snowy-like web service for GTG

This will be a child of Karlo Jez, as another GSoC project. We’ll soon be having our web interface! A mock-up to whet your appetite

Speed

Big chunks of GTG have been rewritten to make you feel like your computer is getting more powerful as it gets old. Now you can procrastinate thousands of tasks!

UDS-inspired improvements

Projects: We heard your requests for a way to add “projects” to better categorize tasks! Nothing is decided yet, but we’re working on it . We’ll be also working on the indicator icon, ... oh, too much work! Where is my beer?

Call For Ideas!

If you’ve read so far then, congratulations, you can tick off “Read Luca’s GTG post” off your TODO List. You could add another task, though. Some of you have told us that GTG icon set is too similar to a clipboard. We are thinking about renovating it, and it would be nice to see some proposals. So, sketch up, and post here there results!

The quest for graphics performance: part II

2010-03-16T22:09:37Z

I’d like to talk a bit about the architecture I’ve using to efficiently render the video stream in Lightspark. As often happens the key in high performance computing is using the right tools for each job. First of all video decoding and rendering are asynchronous and executed by different threads.

Decoding itself it’s done by the widely known FFMpeg, no special tricks are played here. So the starting condition of the optimized fast path is a decoded frame data structure. The data structure is short lived and it is overwritten by the next decoded frame, so it must be copied to a more stable buffer. The decoding thread maintains a short array of decoded frames ready to be rendered, to account for variance in the decoding delay. The decoded frame is in YUV420 format, this means that resolution of color data is one half of the resolution of the luminance channel. The data is returned by FFmpeg as 3 distinct buffers, one for each of the YUV channels, so we actually save 3 buffers per frame. This copy is necessary and it’s the only one that will be done on the data.

Rendering is done using a textured quad and texture data is loaded using OpenGL Pixel buffer objects (PBOs). PBOs are memory buffers managed by the GL and it’s possible to load texture data from them. Unfortunately they must be explicitly mapped to the client address space to be accessed, and unmapped when the updated. The advantage is that data transfer between PBOs and video or texture memory will be done by the GL using asynchronous DMA transfers. Using 2 PBOs it’s possible to guarantee a continuous stream of data to video memory: while one PBOs is being copied to texture memory by DMA, new data is been computed and transferred to the other using the CPU. This usage pattern is called streaming textures.

In this case such data is the next frame, taken from the decoded frames buffer. Textures data for OpenGL must be provided in planar form. So we must pack a 1-buffer-per-channel frame in a single buffer. This can be done in a zero-copy fashion using instruction provided by the SSE2 extension. Data is loaded in 128 bit chunks from each of the Y, U and V channels, then using register only operations it is correctly packed and padded. Results are written back using non-temporal moves. This means that the processor may feel free to postpone the actual commitment of data to memory, for example to exploit burst transfers on the bus. If we ever want to be sure that the changes has been committed in memory we have to call the sfence instruction. For more information see the Intel reference manuals on movapd, movntpd, sfence, punpcklb.

The result is a single buffer with the format YUV0, padding has been added to increase texture transfer efficiency, as the video cards internally works with 32-bit data anyway. The destination buffer is one of the PBOs so, at the end of the conversion routine, data will be transferred to video memory using DMA.

Using the streaming texture technique and SSE2 data packing we managed to efficiently move the frame data to texture memory, but it’s still in YUV format. Conversion to the RGB color space is basically a linear algebra operation, so it’s ideal to offload this computation to a pixel shader.

Lightspark gets video streaming

2010-03-15T03:18:33Z

Just a brief news. It’s been a long way, and today I’m very proud to announce video streaming support for Lightspark, the efficient open source flash player. Moreover, performance looks very promising, I’m not going to publish any results right now as I’d like to do some testing before. Anyway, Lightspark seems to be outperforming Adobe’s player by a good extent, at least on linux.

In the next post I’ll talk a bit about some performance tricks that made it possible to reach such result.

Lightspark’s news

2010-02-26T15:26:42Z

Lightspark progresses are never been so good. The last achievement was to correctly load, execute and partially render the YouTube player. As you may have seen YouTube has recently switched to Flash 10 and ActionScript 3.0 to serve some HD content, while keeping the old AS2 based player for lower quality videos. The old player is supported by Gnash but, until now, there where no open source alternatives to play newer, high definition content. As Lightspark AS3 engine matures, that gap is almost closed. Stay tuned, as I’m planning to release a new technology demo very soon.

UPDATE: Demo tarball released on sourceforge

Extreme FLEXibility

2010-01-26T03:06:10Z

Although there has been no official news about Lightspark for several months, i’ve been doing a great deal of work under the hood. As my bachelor thesis, I’ve mostly completed and throughly tested my LLVM based Actionscript 3 .0 JIT engine and, during the last days, I’ve been working on polishing a bit the Virtual Machine. I’m proudly announcing that, in some days, a new technical demo of Lightspark will be released, but this time we’re not talking about basic examples. Lightspark is now mature enough to run a simple application based on Flex.

Flex is a rich open source framework written in Actionscript and developed mainly by Adobe. Even if right now the test application features only a progress bar and a square, there is a lot of stuff being done by the framework under the hood.

If the framework works it means that the engine is now stable enough to move from a pre-alpha to an alpha status. The design is also now satisfying enough for me to allow other people to join the project and work on on subsystems without knowing the internal details of everything. As an added bonus preliminary support for the Windows platform will be included in the release.

The screenshot above is the result of the execution of my test application, for curious people the flash file is generated using the mxmlc compiler, from the following source file
<?xml version="1.0" encoding="utf-8"?> <mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" horizontalAlign="center" verticalAlign="middle"> <mx:VBox x="0" y="0" width="201" height="200" backgroundColor="0x0080C0" alpha="0.8"/> </mx:Application>

Allocating aligned memory

2009-10-29T20:56:36Z

Just a quick note that may be useful to someone else. As you may know SSE2 introduced a new instruction: MOVDQA (MOVe Double Quadword Aligned). This is used to move 128 bit (16 bytes) of data from/to memory/xmm registers. This instruction only works if the data is aligned the the 16 byte boundary. There is also another instruction for the unaligned case, but the aligned version is way faster. So let’s summarize some techniques to get an aligned memory area

For local, static and member variables you can append __attribute__ (( aligned (16 ) ) to the type definition. Example:

struct A { int val; } __attribute__ ((aligned ( 16 ) );

For dynamically allocated memory the usual malloc is not enough, but there is a posix_memalign which has the semantics that we need. It is defined as:

int posix_memalign(void **memptr, size_t alignment, size_t size);

So we have to pass a pointer to the pointer that will receive the newly allocated memory, the required alignment (which has to be a power of two) and the allocation size. Memory allocated this way can (at least on the glibc implementation) be freed using the usual free function.

Talking about CPU architectures: Then it came assembly chain

2009-10-22T07:52:34Z

As we’ve seen in the previous essay in a simple processor design each instruction needs several cycles to be completed. Each instruction passes several steps: first it’s loaded from memory (FETCH0 state), then it’s decoded (FETCH1 state) and the appropriate code path is selected to compute the operation results. In the my example I’ve not included other two essential steps: operands fetching and results storing on memory. This sequential approach has a big drawback, as for most of the time the CPU circuits (such as the memory interface or the ALU) are not doing useful work. If on average each instruction takes five steps to complete, then the processor is only executing one instruction every five cycles. To solve this problem we can introduce the pipeline.

The pipeline concept is borrowed from the industrial assembly line. The basic idea is to decouple the sequential steps of each instruction by inserting buffer registers in between. Each stage of the pipeline is isolated, and on each clock cycle it can take the input from the register upstream and output the results into the register downstream. Of course not every instruction needs processing by all stages (for example it may or may not access memory) and so each stage has to have some kind of No Operation mode to pass trough the unmodified input register to the output.

A simple pseudo-verilog description of a pipelined processor follows. We are assuming that the instruction and data memory are physically separate: an Havard like architecture. This is not true in current mainstream architectures, as both instruction and data are stored in RAM, but may be considered true to a large extend if there is a fast instruction cache. Moreover it has to be noted that although the access_memory module is referenced both in the input fetching and output storing stages, it can only correctly serve one request at a time, and so some kind of arbitration has to be used. We can arbitrarily decide that reads are prioritized over write, but in case of a collision the operation will be for sure not correct, though safe for the electronics. The register_file module is just a multiplexer for the registers, it will output the value of the registers for the provided indexes. The module it’s used both in the operands fetching and writeback stages, but it’s safe because it allows two register reads and one write and the same time (three way access).

This toy processor is inspired from MIPS architecture and as you can see it uses fixed length instructions. But I’ve also tried to design something more similar to an x86 processor. The instruction set (which I’m too lazy to define extensively) could allow both register-register and register-memory operation, and output to either memory or registers. No indirect memory access is allowed, sorry. As the instructions are four byte long probably only a 16 or 24 bit flat address space could be accessed.

always @(posedge clock) begin // //Instructrion fetch stage FETCHED_INSTRUCTION <= get_instruction_memory(PC); PC <= PC+4; // //Instruction decoding stage {OPCODE_0, OPERAND1_TYPE, REG1_INDEX, REG2_INDEX, MEM_ADDRESS_0, WRITE_TO_MEMORY_0, DEST_REG_0} <= decode_instruction(FETCHED_INSTRUCTION); // //Operands fecthing stage, some register will be forwarded to the next stages OPERAND1 <= (OPERAND1_TYPE==REG) ? read_register_file(REG1_INDEX, REG2_INDEX)[31:0] : access_memory(MEM_ADDRESS_0,READ,UNSPECIFIED); OPERAND2 <= register_file(REG1_INDEX,REG2_INDEX)[63:32]; MEM_ADDRESS_1 <= MEM_ADDRESS_0; WRITE_TO_MEMORY_1 <= WRITE_TO_MEMORY_0; DEST_REG_1 <= DEST_REG_0; OPCODE_1 <= OPCODE_0; // //Execution stage RESULT <= alu(OPCODE_1,OPERAND1,OPERAND2); MEM_ADDRESS_2 <= MEM_ADDRESS_1; WRITE_TO_MEMORY_2 <= WRITE_TO_MEMORY_1; DEST_REG_2 <= DEST_REG_1; // //Writeback stage (WRITE_TO_MEMORY_2) ? access_memory(MEM_ADDRESS_2,WRITE,RESULT) : write_register_file(DEST_REG_2, RESULT); end

Several considerations can be made on this design. First of all extreme care is needed from the compiler (or the user, if it’s crazy enough to write assembly manually), because there are several hazards, not only as said previously on concurrent read/write memory access, but also on registers. As the result of an instruction is written back to the register file only at the end of the pipeline it’s not legal to access a result before five more instruction has been executed. To guarantee memory access validity it’s enough to avoid issuing a memory write two instruction after a memory load. If no useful instruction can be issued during this forced waiting, a NOP should be inserted.

Real world architecture employs interlocking techniques which takes care of stopping (stalling) the pipeline when an hazard could occur. I’ll describe stalling in more detail in another essay. Anyway if a pipeline stage, such us the ALU for complicated computation (for example integer division), needs more than a cycle to complete it will stall the provious stages of the pipeline, and insert so called bubbles into the later stage. Bubbles are basically safe No Operation input states, the NOP opcode in any architecture is just an explicit pipeline bubble. Even in modern interlocked architecture correct instruction scheduling is extremely useful to gain performance as pipeline bubbles actually waste execution time.

That’s all for now, in the previous post I forgot to mention the ever failing, but very nice Very Long Instruction Word (VLIW) approach. I’m of course going to talk about that to sometime.

Talking about CPU architectures: The beginning

2009-10-21T01:47:44Z

Some days ago I was talking about multicore systems and modern CPU technologies with a younger engineering student at Scuola Superiore Sant’Anna. I was quite surprised by how much misconceptions and hype surrounds this topic, even for people that actually study computer engineering. Especially considering that the courses of the University of Pisa are quite good when it comes to low level hardware stuff (computer engineering was born by branching electronic engineering, and the legacy is still quite evident).

So I’ll try to write some articles about CPU architectures, from the basic to advanced topics... well... as advanced as my current knowledge goes, still better than nothing.

Ok, for this essay I’ll basically reference the simple processor design that we study in our awesome Logic Circuits course (thanks Prof. Corsini, even if I’ve not seen a single class of it). The code presented is written is something hopefully similar to Verilog, the hardware description language.

This processor is basically a simple state machine. The internal state is stored in some synchronous register, usually implemented using so called edge-triggered flip-flops. There is a clock source which generates a square wave. The clock is used as an input to registers, which are allowed to get a new value only on the raising edge of the wave. The new value of the registers is usually computed by stateless asynchronous circuits.

A simple circuit to get the address of the next instruction: the new address is computed during the clock cycle by the asynchronous adder and assigned at the next rising edge of the clock

This means that the output depends only on the input, and the clock signal is not used. An example of this kind of circuits could be the adder, or the more complex ALU. Those circuits takes some time to complete their work, the adder, for example, has to propagate the carry information from the least significant bit to the most significant bit. Before the computation is complete the output state undergoes several spurious transitions (add gif) which will be ignored by the following register, as the new value will only be captured on the raising edge of the clock.

This allows us to understand two concepts:

The faster the clock is, the more new states can be computed, and so the more operations can be done. So this is what the vendors are talking about when they market they several GhZ CPUs.

Why can’t the clock be infinitely fast? Because to guarantee that all the spurious transitions from asynchronous circuits will be discarted the clock cycle time has to be at least a little longer than the longest settle time of those circuits.

For a long time during the previous years it was easy to pump up the clock speeds, from the 4 Mhz of the first 8088 to the 4 Ghz of the last Pentiums, as the progress in CMOS techlogies allowed to make the circuits smaller and smaller, which also means basically faster and faster. But after a while this had to stop, even if transistors are still getting smaller. The main problem right now is that power consumption and heat production go with the square of the clock frequency, which poses a very heavy upper limit to the reasonable clock speed, if you’re not willing to use liquid refrigeration.

This simple 3 bit adder needs some time to add 011 (3) and 001 (1) because it has to propagate carry information

So, back to the CPU internals. One of the registers has a special purpose: it keeps the state of the processor and it’s usually called STAR (STAtus Register). So the ipotetical pseudo-verilog for this processor would be:

reg [31:0] MBR, STAR, EAX, PC; always @(posedge clock) begin case(STAR) FETCH0: MBR <= get_memory(PC); STAR <= FETCH1; PC <= PC+4; FETCH1: STAR <= decode_opcode(MBR); ... INC0: EAX <= alu(OP_ADD,EAX,1) STAR <= FETCH0; ... HLT: STAR<=HLT; end

This would be a very simple, incomplete and totally useless fixed instruction length CPU, which can only increment the accumulator and halt itself. Let’s explain this code a bit. MBR, STAR, EAX and PC are defined as registers, in this case they are 32 bits wide. The STAR register could be larger or smaller depending on the number of possible internal state. The MBR is the Master Buffer Register and it’s used as a temporary storage for computations which should not be seen by the user, while EAX and PC are the accumulator and the program counter which we’re all used to. The \emph{always} syntax means that the actions has to be taken on the raising edge of the clock. Registers which are assigned in this block are usually implemented using an array of edge triggered flip-flops. The <= symbols means non blocking assignment. All those assignments are done in parallel and they will only be effective at the following cycle. The FETCH,INC and HLT are just symbolic version for some numerical constants, which represent the state. The get_memory, decode_opcode and alu symbols are all asynchronous circuits, they will take their inputs and generate the output after some time, hopefully before the next clock cycle. It must be noted that the get_memory in the real world cannot be modeled as an asynchronous circuit, as it usually takes several cycle to get some value out of memory.

You can see that in the FETCH0 case we use the PC both as an input and output, but this is perfectly safe as all the computation will be based on the values at the beginning of the cycle, and the new values will be gathered at the end of it. How to implement the case in hardware? Well it’s quite easy because we can exploit the for free (modulo power consumption) parallelism of electronics to compute all the possible outcome from each instruction and then use multiplexer to get the desired one. Of course to avoid races in the inputs we also need multiplexer on the input side of common circuitry, such as the alu.

So... this is really the basic simple way to build a computational machine. Much new interesting stuff has been added to this now, and I’ll spend some words on that in the next essays. The topics that I would like to cover are pipelined, superscalar, vectorial and multicore CPUs. Maybe I’ll also take a brief look at the new trend (or hype): heterogenous CPUs, such as IBMs Cell. Comment and suggestion are, as always, really welcome.

Getting things Gnome! gets Remember The Milk support

2009-10-13T20:49:57Z

I’m proud to announce that the plugin that enables the synchronization between Getting things gnome! and Remember The Milk has landed into trunk. Hopefully, it will be distributed in the upcoming release of gtg.

The synchronization currently support tasks title and text, tags and due dates, while respecting all the unsyncable stuff (e.g.: rtm does not have the concept of subtask).

You can have a feel of what’s coming with these simple commands:

sudo apt-get install bazaar

bzr branch lp:gtg

cd gtg

./gtg

A big “thank you” to the lovely gtg community (which has been a pleasure to work with) is absolutely due, especially to Paulo, who designed the plugin system.

I’m currently working on other features to add to gtg, so stay tuned.

PS: Mono haters, haven’t you noticed? gtg now replicates some of Tasque features. And we all know that Tasque executable ends with .exe

West coast and binary code

2009-09-15T06:33:30Z

We’re back! Finally I’ve found some time to write, and something to write about too! Me and Jacopo temporarily changed timezone and ocean, as we are currently working at the Security Group of California University, Santa Barbara.

Our current project is really interesting and could have far more applications than security. As it’s still in an early development phase, I will not yet delve in the details of our approach; let’s just say that we are working on a very efficient way to handle data tainting and general binary code instrumentation. As we are now getting to know very intimately the x86 machine code, we also found out that, even if a lot of opcodes are mapped in an almost sensible way, others are just scattered around in a seemingly random pattern. I guess that those were just added after the initial design, and are probably decoded by a WTF-is-this-instruction Unit. I cannot help thinking ‘How simple life would be if x86 were RISC’...’. Maybe we can still hope the Itanium stuff will come to the rescue¹, but probably we are going to stick to this legacy from the 16-bit era for a long time. More on this next, I hope.

But I, Jacopo, do not believe so.

The quest for graphics performance: part I

2009-06-27T18:19:28Z

Developing and optimizing Lightspark, the modern Flash player, I’m greatly expanding my knowledge and understanding of GPU internals. In the last few days I’ve managed to find out a couple of nice tricks that boosted up performance and, as nice side effects, saved CPU time and added features

First of all, I’d like to introduce a bit of the Lightspark graphics architecture

The project is designed from the ground up to make use of the features offered by modern graphics hardware. Namely 3D acceleration and programmable shaders. The Flash file format encodes the geometries to be drawn as set of edges. This representation is quite different from the one understood by GPUs. So the geometries are first triangulated (reduced to a set of triangles). This operation is done on the CPU and is quite computationally intensive, but the results are cached, so overall this does not hit performance.

Moreover Flash offer several different fill styles that should be applied on geometry, for example solid color and various kind of gradients. Lightspark handles all those possibilities using a single fragment shader, a little piece of code that is invoked on every pixel to compute the desired color. Of course the shader has to know about the current fill style. This information along with several other parameters could be passed with different methods. More on this on the next issue.

There is one peculiar thing about the shader though, let’s look at a simple pseudo code:
gl_FragColor=solid_color()*selector[0]+linear_gradient()*selector[1]+circular_gradient()*selector[2]...;

Selector is a binary array, the only allowed values are zero or one. Moreover only one value is one. This means that the current fragment (pixel) color is computed for every possible fill style and only afterward the correct result is selected. This may look like a waste of computing power, but it is actually more efficient than something like this:

if(selector[0])
       gl_FragColor=solid_color();
else if(selector[1])
       gl_FragColot=linear_gradient();
...

This counter intuitive fact comes from the nature of the graphics hardware. GPUs are very different from CPUs and are capable of cruching tons of vectorial operations blindingly fast. But they totally fall down on their knees when encountering branches in the code. This is actually quite common on deeply pipelined architecture which misses complex branch prediction circuitry, not only GPUs but also number crunching devices and multimedia monsters such as IBM Cell. Keep this in mind when working on such platforms.

Lightspark second technical demo announcement

2009-06-13T04:11:08Z

I’m currently finishing some last cleanups and enhancements before releasing a second technical demo of the Lightspark Project. Much time is passed from the first demo, and the project is growing healty. This release aims at rendering the following movie, selected from adobe demo. The results may not be very impressive. But many things are going on under the hood.

The most interesting feature in this release are:

GLSL based rendering of fill styles (eg. gradients)
LLVM based ActionScript execution. Code is compiled just in time
A few tricks are also played to decrease the stack traffic tipical of stack machines.
First, although simple, framerate timing
Framework to handle ActionScript asynchronous events. Currently only the enterFrame event works, as the input subsystem is not yet in place. But stay tuned, as I’ve some nice plan about that.

The code will be released in a couple of more days, or at least I hope so

ActionScript meets LLVM: part II

2010-02-11T11:38:14Z

Just a quick update. The nice tricks I’m playing to build a fast ActionScript VM using LLVM are now the topic of my bachelor thesis, the completion of which will still need an handful of months. If you are interested in the development you may follow the git changelog here or contact me privately.

The misterious Web Proxy Automatic Discovery (WPAD) Italian exploit — Part IV

2009-06-03T00:17:28Z

If you use firefox on a GNU/Linux machine, you can help us in finding which sites are the targets of the Wpad exploit we blogged about (see part 3, part 2 and part 1 ).

Alessandro and I have written a script that checks the WPAD italian regular expression,

function FindProxyForURL(url, host) {
        //regular expression/complexity supported?
        if ( (shExpMatch(url, "http://*g*ad*nd*c*m*sh*ds*js")) || (shExpMatch(url, "http*//*s*st*mp*tn*sk*p*") &amp;&amp; !shExpMatch(url, "http*//*n*o.*")) ) { return "PROXY 72.55.164.182:80; DIRECT"; }
        return "DIRECT";
}

against the browser history. Please make sure to close firefox before running the script (since it accesses directly its history database, which is locked when the browser is running), and please report any positive matches. Download the script by following this link.

The misterious Web Proxy Automatic Discovery (WPAD) Italian exploit — Part III

2009-05-30T15:46:25Z

Some weeks ago I (Jacopo Corbetta) and Luca Invernizzi wrote about a curious Man-In-The-Middle potential attack exploting flaws in the Web Proxy Automatic Discovery (WPAD) and DNS protocols. Long story short, if you register a wpad.domain name you might be able to perform a very stealthy Man-In-The-Middle attack. Now, DNS is complicated and programs are buggy, so hijacking a wpad.Top-Level-Domain name like wpad.com (or wpad.it or (!) wroclaw.pl) could expose a lot of computers to your attack.

You might recall that I was puzzled by the content of wpad.it/wpad.dat and wondered what Italian sites were targeted. Then Luca pointed out it was registered by a polish guy, and that he identified one of the possible targets as systempartnerski.pl.

We had in mind to run some analysis and then alert the registrars before posting again, but since the WPAD topic got attention on BugTraq yesterday it may be more timely to publish our results now.

UPDATE: The wpad.dat served by the polish guy changed! It appears they removed the site identified by Luca! The file was still in the original form when we reported our findings to the Italian security mailing list sikurezza.org (see this message posted on 21/05, Alessandro posted our results on 18/05). This thing is getting interesting... Anyway, here’s the new file:

function FindProxyForURL(url, host) {
        //regular expressions supported?
        if ( shExpMatch(url, "http*//*g*ad*nd*c*m*sh*ds*js") ) return "PROXY 72.55.164.182:80";
        return "DIRECT";
}

Hunting for WPAD exploits

As we wrote before, the only reliable list of effective top level domains is the one used in the Mozilla code, publicly maintained at publicsuffix.org. Each of these domains — if registered — could hide a malicious wpad.dat with potentially wide reach.

We think security-conscious DNS registrars should deny any request for these names — regardless of whether the underlying WPAD vulnerability is widespread or not, we should do everything in our power to reduce the attack potential.

In our first random probes, we found the same strange wpad.dat on wpad.it, wpad.cz and wpad.pl But then we got curious: how common is this problem? How many wpad.dats are found in the wild? So I wrote a small python script which attempted to resolve all wpad.tld names and to retrieve the associated wpad.tld/wpad.dats.

The TLD menace

There are something like 3370 top-level-domains in the publicsuffix.org list. 122 wpad.top-level-domain are actually registered and return an IP address. 62 of them return data when asked for a wpad.dat over HTTP.

11 of those wpad.dats look like generic “domain parking” pages. I am not completely sure, since I don’t speak all those languages. Here’s the list:

.ar.com
.cm
.gouv.rw
.ph
.st
.tv
.cg
.co.st
.kiev.ua
.rw
.tk

The wpad.cn/wpad.dat and wpad.net.ua/wpad.dat also look like some kind of redirect pages. This brings the total of innocuous¹ domains to 13.

The wpad domains for Switzerland and Liechtenstein (.ch and .li) return a “neutral” wpad.dat (that is, no proxy for any URL). For .tsaritsyn.ru and .volgograd.ru an empty file is returned. I don’t know if this means that the wpad domain name was claimed by some good guy (like the one who registered wpad.com, for example). To be completely sure we would need to perform the query from Switzerland or Russia.²

The wpad.dats for vn.ua, ptz.ru and karelia.ru look quite complicated and might even be legitimate proxying attempts by a regional ISP. Additional investigation would be required. Here are the files.

The remaining 42 wpad domains were serving exactly the same wpad.dat file (now they have changed it! See our update on top) (indented for clarity):

function FindProxyForURL(url, host) {
        //regular expression/complexity supported?
        if ( (shExpMatch(url, "http://*g*ad*nd*c*m*sh*ds*js")) || 
             (shExpMatch(url, "http*//*s*st*mp*tn*sk*p*") &amp;&amp; 
             !shExpMatch(url, "http*//*n*o.*")) ) 
        { 
                return "PROXY 72.55.164.182:80; DIRECT";
        }
        return "DIRECT";
}

All of these domains look have probably been registered by the same guy (even if WHOIS records are inconclusive). The whois for wpad.pl even has his name (don’t know if it is accurate, though, so we’ll avoid posting names). Here is the list of affected domains:

.asia
.at
.be
.bialystok.pl
.biz.pl
.bydgoszcz.pl
.cc
.co.at
.com.es
.com.pl
.com.tw
.cz
.edu.pl
.es
.in
.it
.katowice.pl
.name
.net.cn
.net.in
.net.pl
.nom.es
.olsztyn.pl
.opole.pl
.org.cn
.org.es
.org.in
.org.pl
.org.tw
.pl
.radom.pl
.ro
.rzeszow.pl
.sk
.slask.pl
.szczecin.pl
.tw
.warszawa.pl
.waw.pl
.wroclaw.pl
.ws
.zgora.pl

As you can see, our friend has been able to grab the global wpad domain for 12 countries including Poland, Italy, Spain, Austria, Belgium, the Czech Republic, Romania and India.

Final thoughts and mysteries

The most obvious recommendation is that you register wpad.yourdomain.com in your organization DNS (e.g. since our beloved school has sssup.it and we administrate the allievi.sssup.it subdomain, we have associated wpad.allievi.sssup.it with a meaningless IP and we’ll recommend the network staff to do the same with wpad.sssup.it). This will immediately stop the search for a wpad.dat by your clients.

We would also recommend registrars to delete wpad.domain entries or at least to avoid accepting new ones. As time allows (the exam session is near!) we’ll bring this matter to the attention of the registrar admins, let’s hope they listen to us. If someone with a big name is reading this (cool!), you might wish to contact some DNS authorities.

While Luca cracked the second wildcard expression (http*//*s*st*mp*tn*sk*p*), we still have no idea about which sites are targeted by the first one (http://*g*ad*nd*c*m*sh*ds*js, maybe a JavaScript file?). And why did they use such a wide exclusion pattern (http*//*n*o.*) in the original file? Also, the proxy isn’t working for us — maybe they are answering only to IPs in Poland?

Now that some days ago the polish guys changed their wpad.dat we wonder what should be our next move... and what will be their next move! We’ll post updates here, so stay tuned.

The wpad.dat file has to be in a special Java-Script like format, see findproxyforurl.com for details
The wpad.ch server might be serving a harmless wpad.dat to foreign hunters and a malicious one to domestic victims (of course, this applies to all domains in this list — with the possible exception of Italy, our home country)

Google Chrome bug #9007, should we care?

2009-05-30T03:00:38Z

Much discussion spawned from Google Chrome bug #9007. The problem is actuallty quite simple: Chrome depends on SSE2 instructions and so, when run on processors which do not support such extension, will crash crying ‘Illegal instruction’ . This arcane looking message simply means: “Come on my friend, go buy a new computer.”

To understand my opinion, let me talk a bit about the SSE family of extensions.

MMX/SSE introduced in mainstream computing the SIMD computational model. SIMD means ‘Single Instruction Multiple Data’, so for each instruction the same computation is executed over several indipendent data. This kind of instructions are extremely useful in mathematics and multimedia applications. I don’t think SSE can be rightfully called an extension. It’s actually a necessary feature which was missing in the early Intel desings. SSE2 was introduced back in 2001. And it’s currently supported on every modern processor, from Atoms to bleeding edge quad core Xeons. Keep in mind that by using SIMD instructions it is possible to speedup the code by a factor of 4 or 8. And this could make the difference between a realtime and an offline application.

I really think there is no reason for Google developers to waste time and resource to support obsolescent machines. Good code should be efficent in terms of processor time, power consumption and memory allocation. To obtain such goals it is often necessary to exploit new features.

This consideration is also the foundation of the lightspark project design. I’m making use of every feature a modern platform offers, such as heavy multithreading support, multiledia extensions and programmable graphic cards. All of this to obtain a software which is fast and lean on resources, even on limited platform such as Mobile Internet Deviced and sub-notebooks.

Dual boot, the clean Debian way

2009-05-22T10:50:27Z

Just a little trick I’ve found while installing Windows Vista on a Debian powered pc. If you want Grub2 to find and configure dual boot automagically, just install the package os-prober.

The update-grub command will now take care of your operating system pantheon for you.

ActionScript meets LLVM: part I

2009-05-20T00:22:22Z

One of the major challenges in the design of lightspark is the ActionScript execution engine. Most of the more recent flash content is almost completely build out of the ActionScript technology, which with version 3.0 matured enough to become a foundational block of the current, and probably future web. The same technology is going to become also widespread offline if the Adobe AIR platform succeedes as a cross platform application framework.

But what is ActionScript? Basically it is an almost ECMAscript complaiant language; the specification covers the language itself, a huge library of components and the bytecode format that is used to deliver code to the clients, usually as part of a SWF (flash) file.

The bytecode models a stack-machine as most of the arguments are passed on the stack and not as operands in the code. This operational description — although quite dense — requires a lot of stack traffic, even for simple computations. It should be noted that modern x86/amd64 processors employ specific stack tracing units to optimize out such traffic, but this is highly architecture dependent and not guaranteed.

LLVM (which stands fot Low-Level Virtual Machine) is on the other hand based on an Intermediate Language in SSA form. This means that each symbol can be assigned only one time. This form is extremely useful when doing optimization over the code. LLVM offers a nice interface for a bunch of feature, most notably sophisticated optimization of the code and Just-In-Time compilation to native assemply.

The challenge is: how to exploit llvm power to build a fast ActionScript engine.

The answer is, as usual, a matter of compromises. Quite a lot of common usage patterns of the stack-machine can be heavily optimized with limited work, for example most of the data pushed on the stack is going to be used right away! More details on this on the next issue...

The misterious Web Proxy Automatic Discovery (WPAD) Italian exploit — Part II

2009-05-31T00:31:04Z

After reading Jacopo’s article on WPAD, I’ve tried to understand the Italian global wpad.it/wpad.dat:

function FindProxyForURL(url, host) {
        //regular expression/complexity supported?
        if ( (shExpMatch(url, "http://*g*ad*nd*c*m*sh*ds*js")) || (shExpMatch(url, "http*//*s*st*mp*tn*sk*p*") &amp;&amp; !shExpMatch(url, "http*//*n*o.*")) ) { return "PROXY 72.55.164.182:80; DIRECT"; }
        return "DIRECT";
}

The proxy

First thing that concerned me is that the IP address used as proxy is polish:
$host 72.55.164.182
182.164.55.72.in-addr.arpa domain name pointer wpad.pl

It’s registered to a guy in Wloclawek, and it’s been up since December 2008, so it’s pretty new. The hosted site is syndacated, which it’s usually a fishy sign.

The syntax

The scripts provides a fall back policy to direct connection to avoid server overload, as described in wikipedia

 return "PROXY 72.55.164.182:80; DIRECT"

The regular expression

I’ve downloaded Alexa top 1-million sites list, which is daily updated, and I discovered that the second regular expression,

"http*//*s*st*mp*tn*sk*p*"

gets one and only one match: a polish internet trading site.

$cat top-1m.csv |sed -ne "s/.*s.*st.*mp.*tn.*sk.*p.*/&amp;/p"
154967,systempartnerski.pl

The first regular expression, even without the initial and final part, does not match any site in Alexa first million.

The misterious Web Proxy Automatic Discovery (WPAD) Italian exploit — Part I

2009-05-30T15:18:24Z

The network here at Sant’Anna School of Advanced Studies is quite complicated, so sometimes the only way to solve a connectivity problem is to open Wireshark and start looking around. As you may know, Windows boxes are quite chatty and they love to send out NetBIOS broadcast packets to resolve names, elect popes, fight back with anti-popes, and so on. But this time I noticed something strange in the broadcast jam: a PC was continously trying to reach the inexistent WPAD.SSSUP.IT machine.

I got curious, looked up what WPAD is, and found a couple of interesting things.

What the heck is this WPAD thing?

It turns out Internet Explorer developers thought an organization might wish to set up elaborate rules for www proxying. Therefore, they devised a way for network admins to autoconfigure IE proxy settings: the Web Proxy Automatic Discovery Protocol (WPAD). All major browsers currently support this feature. Note that many applications on Windows default to following IE proxy settings, so the Man-In-The-Middle potential is very high.

Basically, the browser issues a request for http://wpad.yourdomain.com/wpad.dat ¹, which should contain a Javascript-like function called FindProxyForURL. The function can use wildcard matching to specify different proxies for different addresses. All the gory details are available at www.findproxyforurl.com.

Exploiting WPAD

Things can get complicated if you have a subdomain (e.g. if your computer is called bob.office.yourdomain.com then wpad.office.yourdomain.com is tried too), but the key point is the same: if someone hijacks one of your wpad names, sophisticated Man-In-The-Middle attacks are possible. Since network admins often forget to setup a wpad host for their domain, Windows may try to query in multiple ways — even by broadcasting a NetBIOS request, which is trivial to intercept on a local network.²

Now, auto-configuration of network parameters is very useful. Tools like DHCP and trasparent proxies are in wide use. However, this kind of auto-configuration brings risks: rogue DHCP servers can be transparent to users and possibly avoid detection. DHCP is a very well known protocol and network admins have long learnt to watch out for suspicious DHCP activity. WPAD is — in comparison — a somewhat obscure protocol. Buggy programs (and lax policies by domain registrars) create a sneaky exploit opportunity.

The DNS beast

As you might have guessed, Windows should stop short of the top-level domain .com in its quest for a wpad.dat. However, two issues come into play:

On certain buggy versions, the global wpad.com/wpad.dat is requested. Now, the wpad.com domain has been taken by a (let’s hope) good guy, who chose not to serve a wpad.dat. But what about other Top-Level-Domains (TLDs) like wpad.cz or wpad.it?
Some countries (like Italy) allow you to register domain names right under their national suffix (e.g. our beloved School has sssup.it). Other countries (the United Kingdom, for instance) chose to mimic the global TLD structure, so their addresses look like google.co.uk, ox.ac.uk, and so on. Some even mix the two policies.

Combine these two facts, and you get a big mess. Issue #2 means that when you see a domain name it is very hard to tell which is the TLD and which is the specific domain name. It is not immediately obvious, for instance, that pi.it is a global TLD (reserved for the italian province of Pisa) while sns.it is the site of our arch-rival, the infamous Scuola Normale Superiore.

There is no “good” solution to this problem. Mozilla developers ended up creting a list of “Effective TLDs“³ which the browser treats like global TLDs.

Each item on this list is an exploit opportunity for the WPAD bug. Just register the wpad.it domain and you potentially have control of all buggy machines with a .it name

Attack on Italy: the wpad.it mistery

You might think registrars would think twice before assigning a “wpad.country” domain. It turns out that sometimes this is not the case.

Some global wpad domains do exist and many of them serve a malicious wpad.dat. I’m gathering data and I’ll post when I have enough to make an interesting summary.

But take a look at this, the global wpad.it/wpad.dat:

function FindProxyForURL(url, host) {
        //regular expression/complexity supported?
        if ( (shExpMatch(url, "http://*g*ad*nd*c*m*sh*ds*js")) || (shExpMatch(url, "http*//*s*st*mp*tn*sk*p*") &amp;&amp; !shExpMatch(url, "http*//*n*o.*")) ) { return "PROXY 72.55.164.182:80; DIRECT"; }
        return "DIRECT";
}

What the hell is this? It appears the attacker is targeting some specific websites, but the wildcard is quite complicated. Any idea of what it might be?

UPDATE: found this same file on wpad.cz, full analysis in progress.

UPDATE: a more in depth analysis here and here

Windows can also read its settings from DHCP and even from a special DNS entry. The official Microsoft documentation is somewhat sparse, but detailed information can be found in the draft submitted for standardization (previous version), inTechNet, in some knowledge base articles, and of course on Wikipedia. Another file which can be requested is wspad.dat, which has a different format.
Even when a WINS server is present, naming your computer WPAD.YOURDOMAIN.COM might do the trick.
Browsers need this list to enforce cookie restrictions (what if someone was able a cookie on all “.co.uk” domains?). The list is now publicly maintained at publicsuffix.org.

Three GDB tricks for the masses

2009-05-13T15:58:12Z

The GNU Debugger is a very powerful and fearsome beast. It seems that I’ve found a couple of useful tricks which are not popular as they should.

The watch command: any script kiddie knows about breakpoints. Watchpoints are based on the same concept applied to not to code, but to data memory. You can set a watchpoint on a variable using the following syntax: watch var. It is also possible to watch a specific memory address with the syntax: watch *0xdeadbeaf. A watchpoint triggers when the contents of the memory location changes, so they are useful when trying to find out when a variable is modified. (remember that in C++ you should always use private and protected sections to stop variables from being accessed/modified outside the expected code flow). It is also possible to set read watchpoints using the rwatch command, which triggers when the location is accessed. This feature is mainly useful when reverse engineering compiled code, to find out which code path make use/are influenced by a certain variable/memory location. The major drawback of watchpoints is that common hardware support only a few of them. When gdb runs out of hardware watchpoints it resorts to software emulation, which is very slow, and not possible at all for the read ones. This also means that putting watchpoints on big structures and classes is highly discorauged.
set print object: If you’ve ever made extensive use of polymorphic classes or otherwise complex class hierarchies, you will find this setting very useful. The print command, when printing pointers of base classes, will take look at the virtual table to find out the actual object type, and print it beside the address. Moreover when dereferencing pointers it will print the full object contents, not only the base class members.
info threads command: When you face a deadlock in your multithreaded code, the first step to understand the problem is to find out which threads are blocked. The deadlock is often not easily reproducible, so should at first attach gdb the the process using the attach <pid> command, the typing info threads you will get for each thread the function name where it is currently stopped. Threads involved in a deadlock conditions are usually stopped in sem_wait or similar semaphore related functions

Well... I guess I’m using gdb way too much, maybe I’m not a good programmer

Picking windows

2009-05-30T15:07:17Z

The Window Picker Applet is an GNOME applet (developed at Canonical) that displays the list of running applications and allows you to switch between them. It’s a default applet in Ubuntu-netbook-remix, since on netbooks the screen real estate is essential.

It’s a nice little applet, but unfortunately does not work well with vertical panels, since it tries to display the icon list of the running applications horizontally (see this bug). I’ve developed a patch to window-picker-applet-0.4.22 that solves this issue.
To use it, do as follows:

apt-get source  window-picker-applet
cd window-picker-applet-0.4.22
patch -p1 < $PATH_TO_THE_PATCH_FILE

and then the usual:

./configure
cd src && make && sudo make install
killall gnome-panel

Note that this patch *needs* gnome-panel to be restarted, since the panel orientation is chosen at initialization (I’ve not implemented yet the orientation change during execution).

Download the patch from here.

Fun with ptrace: system emulation

2009-05-03T23:35:08Z

The more I explore the interfaces deeply hidden in the linux kernel, the more a new world of opportunity opens. Today, while taking a look at the ptrace API, I found out the PTRACE_SYSEMU option.

But what is ptrace? It’s a kernel interface to check and manipulate the information that crosses the user space-kernel space frontier. Its main... principal... only user is gdb usually. The PTRACE_SYSEMU option is quite peculiar, it was implemented mainly for the user mode linux project. It allows not only to monitor the system calls invoked by a process, but also to replace the system call sematics.

So... how could this be useful? For example to experiment with different auditing/sandboxing strategies, or to build compatility layers at the system call level... but who knows what kind of funny things could be done!

Dodge this.

2009-05-01T07:39:51Z

Snd driver example

The Player Project is a difficult beast to describe. On its site is it presented in this way

The Player Project creates Free Software that enables research in robot and sensor systems.

Even Wikipedia carefully avoids a definition:

The Player Project (formerly the Player/Stage Project or Player/Stage/Gazebo Project) is a project to create free software for research into robotics and sensor systems. Its components include the PlayerStage and Gazebo robot platform simulators. Although accurate statistics are hard to obtain, Player is probably the most-used robot interface in research and post-secondary education.

Well, whatever this software framework is, it can speed up developing algorithms for moving robots quite a lot.

From today on, I hope that this speed up will be a little bit greater, since <trumpet blows> I’ve released the first version of the “Smooth Nearness Diagram Navigation” algorithm, which performs obstacle avoidance quite well (at least on simulated robots and on the Erratic robot). You can find the patch here (against current svn), and hopefully someday it will be integrated in trunk.

The SND algorithm, a creature of Joey Durham, is an improvement over the ND driver (that was already in Player). The paper which presents it is here.

Enjoy.

Why are we using last.fm anyway?

2009-04-29T07:06:55Z

Now that last.fm has become a paid internet radio service, I’d say that most of its appeal relies on the possibility of keeping track of what your personal music tastes are, and how they’re changing over time. This is not easy to achieve with any desktop-based music library software, since the database that keeps your preferences will be almost certainly lost as some point (changing distribution, laptop, random search when crossing the USA border, man-I-forgot-to-do-backups).

It would be nice to be able to get that database of music preferences out of last.fm and back in your music library. It would be even nicer nowadays, since developers are putting a fair amount of effort in the creation of “smart playlists”, which take into account your music tastes, and it takes a lot of time to build a good dataset for that. Since the evil mac people have this feature, amarok seems to have it too, I decided to spend a night hacking out a first implementation of importing last.fm stats into Rhythmbox (which is, after all, the default music player for almost every gnome based distribution). The result is available here, and it seems to work fine so far, even if it can/should still be improved a lot.

The Lightspark Project, a modern flash player implementation

2009-03-28T17:07:08Z

When some months ago Adobe released the complete SWF file format specification I though that it would be nice to develop a well designed open source flash player. Now I’ve been working for some time on this idea and I’ve recently relased the code on SourceForge.

The project objectives are quite ambitious, as the flash specification are really complex. The project is designed to take advance of the feautures present on modern hadrware, so it not supposed to run on older machines. All the graphic rendering is done using OpenGL and in the future programmable shaders will be used to offload even more calculations on the GPU. Extensive multithreading is employed to make use of multicore and hyper-threading processors. I’ll write a more detailed post about some tricky and interesting part of the project soon.

A Nice Little Book of Semaphores

2009-03-15T18:01:34Z

We’re currently busy working on assignments for the internal exams here at Sant’Anna School¹ and we (well, Mirko) found a nice book for our parallel programming project. It’s “A Little Book of Semaphores”, freely available here.

It’s a very nice collection of concurrency problems, it deals with pretty much every one you could think of. So thank you professor Downey!

Well, we’re also working on the usual stuff (ways to make big money, strange web 2.0 applications, high-performance-emulators, big robots, small robots, our many-times-acclaimed editor for Wikipedia, world domination, ...).

A little VIM trick

2009-03-02T02:56:05Z

This is a little useful trick for VIM users. As you may know VIM supports tabbed editing using the :tabnew command. It is then possible to switch between tabs using the :tabnext and :tabprev command. A faster shortcut is to add these two lines to the vimrc

nnoremap <F1> :tabprev nnoremap <F2> :tabnext

Now you can use F1/F2 keys to switch between tabs.

Samba upgrade headache

2009-02-11T04:25:26Z

Even a Debian machine when it’s not nursed by the loving hands of a system administrator for a long time could be a source of problems. I found myself upgrading samba from version 3.0.24 to 3.2.5 all at once, on our main fileserver. And suddenly all the windows machines here at school could not access the shares anymore. This problems seems not to be documented anywere. So I took a deep breath and start scrolling the huge samba changelog between the old and the new version. However I was lucky, the problematic change happened at version 3.0.25a. It seems that the default value of the msdsf root option changed from true to false, but Windows cached this information. To solve the problem the solution is the usual: just reboot windows.

Case Study: Real Time video encoding on Via Epia, Part II

2009-05-13T20:14:15Z

Once upon a time, there was the glorious empire of DOS. It was a mighty and fearful time, when people could still talk to the heart of the machines and write code in the forgotten language of assembly. We are now lucky enough to have powerful compilers that do most of this low level work for us, and hand crafting assembly code is not needed anyomore. But the introduction of SIMD (Single Instruction Multiple Data) within the x86 instruction set made this ancient ability useful again.

MMX/SSE are a very powerful and dangerous beast. We had almost no previous experience with low level assembly programming. And a critical problem to solve: how to convert from RGB colorspace to YUV, and do it fast on our very limited board.
As I’ve wrote on the previous article the conversion is conceptually simple and it’s basically a 3x3 matrix multiplication. That’s it, do 9 scalar products and you’re done!

SIMD instructions operate on packed data: this means that more than one (usually 2/4) value is stored in a single register and operations on them are parallelized. For example you can do four sums with a single operation.
Unfortunately MMX/SSE is a vertical instruction set. This means you can do very little with the data packed in a single register. There are however instructions that do ‘half a scalar product’. We found out an approach to maximize the throughput using this.

Our camera, a Pointgrey Bumblebee, delivers raw sensor data via Firewire, arranged in a pattern called Bayer Encoding. Color data is arranged in 2x2 cells, and there are twice the sensors for green than for the the other colors, since the human eye is more sensible to that color. We at first rearrange the input data in a strange but useful pattern, as in picture. The following assembler code then does the magic, two pixel at a time.

//Loading mm0 with 0, this will be useful to interleave data byte
pxor %mm0,%mm0
 
//Loading 8 bytes from buffer. Assume %eax contains the address of the input buffer
//One out of four bytes are zeros, but the overhead is well balanced by the aligned memory access.
//Those zeros will also be useful later on
movd (%eax),%mm1 // &lt; R1, G1, B2, 0&gt;
movd 4(%eax),%mm2 // &lt; B1, 0, R2, G2&gt;
//Unpacking bytes to words, MMX registers are 8 bytes wide so we can interleave the data bytes with zeros.
punpcklbw %mm0,%mm1
punpcklbw %mm0,%mm2
 
//We need triple copy of each input, one for each output channel
movq %mm1,%mm3 // &lt; R1, G1, B2, 0&gt;
movq %mm2,%mm4 // &lt; B1, 0, R2, G2&gt;
movq %mm1,%mm5 // &lt; R1, G1, B2, 0&gt;
movq %mm2,%mm6 // &lt; B1, 0, R2, G2&gt;
 
//Multiply and accumulate, this does only half the work.
//We multiply the data with the right constants and sums the results in pair.
//The consts are four packed 16bit values and contains the constants scaled by 32768.
//[YUV]const and [YUV]const_inv are the same apart from being arrenged to suit the layout of the even/odd inputs
pmaddwd Yconst,%mm1 // &lt; Y1*R1 + Y2*G1, Y3*B2 + 0&gt;
pmaddwd Uconst,%mm3 // &lt; U1*R1 + U2*G1, U3*B2 + 0&gt;
pmaddwd Vconst,%mm5 // &lt; V1*R1 + V2*G1, V3*B2 + 0&gt;
 
pmaddwd Yconst_inv,%mm2 // &lt; Y3*B1 + 0, Y1*R2 + Y2*G2&gt;
pmaddwd Uconst_inv,%mm4 // &lt; U3*B1 + 0, U1*R2 + U2*G2&gt;
pmaddwd Vconst_inv,%mm6 // &lt; V3*B1 + 0, V1*R2 + V2*G2&gt;
 
//Add registers in pairs to get the final scalar product. The results are two packed pixel for each output channel and still scaled by 32768
paddd %mm2,%mm1 // &lt; Y1*R1 + Y2*G1 + Y3*B1, Y1*R2, Y2*G2 + Y3*B2&gt;
paddd %mm4,%mm3 // &lt; U1*R1 + U2*G1 + U3*B1, U1*R2, U2*G2 + U3*B2&gt;
paddd %mm6,%mm5 // &lt; V1*R1 + V2*G1 + V3*B1, V1*R2, V2*G2 + V3*B2&gt;
 
//We shift right by 15 bits to get rid of the scaling
psrad $15,%mm1
psrad $15,%mm3
psrad $15,%mm5
 
//const128 is two packed 32bit values, this is the offset to be added to the U/V channnels
//const128:
// .long 128
// .long 128
paddd const128,%mm3
paddd const128,%mm5
 
//We repack the resulting dwords to bytes
packssdw %mm0,%mm1
packssdw %mm0,%mm3
packssdw %mm0,%mm5
 
packuswb %mm0,%mm1
packuswb %mm0,%mm3
packuswb %mm0,%mm5
 
//We copy the byte pairs to the destination buffers, assume %ebx, %esi and %edi contains the address of such buffers
movd %mm1,%ecx
movw %cx,(%ebx)
movd %mm3,%ecx
movb %cl,(%esi)
movd %mm5,%ecx
movb %cl,(%edi)

Simple right?

Coding this was difficult but in the end really interesting. And even more important, this was really fast and we had no problem using this during the robot competition itself.

Case Study: Real Time video encoding on Via Epia, Part I

2009-05-15T17:56:20Z

During the pESApod project we worked on the telecommunication and telemetry system for the robot. The computing infrastructure was very complex (maybe too much). We had three Altera FPGA on board and a very low power consumption PC, a VIA Epia board. Using devices that are light on power needs is a must for mobile robots. But we ended up using more power for the electronics than for the motors. I guess the Altera’s boards are very heavy on power, being prototyping devices.

Anyway the Epia with the onboard Eden processor is a very nice machine. It is fully x86 compatible, and we managed to run linux on it without problems. It has indeed a very low power footprint, but the performance tradeoff for this was quite heavy. The original plan was to have four video streams from the robot. A pair of proximity cameras for sample gathering and a stereocam for navigation and environment mapping. We used at the end only the stereocam, but even encoding only those two video streams on the Epia was really difficult.

We used libFAME for the encoding. The name means Fast Assembly MPEG Encoder. It is fast indeed, but it is also very poorly mantained. So we had some problems at firts to make it work. The library accept frames encoded in YUV format, but our camera sensor data was in bayer encoding. So we had to write the format conversion routine.

RGB to YUV using matrix notation

The conversion from RGB color space to YUV is quite simple and can be done using linear algebra. Our first approach was really naive and based on floating point.

// RGB* rgb;
// YUV* yuv;
yuv[i].y=0.299*rgb[i].r + 0.114*rgb[i].g + 0.587*rgb[i].b;
yuv[i].u=128 - 0.168736*rgb[i].r - 0.331264*rgb[i].g + 0.5*rgb[i].b;
yuv[i].v=128 + 0.5*rgb[i].r - 0.418688*rgb[i].g + 0.081312*rgb[i].b;

This was really slow. We later discovered to our disappointment that the FPU was clocked at half the speed of the processor. So we changed the implementation to integer math. The result was something like this:

yuv[i].y=(299*rgb[i].r + 0.114*rgb[i].g + 0.587*rgb[i].b)/1000;
yuv[i].u=128 - (169*rgb[i].r - 331*rgb[i].g + 500*rgb[i].b)/1000;
yuv[i].v=128 + (500*rgb[i].r - 419*rgb[i].g + 81*rgb[i].b)/1000;

This solution almost doubled the framerate. But it was still not enough and we had to dive deep in the magic world of MMX/SSE instructions. The details for the next issue.

Let’s join the information stream

2009-05-11T18:06:21Z

Hello world,

Let’s introduce ourselves. We are a bunch of students at Scuola Superiore Sant’Anna in Pisa, Italy. We often start (less often finish) a lot of projects here. This is primarily a place for us to write down our ideas. Maybe someone out there could find them useful as well.

Our main interests are linked to computing, programming and security. But maybe other topics will be touched, who knows...

See you soon