Agentic architectures and workflows

Download slides

In this talk, I dove deep into the world of agents, starting with some history of the term [00:09]. We explored the core properties of agents, like autonomy, social ability, reactiveness, and proactiveness, and how these translate into the agentic systems we’re seeing today [02:36]. I broke down the anatomy of a foundation model-based agent, discussing environments, memory types (procedural, semantic, and episodic), and the role of external tools [04:35], [13:45]. We also looked at how these agents reason and plan, and even briefly touched on the concept of “theory of mind” [30:00], [43:53]. Finally, I walked through some examples from research papers, like generative agents and software developer agents. I wrapped up with a look at the open research challenges, including control, human-agent interfaces, and evaluation [01:05:08].

RAG Engineering

Download slides

In this talk, I gave a basic rundown of Retrieval-Augmented Generation (RAG). I covered why RAG is needed, the main concepts, advanced retrieval, and RAG patterns, and touched on some applications in software engineering. I broke down the pre-processing, retrieval (both sparse and dense), and post-processing [07:47] phases of RAG. We also dove into advanced retrieval concepts like contextual retrieval and self-reflective RAG [37:37]. In the end, we also looked at the limitations and productionizing challenges of RAG.

As this field is rapidly evolving, some of the work presented here may already be outdated after just five months. But my goal was to provide everyone with a solid understanding of the fundamentals so they can do their own research and figure out what components they need to focus on for their use cases. Based on the feedback I’m getting after the event, I think the attempt was successful.

I just realized I’ve been using the Steam Deck for two years, and this is a good time to reflect on my experience. This post is a mixture of thoughts about the device itself and a few notes I realized along the way about my gaming habits.

• Stream deck is an excellent combination of hardware and software. The device is pretty solid and feels comfortable in the hands. I like the touchpads for navigating in desktop mode and some games requiring mouse input. The back buttons are pretty handy for mapping combo moves. However, I’m yet to master the Gyro controls. The Arch Linux-based SteamOS nicely complements Steam Deck’s hardware capabilities.
• The options are not limited to my Steam library at all. The open-source community has made GoG, Epic, and other libraries available on Steam Deck via projects like Heroic, Lutris, Junk-Store, and non-Steam launchers. You can get a non-steam game to work without much hassle using any of these tools most of the time. On a rare occasion, if things are not working, upgrading the proton-ge version will fix things. I’ve had much success with the Heroic launcher, and now I have moved to the non-steam-launcher. This also means I don’t have to spend too much time playing games that are too difficult or don’t spark joy. I have many options to enjoy and genres to discover.
• Steam deck helps me reduce my gaming backlog. With its flexibility to install games from multiple libraries, mobility, and ability to play games in short bursts, I’ve completed many games in the last two years. For example, I had fun playing and completing the main storyline/missions of Tomb Raider (2013), Rise of the Tomb Raider, Middle-earth: Shadow of Mordor, Assasin’s Creed Origins, Lara Croft and the Temple of Osiris, Blacksad: Under the Skin, Tell Me Why, Haven Park, and Warhammer: Vermintide 2.
• I usually have a few games belonging to two main categories installed on SD, which I play on rotation. The first category is grindy games that do not require a lot of brain power. These games can usually be played on exhausted weeknights, mostly with muscle memory. (Warframe used to be my go-to choice for this. Now, Monster Hunter Rise has taken its place. Rocket League and Fall Guys were also enjoyable for some time.) The next category is single-player games that have advanced storylines, complicated controls and/or require solving puzzles or looking up guides to make progress. (I put Control, Prey, Horizon Zero Dawn, Dishonored, etc. into this category) These need more considerable time investment and more attention. So I play them on weekends or holidays if I can.
• I like RPGs, specifically those with a medieval/fantasy flavour. Skyrim, Assassin’s Creed Origins, Dragon Age: Inquisition, Outward, Bioshock Infinite, and Thief are some of my favourites.
• I can’t really get into some games, even after multiple tries, such as Dark Souls 3 and the Metro series.
• Steam Deck Desktop mode is a lifesaver for quick tasks that require a Linux machine. If the task can be completed with a flatpak application, a podman container, or by simply SSHing to a server, Steam Deck can help. A wireless keyboard/mouse and an external monitor will also be helpful in such cases.

In conclusion, Steam Deck is one of those gadgets I thoroughly enjoyed using for many hours. It’s definitely well worth the price.

After looking around for what’s available, I narrowed down my choices to Kobo Elipsa 2E, Boox Note Air 2, and Kindle Scribe. A few weeks ago, with a heavy discount from Kindle, I decided to go with the Scribe. After using it mostly for academic reading (and some brief recreational reading) I decided to write down my thoughts about the device.

Good Things

eInk technology has improved a lot. I used to own a classic Nook Reader sometime back. (Read my review of it here) So I knew what I was getting into with the latency and ghosting. These limitations are still there compared to other screen technologies but reduced a lot to become unnoticeable. With the high resolution (300 PPI to be exact) and smooth backlight, the text looks sharp and easy on the eyes.

Good Support for PDFs. We can use the Send-to-kindle service to upload PDFs to the device from a mobile app or from a web browser. They reach the device instantly and render without any problems. Out of close to 50 documents that I read on the Scribe so far, only one failed to render (for some unknown reason). You can also conveniently use the stylus to write notes on the margins of the PDFs that are uploaded this way.

Build quality is great. The device has a matte metallic finish and feels sturdy in the hands. (I ended up getting a folio case to protect the device while on the go, just in case.) The screen real estate is big enough to comfortably read a two-column PDF without using any zoom.

Bad Things

Lack of RSS, Libby, or Pocket support. I heavily use the Libby (formerly OverDrive) app to borrow books from public libraries and university libraries. I also use the Pocket service to save long articles from the web and read them later. While Kobo devices support these services natively in their eReaders, Kindle Scribe supports neither. It’s a bummer because both of these are free services that I use a lot to satiate my need for reading. As a rudimentary workaround for the lack of Pocket integration, and for checking RSS feeds I use Scribe’s built-in web browser which brings me to the next point.

Lack of Unicode Support. One of the first things I do with any device that I get hands-on is check its Unicode compatibility (as you can see here and here). Unfortunately, Kindle Scribe gets zero points here. I checked with a few non-roman alphabets in Scribe’s browser and all of them appear as small boxes. Come on, Amazon! It’s 2023! I understand the need to keep resource usage minimal and therefore keep the browser features limited. But, Unicode is a basic necessity in a web browser at this point!

Multi-tasking Support. A basic use case when doing intensive reading is taking detailed notes about what you just read. Although Kindle Scribe has all the primary ingredients to make that happen, the user experience is not well implemented. There is reading support for all sorts of eBook formats. There is note-taking support with a well-made stylus and notebooks of different templates. However, to get to a notebook while reading, you have to close the currently reading document and click through multiple screens. You may lose the train of thought by the time that you get to the place to write after navigating the not-so-fast UI.

No two-way cloud syncing. You can currently send PDFs to the device, and e-mail back the PDF with notes and highlights when you’re done with reading on the device. However, if you’re ingesting a document over a long period you need to avoid the hassle of sending the marked document back and forth. Proper syncing of documents and notebooks to a cloud service such as Google or Dropbox is a must-have feature these days.

In conclusion, I’m enjoying Kindle Scribe so far, and finding great use of it. All the drawbacks that I mentioned above can be fixed via software updates. I’m hoping that these get fixed in future over-the-air updates. However, if the lack of these features is a dealbreaker, you may find eInk tablets that run on Android (from vendors like Boox) to be useful.

I visited Buenos Aires, Argentina in May 2017, and it was very different from the cities I have visited earlier. I stayed in the old barrio of San Telmo but was able to visit few other barrios such as Recoleta, San Nicolás, Monserrat, Puerto Madero, and La Boca.

In the San Telmo neighborhood, you can walk along the cobblestone streets admiring the architecture from the Spanish colonial times. The area has beautiful large houses, or ‘Quintas’, built for elite families in the 17th century. Due to a yellow fever epidemic in 1871, the upper classes have left the area and moved to Recoleta. So these houses were converted to Conventillos which were rented to poor European immigrant workers. It is said that in the central courtyards of these overcrowded urban housing schemes Tango was born. Pasaje Defensa, or the house of the Ezeiza family, is one such house we got to visit. Calle Defensa from Plaza de Mayo to Plaza Dorrego is closed to motor traffic on Sundays and turns into a flea market. You can walk along the side streets enjoying the tango dancers, antique shops, and hand-made jewelry vendors.

Other interesting places we visited in San Telmo are the Mafalda monument (a tribute to the famous cartoon character) and Mercado de San Telmo (the indoor market). The best meat empanadas I had were at a small shop inside the market (200 pesos) and at a bakery (Panadería Confitería) called ‘La Nueva Independencia’ (180 pesos). For meat-based fast food like Chorizo or Bondiola sandwiches, I recommend Nuestra Parilla near the market.

In the Monserrat barrio, the main attraction was the Plaza de Mayo named after the May 1810 revolution which led to Argentina’s independence from Spain. A bit north of the plaza is the cultural center where we attended a nice concert by the National tango orchestra. The admission is free for this. But you have to get tickets during the day.

If you walk few blocks northwest of the Plaza de Mayo, you reach the Obelisco, the famous landmark. There was a big protest against wage limits going on here when we visited. But with the firecrackers, drums, and chants, it appeared more like a carnival rather than a protest. Apparently, this kind of protests is a very common thing in downtown Buenos Aires.

We took a 6-pesos bus ride from San Telmo to arrive in the La Boca barrio. This neighborhood has got the name because it is at the mouth (‘Boca’) of the Matanza-Riachuelo River. We visited La Bombonera, the home ground of the famous Boca Juniors football club. Then we walked through the Caminito, with the colorful houses and local artists selling their paintings on both sides of the road. With their murals everywhere you will soon understand how important are Evita (Eva Perón), Carlos Gardel, and Maradona as cultural icons in Argentina.

Puerto Madero is a failed port project by the local businessman Eduardo Madero in the 1880s which was redeveloped in the 1990s to have luxury hotels, apartment towers, offices and universities along the waterfront. Highlights for me in the area were Presidente Sarmiento museum ship and Puente de la Mujer (‘Women’s Bridge’) designed by the famous architect Santiago Calatrava to resemble a couple dancing the tango.

Travel Tip: Although I did not come across any problems walking at night in San Telmo or Monserrat, I heard one of my colleagues got robbed using the notorious white-paint-squirt-on-bag trick. The locals advised us not to go off the well-known tourist routes in La Boca and San Telmo. It will be very useful if you can speak Spanish when you walk around.

Popular Local Beer: Quilmes

Photo Credits: Eduardo Coronado-Montoya

Here are some articles I found useful to answer the questions I had after visiting this interesting city.

• Why do Argentina, and South America in general, have had so many political and economical problems in the recent past?
• How did the Matanza-Riachuelo become one of the most polluted rivers in the world?

I visited Tokyo, Japan in March 2017. It’s a huge, beautiful city and you cannot cover everything  awesome in few days. I spent 4 nights in Tokyo and 5 nights in Kamakura.

Travel Tip: Tokyo has a great train network. But they are operated by different companies like JR, Tokyo Metro, and Toei Subway. Make sure you buy a prepaid IC card like PASMO or Suica at the airport to avoid wasting time at ticket booths every time you transfer between lines. Also, make sure you are looking at the correct train network map when you’re planning the trip.

In Tokyo, I stayed in a hostel near the Asakusabashi station because it’s in the center of several attractions such as Asakusa, Ueno, Akihabara and Ryōgoku. In Asakusa, I visted the Sensō-ji Temple. It is a colorful, ancient, Buddhist temple.

If you’re a fan of anime, manga, board games and electronics, you should visit Akihabara, the geek town of Tokyo. In Shibuya, you’ll find the busiest intersection in the world, the Shibuya crossing, and the Hachiko memorial.

Edo-Tokyo museum, situated near the Ryogoku station, provided me with a concise view of the Edo period and the recent history of Tokyo. The kids will love the hands-on exhibits and photo-ops as well. There are free traditional magic shows and other cultural events in the weekends as well.

If you are interested in a more in-depth tour covering the history of Japan and Buddhism in Asia, make sure you check out the Asian gallery of Tokyo National Museum, in Ueno. The highlight for me was the large collection of Buddhist sculptures from the Gandhara, Mathura and Gupta periods. Tip for the history buffs: Ueno area is the home to several other museums as well, National Museum of Western Art, National Science Museum, and the Shitamachi Museum.

In Kamakura area, the highlight for me was the Great Buddha Statue (Daibutsu) at the Kotokuin temple. I also visited the Hase-kannon Temple and the Hachiman-gu Shrine in Kamakura. In Japan, shrines are built to worship elements from both Shinto and Buddhism, while temples are dedicated to Buddhism.

I tasted some special food items in Tokyo as well. I tried Kibi dango and Ningyo-yaki (snacks made out of rice flour and redbean paste) at the Nakamise shopping street in Asakusa. Then there were delicious sushi and sashimi at some local restaurants. Okonomiyaki is a savory pancake that can have a mix of ingredients. Kirin, Asahi, and Sapporo are the most famous beer brands in Japan.

Travel Tip: If you want to taste some authentic Japanese food for cheap for one last time before leaving Tokyo, there are some good restaurants at the Narita and Haneda airports.

What We Actually Know About Software Development, and Why We Believe It’s True

by Greg Wilson

This talk was done in 2010. So kind of old, but still relevant. You can find more notes on this here.

What makes useful research in Software Engineering?

by Andreas Zeller

The 3 questions: Was it hard? Is it elegant? And is it useful?

Engineering Complex Software Systems: Challenges, Solutions, and Outlook

This is bit long and consists of 4 parts.

Software Engineering’s Greatest Hits

by Greg Wilson

An update to the talk in 2010

Today is a great day. A pioneering advocate of open source software, Chris DiBona accoladed Sri Lanka for the contribution in GSoC and FOSS, in this interview with TechRadar/LinuxFormat. In his own words:

Every year that goes by we see more people from outside of the US take part [in GSoC]. The US still has a healthy proportion – 250 or something – but it’s amazing to see where people pop up – like Sri Lanka. Even during the civil war we still had Tamil and other Sri Lankan students taking part in the Summer of Code; it’s like, how did it transcend borders in that way in that country? And so, Sri Lanka has always been really interesting to us in ways that even India and China are not.

Here’s basically a very small nation, and if you look at it, there’s a couple of universities that really glommed onto Summer of Code as a way of expanding their curriculum. Think about that. 79 Computer Science students in a small university in a small country in the midst of a civil war, all doing remarkable work. This is the promise of the internet and computer science made flesh.

I think we should all take pride, as students, mentors, well-wishers and finally as Sri Lankans, for contributing with small steps, inspiring people around us, making a mark on the world map for a good thing! I should specially thank few people like Pradeeban Kathiravelu, Kasun Gajasinghe, Thilanka Kaushalya (from University of Moratuwa), Arunoda Susiripala (from University of Kelaniya), Buddie Kurera (from University of Peradeniya), Suranga Nath Kasthurirathne (from IIT) and many who came before them, for pioneering in spreading the word, organizing meetups and going places! AFAICR we did sessions in University of Jaffna, Pera, IESL etc and we used to hang-out in the #gsoc-lk irc channel in FreeNode too.

So let’s pat ourselves on the back once more and get motivated to do better in the coming years, not only in GSoC or Code-In, but in the open source community as well. As Sri Lankans let’s all have a greater impact on realizing “the true promise of the internet and computer science” as Chris DiBona said.

Bravo!

[This post was adapted from an e-mail initially sent to GSoC Sri Lanka Mailing List]

Some Sri Lanka FOSS Facts

• Sri Lanka has a higher per-capita rate of Apache developers than any other nation on earth.
• Sri Lanka is home to The Lanka Software Foundation (LSF) founded by Dr.Sanjiva Weerawarana, which has successfully incubated globally renowned open source projects such as Apache Axis 2 and Sahana Disaster Management Software.
• WSO2 Inc, the open source middleware company backed by Intel Capital, is based in Sri Lanka.

Some more links – just for the kicks – [UPDATED]

• The GSoC community activities in Sri Lanka – A write-up by Pradeeban Kathiravelu posted on March 24, 2012
• For the 7th consecutive year, University of Moratuwa in Sri Lanka has claimed the top spot! – Google Open Source Blog
• Google Summer of Code Stats – Top 10 Schools 2005-2009 by # of Accepted Students
• Other posts from the Google Open Source Blog about Sri Lanka

When a language dies, we don’t know what we loose with that language

This veteran English teacher, with over three decades of experience teaching English in Arabic countries, brings out some interesting points about tests of English language fluency being a barrier for education of non-native speakers.

• English as Liberation – http://bit.ly/1jZkKBX
• Why English Language Teaching in Sri Lanka continues problematic – failures of coordination, consultation and continuity – http://bit.ly/1h8yTdR

• The pull request which started all this 2 days ago – Removed use of gendered pronouns
• Then the merge commit was reverted setting things on fire – Revert “doc: Removed use of gendered pronouns”
• The response from Joyent – The Power of a Pronoun
• Response from StrongLoop – Collaboration not Derision in the Node Community
• One good thing that came out of the discussion – Guidelines for contributing to the project
• A good discussion on HackerNews about the brouhaha – link
• Another great blog post – The Open Source Bikeshed
• Wikipedia article on Parkinson’s law of triviality, also known as bike-shedding which sums up the behavior – link

Thanks Madhuka for the pics.