TLDR: Traditional software engineering is becoming commoditized. Infrastructure, deployment, and development have become incredibly easy thanks to modern tools and platforms. While this means utility software (business applications) will likely be automated away, there’s still room for creativity and personal impact. Even though most software has already been written or will be handled by AI, developers should focus on writing software that matters to them or makes a difference to others, rather than waiting for even better tools.

The world of software engineering and development is changing at a breakneck pace. For someone who’s been in SWE for nearly 40 years (since I was 6 years old) and professionally for nearly 25 years (since I started getting paid for SWE work), I am concerned, but still hopeful.

What do I mean by “no more software left to write”? It means a few things:

• Software infrastructure has become so widely developed that writing new applications today — by hand, we’ll get to AI later — is 100x easier, faster, secure and optimized than just 5 years ago, and this rate of development is constant, meaning that in 5 years it would be 100x faster / easier / better than today.
• The amount of software written has risen dramatically. Just the sheer volume of applications and projects has increased, and within those – open-source projects that are easily copiable or integrable. It’s nearly impossible today to find an alcove of human pursuit that has been untouched by software or digitization.
• Software (and software engineering) has become as much like LEGO as it has ever been, where engineers (builders) can piece together an application in minutes! With advanced features and production-ready backends with just lines of code.
• Software deployment has become automated to immense degrees, where an app (web, mobile, desktop) can be packaged and served online (or dished out as download executable) with a single line of terminal. All backend services, all testing and integrations fully managed by someone else.
• AI coding has made it so full applications can be “written” ad-hoc to the user’s needs almost instantaneously, making the need for bespoke engineering obsolete – apps can be generated on the fly for a single use! The age of disposable food containers has arrived in software engineering.
• Hardware platforms are more generous than ever before! With memory, compute and disk capabilities that really make runtime optimization a thing of the past. You can brute force your way to software success and deal with consequences later, if at all it will become an issue.

All this means is that it has never been a better time to be a software engineer. And it has never been a worse time to start as a software engineer. If you’re starting out today, take note of the rate of change in the field – it is exponential. Tools and paradigms used today will be obsolete or woefully outdated in just 2-3 years. Except for the deepest of technologies, engineering applications has been commoditized to a pulp.

What it means for Utility and Creativity

I think the days of software engineering for utility are numbered. Meaning, to build utility software that solves a business need is going to go extinct, very soon. Just because most business problems already have some software covering it, or – it will be embarrassingly easy to write the software with minimal time and effort. So, utility, I believe will not drive many engineers of the future. Creativity and gratification however – will remain in demand. These are the things that makes us human — just like eating and sleeping — and they will not change much in the next 1,000,000 years.

See, creativity is nothing like utility. It’s not things done to solve problems. Sometimes it’s a mental exercise, sometimes it doesn’t have any material purpose, it might be art, or it could be done as therapy even. I think creativity in software engineering will stay a vibrant theme. People will write innovative software for creative pursuits (like media, arts, gaming etc.) or solving problems in a creative way (like porting Doom to an electric toothbrush).

But all in all, the major usage for software in our world will be taken care of by the arguments made in the beginning. Software to solve actual business problems (but also e.g. games for mass consumption) will eventually be automated away not leaving much for humans.

A Big Push on Infrastructure

A major part of the “no software left to write” paradigm is infrastructure, and not necessarily automation or AI. In just a few short years, the tooling for developers to create and deploy scalable software has exploded upwards. App development and deployment has been commoditized, meaning, it is now commodity – anyone can get it for a fair price. Contributing companies were Firebase, Vercel, Supabase, Netlify, AWS Amplify. They won’t write your frontend – but the infrastructure is dead simple: db, auth, host, analytics, server-side, APIs – you got it. And the price you pay for this peace of mind — no one got fired for putting data in Google cloud — is ridiculously low. For anything sub immense scale it’s basically free (except if you’re building a business – you should be happy to pay! That means you’ve reached scale). Most of these are a one-stop-shop end-to-end solution for apps, you don’t need anything else. Even if you’re not web based, with Electron to do some serious desktop damage and Qt + PyInstaller as an alternative – desktop is not taboo anymore (cf. projects like LM Studio as a case in point).

That’s infrastructure. On frontend though, things went absolutely wild. I can’t remember an era where developers could be designers and designers could be developers. With frontend tools like React (of course), Tailwind, Shadcn, Bootstrap, Material, the task of putting up some components on screen and make them look great — or at least palatable — became trivial. You want dark mode? (You know you do.) It’s built in. I remember a distant past where software/tech companies developed “design systems” and “component libraries” and hired a big design team. This is a thing of the past. I can now see tailwind-shadcn everywhere, and for good reason – it’s all you ever need! Just pick colors out of a premade palette generator and theme your way to app design glory.

Right, so infra is solved, frontend is done – how about backend and middleware? Surely – backend is hard! Well guess again because middleware and backend smooth as silk like Next, Express, Nuxt and others – you really don’t have to sweat anymore, just focus on building.

I think in today’s world, even if you’re working by hand — you luddite — the apps write themselves!

A World Full of One Man Shows

I think it was inevitable that the world of software tech would eventually come to this. As you probably know, software engineers are — in broad strokes — the reclusive kind, sort of introverts, like to work alone and get things done pretending they are slaying dragons. Well back in the day a single developer couldn’t get anything done at scale even if they were insanely good. It was because things simply took time to do. Provisioning server operating systems (let alone install them physically!), bringing up db services, pouring over API docs for obscure identity directory services for auth (remember LDAP?), and worrying about replication and redundancy and load balancing… I’m glad to see all that is now in the past.

Today, a single good-enough developer can actually slay dragons. It’s because they get their work done so quickly, they have all the time they want to play Baldur’s Gate. Seriously though, one-dev shops are popping up like mushrooms after the rain, where the rain is the crazy push on infra that allows them to translate needs into apps in mere minutes, hours or days. Little startup companies can churn through ideas and pivots at breakneck speeds and look for PMF without overly risking their financials. This is a very good thing.

At the same time, software has forever been a collaborative effort. Some people make the infrastructure and some make the applications. That’s how it always has been, and largely how it still works today. The fundamental shift though is that value has gone from infrastructure to the application layer. In the past if you laid down “copper” (think any medium for information transfer) and “metal” (servers) – you had a “forever business” because you could not be uprooted. But information infrastructure has commoditized, and value shifted to applications where you can charge the customer directly. Government declared internet an essential service just like water and electricity. The same is happening to software infrastructure today. How long would it be before cloud services will be declared an essential service?

Quantity Over Quality

I’ve written in the past that I believe the next push in AI agents would be “brute force”. A situation where you can have 1,000 weak competing agents on the same task, and all it takes is for one of them to succeed. Note this is different than “swarms” where each agent has a separate distinct job they must perform well on. What I suggest is that cheap agents with low probability of success will compete, but we will beat low probability with sheer numbers.

At the heart of “Quantity Over Quality” stands this principle: low probability can be beaten by repeated trials. The startups above with low probability of hitting PMF can now try 10-20 or even 50 separate ideas before running out of money. The AI coding agent[s] can try 100s of solutions for solving a bug — in parallel! — until something hits all the unit tests.

Think of software for doing the same thing in principle. We probably have in our world 100s of “versions” of Salesforce — primarily a CRM and ERP tool once geared towards sales but now has metastasized to every part of an organization. Everyone is trying to replicate the immense success. This is “brute force software” – low probability, multiple trials. And this experiment is repeated in pretty much every software you know! Like, WordPress (which I’m using for this blog), Windows, Google Docs, VS Code, Slack, Chrome, etc. each have 10s or 100s of viable alternatives.

Software Worth Writing

So what’s left for us lifetime software engineers? Everything has been done before 100s of times over, and the things not yet done will be made by swarms of swarms of competing AI agents. Is it even worth opening up the IDE today? Because, if we just wait a bit – software will be x1000 easier to write very soon, so why bother today? This is a principle called “The Waiting Calculation” or the “Wait/Walk Dilemma“, where it’s better to wait for the bus and go faster than to walk slowly.

I however think it’s much better to be active today than to wait. By acting you influence the future and shape the technologies. It may seem like the efforts of one person may be negligible in face of a global movement – but that is untrue. We are all part of a tight fabric, and we all pull and push on the people around us. We have colleagues, students, mentors, family and friends, and strangers too that we may meet online. We influence them, and they influence us. Overall, we are all grabbing a tiny little part of the helm of the humanity ship.

Today you should be writing software worth writing. In other words, write for impact – personal or societal. Replicating a social network just to learn how it works? Great project! Definitely worth writing. A script for cropping images by right click on the file? Wonderful idea! Anything that makes an impact on your life or others is worth writing, even today and in the future.

So while there’s no more software left to write, you — personally — definitely have plenty of more software worth writing.

TLDR: The tech industry is becoming a winner-takes-all arena. Traditional paths are dying, AI is eating entry-level jobs, and you have about 5-10 years to get rich or get replaced. Stop preparing, start taking.

“In tomorrow’s tech world, you’re either the one building the AI, or the one being replaced by it.”

Let’s cut straight to the chase: the odds are stacking up against junior folks trying to break into tech today:

• Mass layoffs have frozen hiring across the industry, with even tech giants slashing thousands of positions monthly
• AI and automation are breathing down our necks, replacing entry-level positions at an alarming rate
• Hiring managers have become numb to “average” achievements after seeing thousands of identical bootcamp projects
• Global talent pools have exploded with remote work and immigration, turning local competitions into worldwide battles

The tech landscape isn’t just harder – it’s transformed entirely. The bar for junior engineers and career switchers isn’t just higher; it’s in the stratosphere. We’re talking about a fundamental shift in what it means to be “qualified” for a tech role.

Is this a bad thing? On the grand scheme, probably not. It’s forcing people to be more driven, more motivated, more dedicated to succeed. The days of skating by on a basic CS degree are dead and buried. But let’s be real: not everyone with a CS degree is landing that $300K/year cash position at FAANG (e.g. Facebook, Google, Amazon, Microsoft, Apple, etc.). In fact, the vast majority have zero shot at such roles straight out of school. And I’m not talking about a small majority – we’re looking at 95%+ of graduates who won’t even get past the resume screen.

People ask me for advice on how to get ahead of these curves. Here’s the unvarnished truth, and fair warning – you’re going to dislike it.

The De-Sensitivity Crisis

Hiring managers and recruiters are drowning in a sea of “qualified” candidates. Making an impression isn’t just harder – it’s nearly impossible without something extraordinary. The problem isn’t just volume; it’s that everyone looks the same on paper. Another TODO app? Delete. Another machine learning project that classifies cat pictures? Straight to the trash. Another blockchain wallet? Please.

But what does “making an impression” actually mean for an engineer? This distinction is crucial, and most candidates get it completely wrong.

To be impressive, you need something that’s undeniably yours. Not group work, not team achievements – something personal that you can claim 100% credit for. No “I was part of the team that…” or “I contributed to…” statements. Here’s what counts:

• Open-source projects with serious traction (we’re talking hundreds, preferably thousands of stars and million+ downloads). Your weekend project with 5 stars from your bootcamp buddies? Not even close.
• First-author papers in top-tier venues (at least one for MS, four+ for PhD, plus 2-3 / 4-5 supporting papers equally). And no, your university’s research journal doesn’t count.
• Patents or awards from major federal programs or national institutions (your name, front and center). The local hackathon prize won’t cut it anymore.
• Massive social following (tens of thousands of followers minimum, millions of engagement marks). And we’re talking about real engagement, not bot farms or engagement pods.
• Money. Numbers. If you can say you’ve personally been able to clock millions of $$$ in revenue to your company and be able to show it – that’s all it takes.

Anything less? Hiring managers won’t even blink. They’ve seen it all.

The Path to These Achievements

Let’s be brutally honest: you need to borrow, hustle, and fight for every opportunity. And when I say fight, I mean it literally – you’re in a gladiator arena where only the most aggressive survive. Mostly because the other gladiators are faceless, nameless ghosts that just want to feast on your human brain.

Publications

Want papers? Muscle your way into top labs. Offer to work nights and weekends. Do the grunt work – data cleaning, figure creation, statistical analysis. Start as the fourth author and claw your way up. Be relentless. Take rejection as a challenge, not a stop sign. I’ve seen successful candidates camp outside professors’ offices, volunteer for the worst tasks, and work 80-hour weeks just to get their name on a paper. That’s your competition.

Projects

Building something great isn’t enough – you need to be a marketing machine. Spend as much time promoting as coding. Give away value, but make it count. Target audiences beyond the US if needed. Learn SEO, social media algorithms, and growth hacking. Every star, every download, every GitHub fork is a battle you need to win.

And here’s the controversial take: don’t dilute your success with contributors. This is your spotlight – own it. I’ve seen too many promising projects die because their creators were too nice, too willing to share credit. In 2025’s tech landscape, being nice is a luxury you can’t afford.

Awards

Unless you’re that one-in-a-million genius (newsflash: you’re probably not, just like 99.99999% of us), you’re competing on pure grit. Even with talent, you’ll need insane work ethic. I’m talking about the kind of dedication that makes your friends think you’ve lost your mind.

Awards are lottery tickets, but you can read the odds. If your gut says you don’t have a shot, move on fast. Time is your most precious resource, and you can’t waste it on long-shot bets. I’ve seen candidates spend years chasing the wrong awards, only to end up with nothing to show for it.

Social Following

If you haven’t started building your platform 6-8 years ago, you’re already behind. Way behind. Today’s tech influencers didn’t just appear overnight – they’ve been grinding for years, building their presence post by post, tweet by tweet.

But here’s the silver lining: those other achievements (projects, awards, papers) can catapult your social presence. Just keep it organic – people smell fake engagement from miles away. And remember, in tech, your social media presence is becoming as important as your GitHub profile. Companies want engineers who can be advocates, who can attract talent, who can represent the brand.

A Glimmer of Hope (For Now)

These barriers seem impossible? They are. That’s the point. Breaking through them – that’s what makes it impressive.

But there’s still the traditional path: Start with a decent (not spectacular) internship, grind out 2-3 years learning big tech’s playbook (frameworks, tools, workplace politics), then make your move upward. This route probably has 6-10 years left before AI closes it for engineers, especially in software. But the window is closing fast, and the competition is getting fiercer by the day.

Consider this: In pre-ChatGPT 2020s, a decent post-grad internship at FAANG might have required a 3.5 GPA and some school project experience. Today? Companies are expecting interns to have contributed to open source, built full-stack applications, published work, and mastery of multiple tech stacks before they even start.

Your advantage? You’re reading this now. You’re aware. You’re planning. This window of opportunity won’t stay open forever. If you’re taking the traditional route, start yesterday. Actually, start last year.

The Next Phase (10 Years Out)

In five years, AI will be crushing engineering roles. Jobs will exist, but only for 10-100x engineers with deep AI milage. Everyone else? Automated into obsolescence. Today’s frameworks already provide so much boilerplate that even basic LLMs can piece together solutions. Tomorrow’s frameworks? They’ll make today’s automation look like stone tools.

Here’s how it plays out: Today, if an LLM agent tackles a complex programming task (reading 100,000 lines, writing 10,000 lines) at $1, with a 1% success rate, that’s $100 per solution. In five years? That same solution costs a penny. Organizations will unleash hundreds of thousands of agents, throwing thousands at each task. When 999 fail but one succeeds – at that cost – humans become optional.

Think about that for a minute. When the cost of failure approaches zero, brute force becomes a viable strategy. Why hire a human who might take a week to solve a problem when you can launch 10,000 AI attempts and get your solution in minutes?

And ten years out? Software development as we know it dies. Need software? An agent fashions it instantly, more like searching a vast directory than building from scratch. The entire concept of “software engineering” becomes as relevant as “horse-and-buggy maintenance.”

The signs are already here. Look at the tools released in just the last six months. They’re not just helping developers – they’re replacing them. Every new LLM release makes more junior dev tasks obsolete. The trend isn’t slowing; it’s accelerating.

How to Prepare

Don’t.

Instead, get rich as fast as you can. Look around – solo entrepreneurs are already building AI-powered empires. They see the writing on the wall: individualism (and tribalism) is back. Be self-reliant. Pay yourself first. Don’t share the spotlight. Be ruthless about your goals.

The era of the comfortable, steady tech career is ending. What’s replacing it? A winner-takes-all gladiator arena where only the most aggressive, most adaptable, and most ruthless survive. You’re either building the AI tools, or you’re being replaced by them.

Your time starts now. Every day you spend not building your empire is a day your competition is getting ahead. Every minute you waste trying to “prepare” for the future is a minute someone else is spending taking what could have been yours.

Good luck. You’re going to need it.

And if you think this sounds harsh or unfair? Welcome to tech in 2025. It only gets harder from here.

I’ve been writing code for the past 36 years. That’s a lot of code. Started at 6 years old, writing BASIC on an Apple IIe that my dad brought home. Then graduated to PASCAL, Visual BASIC, MS-DOS Batch and then Borland C and Java, moved to C++, Objective-C some C# and JavaScript, Python obviously, recently TypeScript, all the while learning the secrets of Bash, ZSH and PowerShell and other more obscure languages like Go, Rust and even Elixir, LISP and Lua. I’m probably forgetting many languages and coding “situations” I went through like MSSQL/TSQL/PLSQL and HTML, CSS of their kind and several other task-specific code-ish things. This is just to explain and show to you, junior coder, that coding (for real) is a lifetime pursuit. Just like my journey, you will hear similar things from just about any other lifetime programmer that you know.

See, coding is in my blood. I may claim that I had no choice but to become a coder, seeing as my dad, a control systems engineer by training and programmer by trade, has brought coding home to me, my older sister and younger brother at a very early age. My dad was programming AS-400 IBM mainframes in RPG to build the backend software for banks, insurance and logistics companies. As young kids we learned about algorithms, data structures, databases and user interface – in an unstructured way, an exploratory way, and as apprentices. Back in the 1980s software had very scarce mediums for sharing, so it often was shared in books! Big bulky book full of code. And my dad set us up to copy lines from to book into the computer to build a game or a graphical software. I was amazed and hooked from the very beginning. But that was a very special circumstance. Code and coding have transformed so much I sometimes can’t even recognize it, but for me it will always remain: a thrill of creation and exploration, a dialogue, a way to communicate thoughts.

In this article I would like to dispense some advice on writing code. But not in way of design patterns or efficiency. I’d like to tell you about the personal side of writing code, and what it can do emotionally, about what it means to be a “Coder”. At the same time, it is a very pragmatic guide, just as much as it is personal. As usual, take all of this with plenty a grain of salt.

How to Love Crafting Code

The above introduction goes to show that for me coding is a passion. In fact, I would say coding is my love. I love coding. I do it every day, for years and years and more years, and I hope to be coding for many decades more. When I’m not coding, I think about coding. I would do it every day given the chance. I also love very much the mental struggle and difficult puzzles that coding poses. It might seem unfair. Many tell me that coding comes easy to me but let me say that no code ever came easy to me. I heard a quote from Henry Rollins recently that goes roughly like: “I’m not as talented or smart as others, so I just get up early and stay up late every day”. This is basically it. Coding is hard, always, even to experienced coders. But it’s precisely because it is hard it is appealing to so many people.

A friend recently asked me how long it would take to code something. I answered: “There’s no such thing as ‘Coding time’ per se, it’s just how long you sit in front of a keyboard banging your head against it until the thing is done.” This is the sad truth about coding, is that it’s simply hard and takes a long time to make a really nice thing. We will discuss AI coding shortly, but nothing would replace careful, measured steps, incremental advancement over a long period of time. Demos and one offs, simple one-file programs or scripts, new projects and boilerplates – all are not real examples of what a programmer would do on a daily basis. Coding a robust long-lasting module or package just takes time.

Coding is a craft. Just like woodworking or cooking. It relies heavily on past knowledge, drawing parallels from other projects, habits, rules of thumb with a dash of curiosity and trying something new. See, no code is truly similar to any other, it’s always a continuation of something. Remember the memes about programmers only needing 3 keys on their keyboard? Stack Overflow, Ctrl-C and Ctrl-V. This cannot be more wrong. Every empty file about to be filled with code is like a clear canvas, to which you take a big brush. First you work in broad strokes! Throwing big blotches of paint, just covering empty areas, making a scene, a background, for your finer work later on. After the canvas is full but really doesn’t look like anything, you take the smaller brushes, start filling in the little details, make smaller adjustments. Finally, you test your work. Take a few steps back, stare at it for a bit, raise an eyebrow. Find a bug here and there, just getting a feel of your creation, learning it, seeing it, connecting with it.

Coding is so much like painting, sketching, sculpting, or writing than you’d imagine. That’s why I believe that myself and others are so in love with coding. Not because we like code! In fact, if I’m being honest, I kind of dislike code. Code really is a way to describe your thoughts to the computer, it’s a medium that shouldn’t exist. But coding – oh boy, that is just wonderful. There is joy in coding. That’s why I fail to understand how come everyone is so eager to get rid of it. “No-code Platforms” or “AI will make all programmers obsolete”. That really is beside the point. People love coding because it’s a craft, an art, an expressive medium. This is the reason you will find folks in their 60s and even 70s still coding. It’s just fun!

How to Love Coding for Work

Ok coding is art and it’s a lot of fun, we get it. But doing it for 9-10 hours of the day for someone else? that’s recipe for “hobby burnout”. You will get sick of doing anything you love doing if you do it too much or for the wrong reasons. So, we must all find a way to still enjoy coding if it’s our job. Insofar as we like getting paid to support our lives and families.

Coding for work doesn’t have to be so bad though. As I stated above, I would be perfectly happy coding 10 hours a day. So, really the problem is in the motivation. If you’re working on a passion project, you’re probably highly motivated. If not for seeing it complete then probably for the skills you’re gaining, the community around it, or maybe it’s impact on others or even yourself. See – all the reasons are there! Treat your work coding time as a chance to enhance your skills, see the impact of your work on others, or find ways to communicate your work. Without you knowing, you’re already doing this. Every hour spent coding you’re getting better, more experienced. This however isn’t a recipe to avoid burnout. Burnout is likely inevitable.

There’s possibly nothing worse than being handed code that someone else wrote and being asked to “make it better” or add a dull feature or refactor it. This regularly happens at work, particularly for juniors. People would likely tell you how it’s actually a good thing: “Learn our codebase through your fingers” they will say. Frankly, I don’t buy that. As I said, I love coding, but I dislike code. In these situations, which you can’t always avoid, I suggest making the most of it. If you take the time to read and understand the code, take an equal amount of time to document it. Make diagrams etc. First, I think your supervisor would instantly have a whole new level of admiration for you (who doesn’t like documentation?), but second, this can factually be good for you. Any ancillary output beside code, for a programmer, is really a rare sight. Programmers like the coding part see, they usually don’t like the extras. If you’re an “extra” kind of coder – that increases your value and makes you a candidate for promotion.

To recap, I’m not suggesting coding for work is going to always (or even sometimes) be as fun and exhilarating as working on your own passion projects. But if you focus on your personal goals – it can just bearable enough to keep you satisfied for a long while. Look for the opportunity to get better and go farther.

How to Love AI Coding

AI coding is a game changer. Personally, I really enjoy it. Many times, I go “oh my this totally just read my mind” and have moments of awe. There are things AI coding or a copilot does absolutely amazingly well. There are also things it does terribly wrong. But for better or worse this is the future. I have no doubt. If you’re not using AI to program in 2024, I’m sorry friend you’re in the wrong field of work.

One thing to get out of the way right away is that using AI assistants for coding really doesn’t take much of anything out of the joy of coding. If anything, I love how the AI does the boring stuff for me, and I can stay in the flow longer. It’s still a paintbrush, but a bigger one that has tiny little paintbrushes on it, so you still make broad strokes, but they also fill in the tiny details at the same time.

The key I found for being successful with AI coding is to deliver your intent. If you’re writing a new function and want that filled in – start with a descriptive comment, if the AI still doesn’t make a good suggestion, start writing the signature of the function using meaningful names. For example,

// This function takes in two numbers and returns the largest
// common denominator.
int largest_common_denominator(

This really should be enough for any AI assistant to complete the rest. Most likely the AI would not even let you finish the comment before it gives a suggestion. Another thing is to get to the point as soon as possible with the least verbiage, for example:

// Calculate the largest common denominator from

At this point the AI will probably already give you a valid suggestion.

One more thing about AI coding. There’s a trending word out there that programmers would be out of a job because of AI coding getting too good. While that might be true someday, this must not deter you from learning how to enjoy coding today. Coding is more than making computers do things, it’s a mental exercise, a puzzle, a craft, an artistic medium of expression. Do it because you love it, or I suggest, in interest of time, you go do something else. There are plenty of professions that AI will not touch any time soon if that’s what you’re worried about. Folks that use AI to write all their software for them and simply copy-and-paste verbatim – are not getting any joy from it. For them, I’m actually very glad that AI can spare their time and misery in writing code. AI coding is much better for a person that enjoys coding, than it is for someone who doesn’t.

Some departing words. Coding is a lot of different things for different people. It’s an art, a craft, a job, a necessary evil, a struggle, a means of providing, a collaboration, a community, a joyful thing. Every coder will tell you another story. But at the basis of all you will find that insofar as coding is an intermediary between machines and humans, it’s always about humans. If machines knew how to program themselves to do exactly what we humans need without any interface – then coding won’t exist. Some people believe in that future. I don’t.

Love coding, and it will love you back.

Working in the service of something bigger than yourself is a pillar of humanity. It enabled all riches and privileges we have in our world. It is how people have built society for eons. There’s no doubt today that being employed keeps you balanced, satiated and healthy (in a way). It’s the best tool to support yourself and others. But at the same time, it takes away many of your freedoms. Before we dive in, let’s set a few things straight.

At the core of employment stands a simple principle: An employer is hiring you for value you bring to the business which is less than the compensation they pay you. Get it? At the core of employment, the principle is that someone is getting bigger value from your work than what they pay you. No business ever, if it’s a well-functioning business driven by real metrics, will pay you more than the value you bring – otherwise they will either hire someone else for less or not hire at all. If you ever own a small business, you will learn this simple principle very quickly. No business is charity.

The next principle is that as an employee you are first – a function. You’re not a person first, you’re not the sum of your goals and aspirations, you’re not your knowledge and expertise. As an employee, you are filling a role, driving value in a specific vector. If you are more than that in an organization, well you’re no longer and employee and more of an owner. All employers will try to convince your otherwise, but at the core – they first need you to perform a role, a task.

Finally, as an employee you are part of a whole. You’re not an individual maximizing your own gain. When you sign up you promise to maximize the gains of the organization. There is comfort in knowing that everyone around you is exactly the same – they are pulling the cart just like you.

Understand, the above principles are not bad or malicious. They are in fact wonderful and unlocked the greatest innovation and prosperity for all of humanity. Because of these gaps in gains, everyone is in effect donating a portion of their efforts to a greater good. It is often misappropriated, but on the grand scheme of things – it is perfectly great.

Being an employee will teach you a whole lot. About your business domain, about people (peers, customers) and how they think and operate, about impossible decisions, about things done at immense scale, about yourself as you stack up vs. others, and about life itself. The key in making the most from employment is to capture these teachings.

Here is some advice I have for you as you’re joining the workforce.

How to be a Bad Employee

Don’t be a bad employee. If you’ve joined a place of work, put in the time and effort to be successful. When I say “bad employee” I mean – understand the principles above, repeat them to yourself every day and on all occasions. And then, think for yourself. What kind of engagements are you looking to have in your personal life? What are your personal goals and metrics of success? Just put yourself first.

To me, being a bad employee is leveling the playing field. Employers will take endlessly but only pay you up the value equilibrium point – the point where you are paid as much money as your employer values your work. As such, you’re in a rigged system. You will never (unless by trickery or felony) be able to get paid more than you’re valued. Again, this is not all bad and most people in the world are better off on one side of this equation.

My advice for being “bad” is to make sure you are always on the right side of the equilibrium, not by cheating and slacking off, but by making the pie bigger. You can always, always! generate more value for yourself than your employer values you. Sure, you can take a second job (moonlighting, “side hustle”), but you’re at risk of double-dipping on the time from your main job, which would be borderline morally wrong. Many people do it, more than you’d think, and they glorify it. What I advise is to look for values in a different place than your time. Here’s the principle:

1+1=3.

Keep that in mind. 1+1=3 is your goal for employment. Always generate more value from one single thing that you do, really squeeze it out. Prepared a report to your supervisor? Also prepare a documentation of how you made the report, or how you would automate it. Wrote a piece of code? Also note down and share (!!) how you did it and what should be done next. Made a presentation for a meeting? Deliver the same presentation again in a different setting. Had a meeting with someone high ranking in the company? Write about your impressions from the meeting and your personal take on things. You were promoted to a leading role? Write a guide for all of things you learned, share it religiously, make sure people know why you were selected, and they will have a lasting impression of you that will last your career! There’s a big group meeting? Ask your boss for a chance to present work you have already done.

Be a bad employee. Generate more value for yourself than your employer needs or hired you for. Paradoxically, this is the best way to advance up the chain at work! Truly, a remarkable thing. If you maximize your personal gain at work by generating more value like discussed above, and the organization will value you more. When you think about it, it makes all the sense, since the organization is hiring you based on how you are valued in the market. Make yourself be worth more. Always grow your skillset, preferably – laterally, and always grow your network inside and out of your workplace.

How to be a Good Employee

As a junior employee, my advice for the first few years is to just keep your eyes and ears open when you join a new place. Look for inefficiency. Organizations, and particularly management, are obsessed with inefficiency. Because of laws of diminishing returns, all orgs that get bigger become inefficient and efficiency is a battle at any workplace.

Attended a meeting and you were bored for half the time? That’s inefficiency that you should share and publicize. The next meeting should be half the time, and if you’re lucky – you will never attend it at all! Had to make presentation from a bunch of data on various systems and that took you forever? Perfect inefficiency – let the people in charge of data systems know. You have to take 3 authentication steps or keycard scans to get to your desk? That’s eating up your efficiency for performing your tasks.

The key here is to verbalize. Never suffer alone. Either commiserate or communicate, preferably to your supervisor but group meetings are great too. Every workplace has tons of points for improvement, and is looking for them, but most people around you either don’t care or don’t mind because they are “getting paid for their time either way”. Don’t be like that, be a good employee. Speak up. But don’t be overbearing or make it all about complaining. Do it from a place of caring. Your goal is to improve conditions for everyone, not just yourself.

The are many ways you can be a good employee, and many guides for it. But the core of all of these things is singular: Generate as much value to the organization as possible. Either you lift other employees up, or optimize operations (save the org money), or through working more efficiently yourself, or by finding new value so far undetected (make the org more money).

How not to be an Employee

For once in your career, my advice is to try very hard not to be an employee and strike off on your own. This is the ultimate test of your skills and stamina as a professional. Being independent is hardest professional move, and definitely not for everyone. But I guarantee that at one point in your career you will feel that you must give it a shot. I’m going to exclude starting a company and focus on self-employment, since by starting a company, you are in fact an employee again.

Self-employment is the toughest thing. Do not attempt it before you have a lot of prestige and credentials in your domain. Finding customers, and holding them, is mostly a matter of marketing your skills and value you can bring. If that hasn’t been substantiated – you’re at risk. Remember, someone will only pay you up to the value that you can provide, and since you will now be charging a premium on your services – that value needs to be equally bigger. Getting to a position where self-employment makes financial sense usually takes years. At first you could only charge roughly as much as you can in your 9-5 job, after taxes you’d be taking home ~50% of the salary you used to make. It’s only when you can comfortably charge x2-x3 your salary that the math works out. Look for calculators online.

Luckily, since you’ve been such a bad employee and you’ve looked out for yourself from the beginning – you are well on your way to establish yourself as a leading figure in your field. That is your currency. Treat the outside perception of persona as equity, almost money in the bank. It’s an asset to you that you must guard and nourish. It is critical for success as a non-employee. So how do you generate this currency?

Personally, I’ve always liked academia. At first it seemed like a wonderful lifestyle; in reality it is just like any other business. But one thing is set apart: People are recognized for their individual contributions. You publish papers with your personal name on them, you receive awards with your name on them, you start labs and call them after yourself. Yes, it’s a very ego-centric domain. But, for that reason exactly it is great for self-employment. Your success is measured by how significant your name in your field is, like how many citations your papers get and other mostly similar metrics. While employed at a company – the exact opposite happens. No one knows what you did exactly and what was your contribution! To make matters worse, most if not all of your projects are group efforts where you had a small role, even as the lead. Try selling that to your customers! “Oh yes I’ve worked on this huge successful project you must have heard of, but I only did some part of it and the rest was taken care of by another group of dozens of people.” That’s not a convincing demonstration of your amazing skills.

To be a non-employee you must first work for yourself while you’re working for others. Building a name for yourself is just about as hard as being self-employed. It’s a matter of getting out there and being visible. Both internally in your company (remember, your colleagues are your professional network forever), and obviously outside of your company. This is work. Constant, ungratifying work, at first. But establishing a following outside of your company is crucial for your success. In some ways, treat establishing a name for yourself more important than your skills. This is done via social networks, in-person social events (e.g. meetups), conferences, article writing, blogging & podcasting, maybe writing a book even. But the most important factor of all of this is consistency. Do this consistently over a long enough period of time and your chances of failure are slim.

Some parting words. There is so much more to being a successful employee or a self-employed person than what I’ve covered here. Countless books have been written. This here is my own reflection on a couple of matters I thought were interesting. But you must write your own philosophies which you will invent through decades of work. Go forth and participate in the largest open human experiment called Employment. Come back to share your experiences.

Lots of bright young folks ask me “How do I get a good job?”, “How do I achieve success in this domain?”, “What should I do next?”, usually in the applied computer science or machine learning field. So, I thought I’d dispense some advice for all future advice seekers, and I may refer some people here, so I don’t have to repeat myself all that much. Although, feel free to come up and ask a question.

Graduating, from anywhere, is daunting. Going from a well understood environment where your achievements are precisely measured, into the world where, frankly, no one cares. You have to make up your own metrics, measure yourself up to them, set goals for yourself and have a roadmap and a timeline. There’s no “graduation” from life. You may think it’s retirement, but that would not only be wrong (since there’s life after retirement) it is also immensely subjective and domain specific. So, what metric would you choose? Money in the bank? Sq. ft of your home? Number of dependents? Assets? Papers published? Books read? Miles traveled? BMI? Social subscribers? See? It’s impossible to pick. It is foolish to pick. Don’t pick. Not right now at least.

Your goals after graduation should be to establish yourself as a professional and an individual. Yes, you should focus on yourself. Maximize gains for your-self. Take all the credits, don’t be shy. Exercise all your rights and privileges. Soak up as much of the accolades. But at the same time appreciate, acknowledge and celebrate the work of others. Never punch down or sideways, only lift people. Make sure that you pack your knapsack very tightly with people you work with, projects concluded, products delivered, etc. These are your fuel reserves for the rest of your career and life – so bulk up!

OK, that sounds good but how to actually do that? this “bulking up”, you may ask. The answer is very short and clear:

Build.

Just – build. Build a product. Build a community. Build a following. Build a reputation. Build relationships. Build a home. Build a family. Build your confidence. Build a nest egg. Build an expertise. Build a hobby. Build. Build. Build.

Treat your first few years after graduating from your post-secondary education as a building period. Everything you do, even if it’s playing video games and going out, should be part of some… thing that you are building. If you’re a social type – you are building a social network. If you’re an engineer – you are building a product that delivers value to someone else. Be in a constant state of building, in many different vectors all the time.

Building is like investing. The consequences of your efforts will only be clear to you after a long time. And just like investing – building compounds. You can never go wrong with being focused on building. If you continue to invest in a project or direction – you will see success eventually. Investing in the S&P 500 for example has guaranteed positive gains if your time horizon is >10 years. It is exactly the same for building, if you stay consistent in your contributions you will keep up with the market and build something with lasting value. Some people refer to the 10,000 hours rule for becoming an expert. Building works the same way.

Now your next question may be: What should I build? Again, I have a very short and clear answer:

Anything.

See the thing is it doesn’t really matter what you build right now. As long as you are focused on building you are doing the right thing. You may have a tendency to a certain area or an interest domain, but truth is – you’re at the very early beginning of your journey that will have a definitely unexpected end point, so it doesn’t really matter which direction you start walking. As long as you are walking. Walking is the focus.

Just focusing on building will teach you a lot. Anywhere and anything you start making will teach you: whether you like it, whether there’s a market for it, whether the domain community is nice, whether it’s something that can compound, whether it has a holistic effect on other things you may build. So many things to learn!

How to Build

This is geared towards students of the engineering disciplines, but it does transfer very well to everything else. I want to give some direct concrete advice on what to work on and how to do it, in case you don’t have a lot of ideas right now. By the way, not having ideas – is a blessing. I wish, one day, to not have new ideas on things to work on, it’s a curse, believe me. But in any case, even if you do have ideas and plenty of them, here are a few things to consider.

Working on an open-source project. Open source is a divine gift given to humanity. It drives the entire world. It creates so much incredible innovation. It is timeless too, and has existed since the beginning of history. It has the power to make you very successful. My advice would be to start a new open-source project and not join an existing one. See above – you should take ALL the credit to yourself at this point, don’t share. But the product of your hard work – give that away for free, again – right now, don’t ask for anything in return. Participate in humanity’s greatest experiment: Benevolence. Work hard, help people through your work, ask for nothing, and success will arrive, it is only a matter of time.

The next question is – what project should you build? Great question! First, I suggest you refer back to your goals. Let’s say your goal is to get hired by a big corporation (a great choice! a big company will teach you everything you need to know about business!). Then my suggestion is that you take one of their products and clone it open source, don’t be shy, don’t fear retribution, I guarantee you they absolutely don’t care (their value is in a completely different place! find out where…). Just the thought and implementation exercises on how to build it will show you: 1. It is not as easy as you thought, 2. all the snags along the way, 3. all the decisions along the way, 4. how to improve it and go beyond – in short: Everything this company wants you to know as a candidate! You’re already prepared for the toughest interviews.

The other advice on what to build is more personal. If your family is lucky to have a family business – build something for that business. The key is to get information on your customer’s needs. When the customer is your dad, well that makes it a little easier (usually, not all dads are the same) to get that insider information. But if you don’t have that – you can reflect on your own life and “build for yourself”. I’m not suggesting you make yet another time management system. Instead, find gaps in your knowledge and fill them. If you’ve always wanted to learn to code in Lisp – do that now! Again, the direction is not super important, the persistence is! Find something you can do over a long period of time, like 1-2 years. Knowledge and education are excellent. If you’re missing a class on a certain topic – create it. Same as if you’re missing a tool to e.g. sort your photo album folder based on how many dogs are in the photo – make that tool. Just start making something useful(-ish?) and put it out there.

Another aspect of building and contributing is how to get noticed. This is important, you want visibility else no one will recognize your hard work. And again, see above – you want ALL the credit at this point. When building open source, use GitHub. Make it as easy as possible for people to access your work. If you have a knack for it – go on YouTube and make some videos. Those assets are compounding investment vehicles too. Go on Twitter, be religious about talking about your work. Remember: this is open source and therefore belongs to the world – better make the world aware it got this gift of your work.

This brings me to the last part about community. Every product, every company, every successful business, every service you see and know – is about People. It’s not about finance or things, it’s about what the products do to change the lives of People. People are the key. So, when you go about your build make sure you have a community of people in mind. When you want to tell people about it, you likely would want to find where they congregate, maybe on a subreddit, maybe a Discord server, or a mailing list (if they’re OGs). Get in there, mingle, and when you’ve got the hang of it – make a community of your own! Opening a Discord server is $0. There’s a similar cost level to starting a mailing list or newsletter. But whatever method you choose, it has got to be open, it must be inclusive, it must be not about yourself but about the community. Become a servant of your community, NOT it’s leader. Be the fuel, be the fire that burns under the cauldron. Facilitate, and participate. Trust me, this will pay off x1000 times by the time you realize it.

How to Fail

You might have heard about “Fail fast, fail early, fail often” and “Learning how to fail well”. I think most of it is BS told to you by people who have already reached success. It is true that at this point in your life you have extremely high risk-tolerance, and pretty much nothing you would do now would damage your long-term success, short of a felony. So, while it is time to experiment, be ready to accept immense measures of failure. Like, so much failure that you’d re-think your entire life.

Here’s how you will for sure fail: You will write something so cool and not a single person would read it or comment about it. You will build a free product that people pay $1000s for the alternative and not one single person will use. You will build a community, and no one will join or won’t say anything. You will make a video, and no one will watch. All this will happen to you, again you have my guarantee.

But guess what. This is absolutely perfect. It’s exactly what should happen. The burning pain you feel in your chest? that will become your battle scar. The feeling of loss of purpose and rethinking everything? that will become your rocket fuel. The crippling impostor syndrome when everyone around you is 10x more awesome than you? That will become an impregnable Kevlar armor. You will look back at this and you will smile and thank the universe for these opportunities to fail. You will not get many more of these beautiful moments of failure again…

The most important thing in failing is the learning that comes from it. That is not a cliche. But it is by far the easiest to learn from failure when it’s about something that you build, by yourself, for your personal success. If you slip up or straight up just be negligent in employment – prepare to accept the consequences. But when you fail working on your own thing, it’s far easier to know how to improve for next time. And see, the only thing you need to do, even if you don’t do anything to improve, is just to keep on building. Stay consistent.

How to Stop

Finally, I want to share a bit of advice on knowing how and when to give up. So far in this article I’ve been pretty adamant on “just keep building and never stop”. That’s a good philosophy. But you should also take care of learning when and how to stop building. It is very easy to get sucked into a whirlpool of never-ending work, and holding the above philosophy you may not have many exit points. To this I want to dedicate a few sentences.

First, always in life and work, always hedge your bets. Always have something to average with. Meaning, if you build – build several things at once. If you’re working at a job – have several streams of work. The only way the numbers are in your favor is if you’re averaging either over time or over space. Remember the S&P 500, you have 10 years to average over. In building – make sure you have multiple horses in the race. When one fails, others will step in to continue the race. This is the easiest way to “Stop”. Just divert your resources to better ventures and let the failing project die. Simple.

But what if things are not simple? and your project is “kind of” working but you’re not sure? This is time to look at the bigger picture. If you have a family to support that is the easiest, focus on their needs (ahead of your own) and adjust course. But if it’s just you, which is most times the case, go back to your goals: Is this project going to get me hired at this place? Is it going to teach me? Does it have a community I want to be a part of? Is it genuinely doing good? Really, don’t be bothered with silly terms like “disruption”, “displacement”, “growth”, “scale”, “market fit”, these are absolutely irrelevant at this stage. Your goal in this period is to build. If building the project doesn’t also at the same time build your reputation – it’s a sign. Everything you do should be 1+1=3. If it’s closer to 2 – it’s a sign. Learn to read signs.

So, how do you know when and where to stop? You will know. It will be when you achieve your goals.

A few parting words to end this article. Always be in motion. Always be building. Motion creates a flywheel effect that works for you when you sleep. Realize nothing will be served to you ever. You will have to build nice things to have them. Alway be learning, always be teaching. Share your knowledge for free at every opportunity. Make your opinions vocal, make them in person preferably. You are ready.

If you’re a fan of OBS (Open Broadcaster Software), you may already be familiar with its vast library of plugins that enhance its functionality and provide added features. One such plugin that I recently developed is the URL API source plugin. This plugin allows you to fetch information from a URL and display it in your OBS stream. In this blog post, we will take a closer look at the source code for this plugin and understand how it works.

To begin with, let’s quickly recap what the URL API source plugin does. When set up as a new source in OBS, it has a few properties, including the URL from which it fetches information, the parsing of the output, styling using HTML tags and CSS, and a timer. The plugin is built in C++ and consists of various functions that handle tasks like making HTTP requests, parsing the response, and rendering the text on screen.

The core functionality of the URL API source plugin lies in a thread that runs continuously in a loop, separate from the rendering thread and the rest of OBS. This thread periodically sends the HTTP request, parses the output, and sends it back to OBS for rendering on screen. By running the requests on a timer, the plugin ensures that the rendering pipeline remains lean, as it doesn’t have to handle the requests frame by frame.

The thread function:

void curl_loop(struct url_source_data *usd)
{
  struct obs_source_frame frame = {};

  while (true) {
    {
      std::lock_guard<std::mutex> lock(*(usd->curl_mutex.get()));
      if (!usd->curl_thread_run) {
        break;
      }
    }

    // Send the request
    struct request_data_handler_response response =
      request_data_handler(&(usd->request_data));
    if (response.status_code != 200) {
      obs_log(LOG_INFO, "Failed to send request");
    } else {
      uint32_t width = 0;
      uint32_t height = 0;
      uint8_t *renderBuffer = nullptr;

      // prepare the text from the template
      std::string text = usd->output_text_template;
      // if the template is empty use the response body
      if (text.empty()) {
        text = response.body_parsed;
      } else {
        // attempt to replace {output} with the response body
        text = std::regex_replace(text, std::regex("\\{output\\}"), response.body_parsed);
      }
      // render the text
      render_text_with_qtextdocument(text, width, height, &renderBuffer, usd->css_props);
      // Update the frame
      frame.data[0] = renderBuffer;
      frame.linesize[0] = width * 4;
      frame.width = width;
      frame.height = height;
      frame.format = VIDEO_FORMAT_BGRA;

      // Send the frame
      obs_source_output_video(usd->source, &frame);

      // Free the render buffer
      bfree(renderBuffer);
    }

    const int64_t sleep_time_ms = (int64_t)(usd->update_timer_ms);
    {
      std::unique_lock<std::mutex> lock(*(usd->curl_mutex.get()));
      // Sleep for n ns as per the update timer for the remaining time
      usd->curl_thread_cv->wait_for(lock, std::chrono::milliseconds(sleep_time_ms));
    }
  }
  obs_log(LOG_INFO, "Stopping URL Source thread");
}

One interesting feature of the plugin is the use of a condition variable (curl_thread_cv above) for the thread’s sleep function. This allows the thread to be interrupted if the source is hidden or destroyed, preventing OBS from hanging until the thread completes its sleep cycle

The plugin supports various options for parsing the output, including JSON, XML, and regular expressions. For JSON parsing, the developer has implemented the Nlohmann JSON parser and JSON pointer for extracting specific information from the response. Similarly, XML parsing is handled using the PugiXML library and XPath for extraction. And for regular expression parsing, the plugin utilizes std::regex library.

Once the request has been completed successfully, the plugin renders the fetched information on screen using the Qt library’s QtTextDocument. This allows for typesetting or layout of the text using HTML or markdown. By creating a template with the desired styling and replacing the text content with the fetched information, the plugin achieves a user-friendly rendering of the data.

The text rendering function:

void render_text_with_qtextdocument(const std::string &text, uint32_t &width, uint32_t &height,
				    uint8_t **data, const std::string &css_props)
{
  // apply response in template
  QString html = QString(template_text).replace("{text}", QString::fromStdString(text)).replace("{css_props}", QString::fromStdString(css_props));
  QTextDocument textDocument;
  textDocument.setHtml(html);
  textDocument.setTextWidth(640);

  QPixmap pixmap(textDocument.size().toSize());
  pixmap.fill(Qt::transparent);
  QPainter painter;
  painter.begin(&pixmap);
  painter.setCompositionMode(QPainter::CompositionMode_Source);

  // render text
  textDocument.drawContents(&painter);

	  painter.setCompositionMode(QPainter::CompositionMode_DestinationIn);
  painter.end();

  // save pixmap to buffer
  QImage image = pixmap.toImage();
  // crop to the idealWidth of the text
  image = image.copy(0, 0, (int)textDocument.idealWidth(), image.height());
  // get width and height
  width = image.width();
  height = image.height();
  // allocate output buffer (RGBA), user must free
  *data = (uint8_t *)bzalloc(width * height * 4);
  // copy image data to output buffer
  memcpy(*data, image.bits(), width * height * 4);
}

Overall, the URL API source plugin for OBS is a simple yet powerful tool for fetching and displaying dynamic content in your OBS stream. The elegance of its implementation lies in the careful handling of HTTP requests, parsing of various data formats, and the efficient integration with OBS’s rendering pipeline.

If you’re interested in diving deeper into the code and exploring the additional features of the plugin, I highly recommend taking a look at the source code yourself. The developer has put in significant effort to ensure the plugin’s functionality and ease of use. By studying the code, you can gain insights into the inner workings of the plugin and potentially even contribute to its development.

In conclusion, the URL API source plugin for OBS is a valuable addition to any streaming setup, providing a seamless way to fetch and display external information in your live streams. Its well-structured source code, efficient threading, and support for various data formats make it a versatile tool for streamers. So, give it a try and see how it enhances your streaming experience.

To begin with, this plugin is an audio filter, which means that it works by filtering the audio entered through one function in the main plugin file.

struct obs_source_info my_audio_filter_info = {
  .id = "cleanstream_audio_filter",
  .type = OBS_SOURCE_TYPE_FILTER,
  .output_flags = OBS_SOURCE_AUDIO,
  .get_name = cleanstream_name,
  .create = cleanstream_create,
  .destroy = cleanstream_destroy,
  .get_defaults = cleanstream_defaults,
  .get_properties = cleanstream_properties,
  .update = cleanstream_update,
  .filter_audio = cleanstream_filter_audio,
}

The cleanstream_filter_audio function is responsible for getting audio frames, processing them, and returning the resultant audio data to OBS. The entire magic of this plugin happens within this one function.

The CleanStream OBS plugin is built on top of Whisper C++, a project by ggerganov that allows building the OpenAI Whisper speech recognition model from the ground up in C++, without any dependencies. To use Whisper.cpp in CleanStream, we just need two source files, two header files. Whisper C++ runs the neural network for Whisper, which is quite slow, in a different thread to continuously run in the background. This threading requires some buffering, so we use a circular buffer (utility provided by OBS) to store the incoming audio data, and a separate thread for continuous background processing. Mind I am using two buffers – one for the raw audio data and one for “info” which denotes how many audio frames are in the data buffer and what’s the timestamp.

// push back current audio data to input circlebuf
for (size_t c = 0; c < gf->channels; c++) {
  circlebuf_push_back(&gf->input_buffers[c], audio->data[c], audio->frames * sizeof(float));
}
// push audio packet info (timestamp/frame count) to info circlebuf
struct cleanstream_audio_info info = {0};
info.frames = audio->frames;       // number of frames in this packet
info.timestamp = audio->timestamp; // timestamp of this packet
circlebuf_push_back(&gf->info_buffer, &info, sizeof(info));

The processing thread uses a loop called ‘Whisper Loop,’ which continuously runs to clean out the audio from any unwanted words or profanities. It gets the shared data for the plugin and checks if the Whisper context is initialized and if there is any data to process or not. If there is data to process, the thread performs some processing, such as resampling and voice activation using a VAD (Voice Activation Detection) algorithm that looks at the energy in all the windows.

float energy_all = 0.0f;

for (uint64_t i = 0; i < n_samples; i++) {
  energy_all += fabsf(pcmf32[i]);
}

energy_all /= n_samples;

if (energy_all < vad_thold) {
  return false;
}

This creates an overlap between different samples coming in, and to set the overlap region dynamically, we check the timing of the Whisper inference function. If Whisper was fast enough we increase the overlap, otherwise we decrease it. It eventually settles on the right value for the overlap.

do_log(gf->log_level, "audio processing of %u ms new data took %d ms", new_frames_from_infos_ms,
        (int)duration);

if (duration > new_frames_from_infos_ms) {
  // try to decrease overlap down to minimum of 100 ms
  gf->overlap_ms = std::max((uint64_t)gf->overlap_ms - 10, (uint64_t)100);
  gf->overlap_frames = gf->overlap_ms * gf->sample_rate / 1000;
} else if (!skipped_inference) {
  // try to increase overlap up to 75% of the segment
  gf->overlap_ms =
    std::min((uint64_t)gf->overlap_ms + 10, (uint64_t)(new_frames_from_infos_ms * 0.75f));
  gf->overlap_frames = gf->overlap_ms * gf->sample_rate / 1000;
}

Speaking of the Whisper inference function, it is a fundamental part of this plugin, which transcribes the audio and removes any unwanted sounds. We use a few interesting things from Whisper CPP to decode the transcription into text, such as get segment text, which is limited to just one segment, and t0 t1, which gives the timings. We also sum up the probability for all the tokens that came up from the Whisper inference function to give us the general sentence probability.

const int n_segment = 0;
const char *text = whisper_full_get_segment_text(gf->whisper_context, n_segment);
const int64_t t0 = whisper_full_get_segment_t0(gf->whisper_context, n_segment);
const int64_t t1 = whisper_full_get_segment_t1(gf->whisper_context, n_segment);

float sentence_p = 0.0f;
const int n_tokens = whisper_full_n_tokens(gf->whisper_context, n_segment);
for (int j = 0; j < n_tokens; ++j) {
  sentence_p += whisper_full_get_token_p(gf->whisper_context, n_segment, j);
}
sentence_p /= (float)n_tokens;

Finally, we detect the fillers and profanities through user-defined detect and reject regular expressions. This detection allows us to process the audio and return it using circular buffers and multi-threading to ensure that everything runs smoothly.

std::regex filler_regex(gf->detect_regex);
if (std::regex_search(text_lower, filler_regex, std::regex_constants::match_any)) {
  return DETECTION_RESULT_FILLER;
}
std::regex beep_regex(gf->beep_regex);
if (std::regex_search(text_lower, beep_regex, std::regex_constants::match_any)) {
  return DETECTION_RESULT_BEEP;
}

Modifying the audio to include beep or silence:

info("beep segment, adding a beep %lu -> %u", first_boundary, num_new_frames_from_infos);
if (gf->do_silence) { // User can enable/disable modification
  for (size_t c = 0; c < gf->channels; c++) {
    for (size_t i = first_boundary; i < num_new_frames_from_infos; i++) {
      // add a beep at A4 (440Hz)
      gf->copy_buffers[c][i] = 0.5f * sinf(2.0f * M_PI * 440.0f * (float)i / gf->sample_rate);
    }
  }
}

In conclusion, the CleanStream OBS plugin offers a powerful tool that filters live audio streams from any unwanted sounds. By providing a detailed code walkthrough of the CleanStream OBS plugin, I hope to have given you a better understanding of how it works and what makes it so powerful. This is a big plugin, so covering everything in this post is impossible. So I highly recommend that you check out the CleanStream OBS plugin on GitHub and try it out yourself to experience its powerful capabilities. It’s an excellent tool that can save time and provide high-quality audio for live streams.

One of the primary things to keep in mind is to connect the keyboard before starting Ableton. Once connected, you will see a red bar above the clips that you have pre-recorded or are going to record. This bar is a selector that helps you play or record the clips. You can scroll up and down with the red selector, and the channels that are armed and ready to play will be yellow.

The arming of these channels is done through the four buttons under the sliders, which also control the output volume of each channel.

The pads on the keyboard also have different functions. The white pads stop the playing when a channel has nothing, and the black channels are armed and ready to record. The play and record buttons work as expected.

If you want to play all channels together, you can use the Pad Bank button. This button plays all the channels in that row. And to clear the clips that are playing, press the Pat Bank button again. If you want to play the second row of clips, scroll down with the knob, select the second row, and start playing.

The knobs on the keyboard can control channel parameters or sends, depending on how you set them up. Just select the channel you want to control, and start using the knobs to adjust the parameters. Since there are 4 knob you will only be able to control 4 parameters of the instrument, or the 4 channels selected by the red selector.

The back and forward buttons or the bank left and bank right keys move the selector square, allowing you to select different channels.

Remember that the keyboard has a global mode function too. Turning it off will allow you to use the ARP or the latch mode. These functions are useful for live looping and recording as they give you shortcuts to control Ableton from your keyboard, saving you time and effort.

In conclusion, the M Audio Oxygen Pro mini is a pretty capable keyboard that can do a lot of things, but not all of them are well documented. With these tips and functions, you can get started with your Ableton Live Lite 11, and start making music the way you want to.

Please refer to the GitHub repo for the full code: https://github.com/royshil/obs-backgroundremoval as I cannot put all code here, just the “interesting parts”.

Plugin Architecture

The OBS Background Removal Plugin is written in C++ and follows the plugin template provided by the OBS project. The main entry point for the plugin is the “plugin-main.cpp” file, which registers the plugin’s functions with OBS. The template provided by OBS serves as a foundation for building plugins, offering essential workflows for building and publishing the plugin across multiple operating systems.

The plugin-main.cpp file only gives the module a name and registers the plugin, for example:

MODULE_EXPORT const char *obs_module_description(void)
{
  return obs_module_text("PortraitBackgroundFilterPlugin");
}

extern struct obs_source_info background_removal_filter_info;

bool obs_module_load(void)
{
  obs_register_source(&background_removal_filter_info);
  blog(LOG_INFO, "plugin loaded successfully (version %s)", PLUGIN_VERSION);
  return true;
}

This is the struct that registers the functions of the plugin with OBS:

struct obs_source_info background_removal_filter_info = {
  .id = "background_removal",
  .type = OBS_SOURCE_TYPE_FILTER,
  .output_flags = OBS_SOURCE_VIDEO,
  .get_name = filter_getname,
  .create = filter_create,
  .destroy = filter_destroy,
  .get_defaults = filter_defaults,
  .get_properties = filter_properties,
  .update = filter_update,
  .activate = filter_activate,
  .deactivate = filter_deactivate,
  .video_tick = filter_video_tick,
  .video_render = filter_video_render,
};

Video Rendering and Effects

The core functionality of the plugin resides in the “background-filter.cpp” file. Within this file, the “video-render” function is responsible for rendering the video and utilizing the GPU for efficiency. The plugin leverages OBS’s effects system, and the video rendering process involves blending the input RGB image with the background mask using a custom effect written in HLSL (High-Level Shader Language). This blending effectively removes the background while preserving the foreground.

This is part of the code for the rendering:

  gs_eparam_t *alphamask = gs_effect_get_param_by_name(tf->effect, "alphamask");
  gs_eparam_t *blurSize = gs_effect_get_param_by_name(tf->effect, "blurSize");
  gs_eparam_t *xTexelSize = gs_effect_get_param_by_name(tf->effect, "xTexelSize");
  gs_eparam_t *yTexelSize = gs_effect_get_param_by_name(tf->effect, "yTexelSize");
  gs_eparam_t *blurredBackground = gs_effect_get_param_by_name(tf->effect, "blurredBackground");

  gs_effect_set_texture(alphamask, alphaTexture);
  gs_effect_set_int(blurSize, (int)tf->blurBackground);
  gs_effect_set_float(xTexelSize, 1.0f / width);
  gs_effect_set_float(yTexelSize, 1.0f / height);
  if (tf->blurBackground > 0.0) {
    gs_effect_set_texture(blurredBackground, blurredTexture);
  }

  obs_source_process_filter_tech_end(tf->source, tf->effect, 0, 0, "DrawWithBlur");

  gs_blend_state_pop();

  gs_texture_destroy(alphaTexture);
  gs_texture_destroy(blurredTexture);

The HLSL effect is very simple. It only multiplies the RGB values by the alpha values from the background mask:

float4 PSAlphaMaskRGBAWithoutBlur(VertDataOut v_in) : TARGET
{
	float4 inputRGBA = image.Sample(textureSampler, v_in.uv);
	inputRGBA.rgb = max(float3(0.0, 0.0, 0.0), inputRGBA.rgb / inputRGBA.a);

	float4 outputRGBA;
	float a = (1.0 - alphamask.Sample(textureSampler, v_in.uv).r) * inputRGBA.a;
	outputRGBA.rgb = inputRGBA.rgb * a;
	outputRGBA.a = a;
	return outputRGBA;
}

Neural Network Inference

To perform background removal, the plugin employs a neural network for image processing. The “video tick” function retrieves the RGB(A) input color image, which is then processed through the neural network in the “run filter model inference” function. This process involves converting the image to RGB, resizing it to the network’s input size, applying any necessary preprocessing, running the inference, and converting the output to the expected format.

Abstracted Model Class

The plugin incorporates various neural network models for different background removal scenarios. These models are abstracted using a parent class called “model,” which handles shared functionalities such as file path retrieval, input/output names, and buffer allocation. Each specific model subclass implements model-specific preprocessing and postprocessing logic, catering to the requirements of different neural network architectures.

The Model.h file holds the abstract Model class:

class Model {
  private:
  /* data */
  public:
  Model(/* args */){};
  virtual ~Model(){};

  const char *name;

#if _WIN32
  const std::wstring
#else
  const std::string
#endif
  getModelFilepath(const std::string &modelSelection)
  {
    //...
  }

  virtual void populateInputOutputNames(const std::unique_ptr<Ort::Session> &session,
                                        std::vector<Ort::AllocatedStringPtr> &inputNames,
                                        std::vector<Ort::AllocatedStringPtr> &outputNames)
  {
    //...
  }

  virtual bool populateInputOutputShapes(const std::unique_ptr<Ort::Session> &session,
                                         std::vector<std::vector<int64_t>> &inputDims,
                                         std::vector<std::vector<int64_t>> &outputDims)
  {
    //...
  }

  virtual void allocateTensorBuffers(const std::vector<std::vector<int64_t>> &inputDims,
                                     const std::vector<std::vector<int64_t>> &outputDims,
                                     std::vector<std::vector<float>> &outputTensorValues,
                                     std::vector<std::vector<float>> &inputTensorValues,
                                     std::vector<Ort::Value> &inputTensor,
                                     std::vector<Ort::Value> &outputTensor)
  {
    
    //...
  }

  virtual void getNetworkInputSize(const std::vector<std::vector<int64_t>> &inputDims,
                                   uint32_t &inputWidth, uint32_t &inputHeight)
  {
    // BHWC
    inputWidth = (int)inputDims[0][2];
    inputHeight = (int)inputDims[0][1];
  }

  virtual void prepareInputToNetwork(cv::Mat &resizedImage, cv::Mat &preprocessedImage)
  {
    preprocessedImage = resizedImage / 255.0;
  }

  virtual void postprocessOutput(cv::Mat &output)
  {
    output = output * 255.0; // Convert to 0-255 range
  }

  virtual void loadInputToTensor(const cv::Mat &preprocessedImage, uint32_t inputWidth,
                                 uint32_t inputHeight,
                                 std::vector<std::vector<float>> &inputTensorValues)
  {
    //...
  }

  virtual cv::Mat getNetworkOutput(const std::vector<std::vector<int64_t>> &outputDims,
                                   std::vector<std::vector<float>> &outputTensorValues)
  {
    //...
  }

  virtual void assignOutputToInput(std::vector<std::vector<float>> &,
                                   std::vector<std::vector<float>> &)
  {
  }

  virtual void runNetworkInference(const std::unique_ptr<Ort::Session> &session,
                                   const std::vector<Ort::AllocatedStringPtr> &inputNames,
                                   const std::vector<Ort::AllocatedStringPtr> &outputNames,
                                   const std::vector<Ort::Value> &inputTensor,
                                   std::vector<Ort::Value> &outputTensor)
  {
   //...
  }
};

This class assumes BHWC data format, but some models are BCHW. To handle that we have the ModelBCHW class which overrides some of the functions that have to do with loading and unloading the tensors from the inference session.

Plugin Properties and Customization

The plugin offers a range of properties and customization options to enhance the user’s control over the background removal process. These properties include threshold adjustments, preprocessing / postprocessing operations (e.g., smoothing, feathering), GPU options, model selection, and more. By leveraging the plugin’s properties, users can fine-tune the background removal based on their specific streaming or recording needs.

Defining properties on the plugin static obs_properties_t *filter_properties(void *data) function will create UI elements:

All of the functions have access to a data pointer that holds important data that we need for processing the video:

struct background_removal_filter : public filter_data {
  float threshold = 0.5f;
  cv::Scalar backgroundColor{0, 0, 0, 0};
  float contourFilter = 0.05f;
  float smoothContour = 0.5f;
  float feather = 0.0f;

  cv::Mat backgroundMask;
  int maskEveryXFrames = 1;
  int maskEveryXFramesCount = 0;
  int64_t blurBackground = 0;

  gs_effect_t *effect;
  gs_effect_t *kawaseBlurEffect;
};

This for example is part of the function that gets the UI properties and stores them in the data struct:

static void filter_update(void *data, obs_data_t *settings)
{
  struct background_removal_filter *tf = reinterpret_cast<background_removal_filter *>(data);
  tf->threshold = (float)obs_data_get_double(settings, "threshold");

  tf->contourFilter = (float)obs_data_get_double(settings, "contour_filter");
  tf->smoothContour = (float)obs_data_get_double(settings, "smooth_contour");
  tf->feather = (float)obs_data_get_double(settings, "feather");
  tf->maskEveryXFrames = (int)obs_data_get_int(settings, "mask_every_x_frames");
  tf->maskEveryXFramesCount = (int)(0);
  tf->blurBackground = obs_data_get_int(settings, "blur_background");
//...

Conclusion

The OBS Background Removal Plugin has emerged as a valuable tool for content creators seeking virtual green screen capabilities and seamless background removal in OBS. Through a well-structured architecture, efficient GPU utilization, and integration with neural network models, the plugin delivers high-quality results in real-time. By exploring the code walkthrough provided above, developers can gain insights into building similar plugins and harness the power of OBS’s plugin system.

As the plugin continues to evolve and improve, its widespread adoption within the streaming community highlights the value it brings to content creators. If you’re interested in delving deeper into OBS plugin development or learning more about the OBS Background Removal Plugin, refer to the official OBS plugin documentation for comprehensive resources and guidance.

In this tutorial, we will explore how to build a simple Telegram bot using serverless with TypeScript and AWS Lambda. We’ll leverage the power of AWS services such as API Gateway and DynamoDB to create a highly scalable and efficient bot. While there are various tutorials available online, this guide aims to provide a more comprehensive and detailed approach. So, let’s dive in!

Setting Up the Server

To begin, we define a CloudFormation template using serverless that includes a single function serving as the entry point for our Telegram bot. This function, called “webhook,” resides in the index module and is responsible for handling incoming messages. TypeScript allows us to benefit from type safety, providing clear understanding of the event types passed from the API Gateway proxy into the Lambda function. Although TypeScript code may appear more verbose, it proves advantageous in the long run.

functions:
  webhook:
    handler: index.webhook
    events:
      - http:
          path: webhook
          method: post

This is the function entry in index.ts

export const webhook = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  const bodyParsed = JSON.parse(event.body!);
  console.log("bodyParsed", bodyParsed);
  // ...
}

Configuring DynamoDB

In addition to the server setup, we create a DynamoDB table to store our bot’s data. This table includes essential parameters and an index to facilitate efficient querying and content modification. While the complete list of attributes is not mentioned here, we focus on indexing for our specific use case of handling to-do items within the bot.

This is how the config looks in serverless.yaml

resources:
  Resources:
    ExampleDynamoDbTable:
      Type: 'AWS::DynamoDB::Table'
      DeletionPolicy: Retain
      Properties:
        AttributeDefinitions:
          -
            AttributeName: id
            AttributeType: S
          -
            AttributeName: chatId
            AttributeType: S
        KeySchema:
          -
            AttributeName: chatId
            KeyType: HASH
          -
            AttributeName: id
            KeyType: RANGE
        GlobalSecondaryIndexes:
          -
            IndexName: chatId-index
            KeySchema:
              -
                AttributeName: chatId
                KeyType: HASH
            Projection:
              ProjectionType: ALL
        BillingMode: PAY_PER_REQUEST
        TableName: ${self:provider.environment.DYNAMODB_TABLE}

The, for example in the TS side we can make operations on the DynamoDB table:

// Find all todos for this chatId
const r = await dynamoDb
    .query({
          ...params,
          KeyConditionExpression: "chatId = :chatId",
          ExpressionAttributeValues: {
            ":chatId": chatId.toString(),
          },
        })
        .promise();
if (r.Items == undefined || r.Items!.length == 0) {
    await bot.sendMessage(chatId, `0️⃣ No TODOs found`);
    globalResolve("ok");
    return;
}
let message = "";
for (const todo of r.Items!) {
    message += `➖ ${todo.what}\n`;
}
await bot.sendMessage(chatId, `📝 Current TODOs:\n${message}`);

Handling Telegram Messages

The entry point for our bot is the Lambda function responsible for processing incoming messages. To achieve this, we parse the message and chat ID from the message’s body. Leveraging the powerful “node-telegram-bot-api” package, we process the parsed message and send it to the bot for further handling. However, since we are using a Lambda function from AWS, we need to parse the message from Telegram and utilize the “processUpdate” function provided by the “node-telegram-bot-api” package. This sets off a chain of events that execute the bot’s commands.

const bot = new TelegramBot(token);

let globalResolve: (value: any) => void = () => {};

export const webhook = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  const bodyParsed = JSON.parse(event.body!);
  console.log("bodyParsed", bodyParsed);
  await new Promise((resolve, reject) => {
    globalResolve = resolve;
    bot.processUpdate(bodyParsed);
    // set timeout to 3 seconds to resolve the promise in case the bot doesn't respond
    setTimeout(() => {
      // make sure to resolve the promise in case of timeout as well
      // do not reject the promise, otherwise the lambda will be marked as failed
      resolve("global timeout");
    }, 3000);
  });

  // respond to Telegram that the webhook has been received.
  // if this is not sent, telegram will try to resend the webhook over and over again.
  return {
    statusCode: 200,
    body: JSON.stringify({ message: "function executed successfully" }),
  };
};

Notice that I’m using a global promise to hang until all the bot’s work is done. This is because Lambda will not wait for any async operations. The TelegramBot is an event queue so things happen out of sync. The bot handlers will resolve the promise and then the lambda will complete with a 200 status.

Implementing Bot Commands

We introduce various bot commands such as listing to-do items, adding new items, and removing existing ones. By following standard boilerplate code, we enable the bot to perform these actions. The complete code for these commands and additional functionalities can be found on GitHub for reference.

For example the /add command:

bot.onText(
  /\/add (.+)/,
  async (msg: TelegramBot.Message, match: RegExpExecArray | null) => {
    const chatId = msg.chat.id;
    const what = match![1];
    const id = randomUUID();
    try {
      await dynamoDb
        .put({
          ...params,
          Item: {
            id,
            chatId: chatId.toString(),
            what,
          },
        })
        .promise();
      await bot.sendMessage(chatId, `✅ Added TODO: ${what}`);
    } catch (error) {
      console.error(error);
      await bot.sendMessage(chatId, `❌ Error adding TODO: ${what} (${error})`);
    }
    globalResolve("ok");
  }
);

Note that I’m calling the global resolve to signal the finish of the bot’s work for this message and clear the Lambda run.

Deploying the Bot

With the server and bot code ready, we need to deploy the solution. Using the serverless framework, we run the deploy command, which sets up the infrastructure on AWS, including the Lambda function and API Gateway. After deployment, we obtain the HTTPS endpoint URL, which we need to configure as the webhook for our Telegram bot. This connection enables Telegram to send messages to our deployed bot.

$ ./node_modules/.bin/serverless deploy

You can also watch your AWS account for the Lambda’s existence, as well as DynamoDB and the API Gateway.

You’re going to have to register your webhook with Telegram by running the following command:

curl --request POST --url "https://api.telegram.org/bot<BOT_TOKEN>/setWebhook" --header 'content-type: application/json' --data '{"url": "<URL>"}'

Use the URL that you received from serverless deploy for the POST endpoint. At that point the bot lambda should start receiving messages from Telegram.

Here’s proof:

Monitoring and Debugging

To monitor and debug our bot, we utilize various tools provided by AWS. CloudWatch allows us to view logs generated by our Lambda function, helping us identify any issues and understand the flow of data. Additionally, DynamoDB provides a handy interface to verify the stored data and perform tests on the bot’s functionality. Utilizing console logs and try-catch blocks for error handling and logging ensures a smooth debugging experience.

Cloudwatch: (all console.logs from your code will appear as lines there)

The Lambda itself will also show the invocations:

Conclusion

By following this tutorial, you’ve learned how to build a serverless Telegram bot using TypeScript and AWS Lambda. Leveraging AWS services like API Gateway and DynamoDB, we’ve created a scalable and efficient bot infrastructure. TypeScript’s type safety provides clarity in handling events, while the “node-telegram-bot-api” package simplifies interaction with Telegram’s API. With the power of AWS and the ease of TypeScript, you can create sophisticated bots tailored to your requirements. The complete code and instructions for this project are available on GitHub, allowing you to start building your own bot right away. Happy bot building!