In this post I show how to simulate Azure IoT device D2C (Device to cloud) messages telemetry, and Device Twin Reported Properties using MQTTX CLI.

Introduction

I have been working in several Azure IoT projects in the past 3 years, where MQTT was the preferred device/server communication protocol. In these projects, there has been a repeatable need to generate MQTT messages to simulate devices’ telemetry, without going through the Azure CLI.

One of the tools that I found recently was MQTTX CLI, which is a light-weight client that can subscribe to and publish topics to MQTT broker, and has a solid and dynamic templating functionality based on JavaScript that makes generating telemetry easy.
Quoting from their website:

“MQTTX CLI is an open source MQTT 5.0 CLI Client and MQTTX on the command line. Designed to help develop and debug MQTT services and applications faster without the need to use a graphical interface.“

Let’s have a look at the Simulate feature.

MQTTX CLI Simulate Feature

The Simulate feature can publish messages with dynamic content generated based on your template file.  The feature uses the simulate command. Let’s examine the parameters to.

Note: behind the scenes, the Simulate command depends on the Publish command, so make sure to check the Publish parameters necessary, e.g. hostname, username, password..etc

In addition to the basic parameters to publish messages, the Simulates command has the following parameters to control the simulation:

• –count: Number of connections: represents the number of MQTT connections that will be created in the process.
• –interval-message: interval of publishing message to the broker (default: 1000ms)
• –limit: the number of messages to publish, 0 means unlimited (default: 0)
• –client-id: the client id, support %i (index) variable. Check the --count parameter above to see how it is used.
• –topic: the message topic. If your topic will depend on the User, Client ID, or the count of simulated messages, then you can use %u (username), %c (client id), %i (index) variables in the value.
• –file: the JavaScript Module file that will be used to generate the payload. This JavaScript has a specific contract that you should adhere to so that it can be used to MQTTX CLI to generate the payload.

Note: that all the connections will be established by the same client id unless you pass “%i”, if you do then each connection will have its unique generated client id based on the count. e.g. clientid_1, clientid_2…etc.

Let’s have a look at the example template file in the docs, this simple simulator generates a simple payload with “temp” and “hum” properties. (in order to use it you pass the name of the file to the --file parameter).

/**
 * MQTTX Scenario file example
 *
 * This script generates random temperature and humidity data.
 */
function generator(faker, options) {
  return {
    // If no topic is returned, use the topic in the command line parameters.
    // Topic format: 'mqttx/simulate/myScenario/' + clientId,
    message: JSON.stringify({
      temp: faker.number.int({ min: 20, max: 80 }), // Generate a random temperature between 20 and 80.
      hum: faker.number.int({ min: 40, max: 90 }), // Generate a random humidity between 40 and 90.
    })
  }
}
// Export the scenario module
module.exports = {
  name: 'myScenario', // Name of the scenario
  generator, // Generator function
}

You can see that the generator function is the contract that will be used by the tool to generate the simulated payload. It takes two parameters: faker that you can use to generate random and fake values, and options if you want use any of the passed options to the command.

All what you need to do in your own file is to implement the generator function with your custom logic to generate the payload. For more inspiration you can look at some of the built-in simulation files.

Simulate Azure D2C Message

Now comes the most important part, sending simulated telemetry to Azure IoT Hub. Microsoft already provided some code samples to use MQTT directly to send messages. The highlight is that we need to send a message to the topic devices/{device_id}/messages/events/

We need to send a message with the following parameters (make sure to replace the placeholder tokens with yours):

• Host: “{iothub_name}.azure-devices.net”
• Port: 8883
• Client Id: “{device_id}”
• User: “{iothub_name}.azure-devices.net/{device_id}/?api-version=2018-06-30”
• Password: “{sas_token}” (this is generated by this command, choose a duration value that suits the lifetime of your test).
• Topic: “devices/{device_id}/messages/events/”
• Message: The payload of your telemetry
• CA certificate: if you want to validate the IoT Hub’s certificate, then you can provide the certificate that can be found here. Otherwise you can pass the --insecure parameter to ignore validating the certificate.
• MQTT Version: this is necessary with MQTTX CLI as its default protocol version is 5.0, while Azure IoT Hub’s one is 3.1.1.

Assuming that we have a device called “emad”, and an Azure IoT Hub called “youriothub”, and a template file called “simulatedTelemetry.js”, then the final command should look like this:

mqttx simulate \
  --file simulatedTelemetry.js \
  -c 10 \
  --interval-message 1000 \
  --insecure \
  -q 1 \
  -V 3.1.1 \
  -h "youriothub.azure-devices.net" \
  -t "devices/emad/messages/events/" \
  --client-id "emad" \
  -u 'youriothub.azure-devices.net/emad/?api-version=2018-06-30' \
  -P 'long-sharedaccesstoken' \
  -l mqtts

Simulate Azure IoT Device Twin Reported Properties

We can also use the same technique above for the Device Twin Reported Properties. The only difference will be the topic value which should be \$iothub/twin/PATCH/properties/reported/.

If you don’t want to use simulation to send the Reported Properties and just send it once with a specific payload, you can use the pub command with a simple JSON payload. Like the following:

mqttx pub \
  --file-read devicePropertyPayload.json \
  --insecure \
  -q 1 \
  -V 3.1.1 \
  -h "youriothub.azure-devices.net" \
  -t "devices/emad/messages/events/" \
  --client-id "emad" \
  -u 'youriothub.azure-devices.net/emad/?api-version=2018-06-30' \
  -P 'long-sharedaccesstoken' \
  -l mqtts

Conclusion

By this, I hope you can use MQTTX CLI to simulate device telemetry and Device Twins Reported Properties.

The post Simulate Azure IoT Telemetry And Device Twin using MQTT Directly appeared first on Emad Alashi.

Update: Tatham had a great thread of tweets about this posts where he filled several gaps in this post, including the need of the MQTT server. Check it out here: https://twitter.com/TathamOddie/status/1357904732027637760?s=20

In this post, I describe what I did to set up an M5 Atom Lite with a capacitive soil moisture sensor, configure it with ESPHome and connect it to Home Assistant. The Home Assistant is deployed in a Docker container on my Synology NAS (DS220+), and my dev machine is a MacbookPro.

Important: In this post, I rely on Tatham Oddie’s in-valuable video, which is a comprehensive guide and introduction to HomeAssistant, ESPHome, and M5 Atom Lite.

So I highly encourage you to check that video before you continue.

Disclaimer: I am not expert in IoT or electricity, please consider this context with this post!

Terminology and References

To understand the rest of the article, this is a quick run-through of the main terms:

• M5 Atom Lite: an ESP32 device that is packaged nicely. You can program it with Arduino Framework (C++), or MicroPython.
• Capacitive Soil Moisture Sensor: measures the moisture of the soil, and produces the readings as analog stream.
• Home Assistant: A home automation server/tool. Connects to all the home-assistant-ready devices and presents a web dashboard where you can read and control these devices.
• ESPHome: a project through which you can program your ESP device and make it home-assistant-ready.

Setting up Home Assistant with Docker

My Synology NAS DS220+ was a perfect candidate to host the Home Assistant server, this is a guide that will tell you how to set it up with Docker.

However, there is a small problem: if you checked Tatham’s video above, you will see that he added ESPHome to Home Assistant as an add-on. The problem is that the Supervisor menu item, through which you can add add-ons to Home Assistant, doesn’t exist in the Docker image of Home Assistant.

To solve this problem, I had to run an independent ESPHome instance in another Docker container to program my device.

To do that, I took the same steps in the guide I mentioned above for Home Assistant. The only difference is that:

1. Used the image “esphome/esphome:latest” of course!
2. I mounted to a different folder under “docker” I called “esphome”
3. And I exposed port 6052

Now you have an ESPHome instance running, and can create the first Node and application on the device.

Creating the Node on ESPHome

To access ESPHome dashboard, I navigate to HTTP://[nas-ip-address]:6052. And to make things easier for me, and for my password manager, I use the awesome service https://nip.io to give this address a proper name like https://esphome-[nas-ip-address]:6052.

In the ESPHome dashboard, I follow the wizard to create my Node, which is a representation of my device. In there,  I can program my device using YAML, which will eventually generate C++ code.

The ESPHome editor already comes with a linter, so your mistakes will be corrected for you in the browser. However, if you want to have access to the generated C++ code, and examine the YAML file in VSCode, you can navigate to the code on the docker folder on your NAS.

Note: When you install Docker on your NAS, the “docker” folder will not be visible on the network. So you have to untick the box ‘Hide this shared folder in “My Network Places”‘.

Generating the code

I started with a very basic YAML to control the LED, I hit Compile menu item in the node to generate the C++ code that will be deployed to my device. Now I am ready to install the application on my device.

esphome:
  name: m5atomlite2
  platform: ESP32
  board: m5stack-core-esp32

wifi:
  ssid: "wifi-ssid"
  password: "wifi-password"

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "M5Atomlite2 Fallback Hotspot"
    password: "0yyQtzyIpw3z"

captive_portal:

# Enable logging
logger:

# Enable Home Assistant API
api:
  password: "123"

ota:
  password: "123"

light:
  - platform: fastled_clockless
    chipset: SK6812
    pin: 27
    num_leds: 1
    rgb_order: GRB
    name: "FastLED Light"

Note: Make sure to use the right password for your wifi, and make sure you pick the right configurations for your LED: the pin number on your board and the chipset number of the LED you have.

When you create the Node in the ESPHome dashboard and compile your YAML file, a folder with the same name will be created in the docker/esphome folder on your NAS (that is if you have followed the steps above, otherwise use the name of the folder you mapped).

Flashing the device for the first time

Update: Eranjo mentioned in the comments that the latest flasher version might not work, so you might need a previous version. Check his comment below for more details.

Cccording to Tatham’s video, I need to use something like esphome-flasher, but there was no version for macOS, and I need to find an alternative.

Tatham mentioned that there are many ways to flash an ESP device, but I consulted with my friend @Amal Abeygunawardana and found his suggestion interesting! Use the PlatformIO IDE on VSCode. This also gave me the opportunity to learn about programming my device without ESPHome, using the Arduino Framework (another story).

Once I opened the folder using VSCode, the PlatformIO IDE extension discovered that this is a folder it understands, and it launched the PlatformIO IDE hello page.

I made sure my device is connected to my computer through USB and hit the PlatformIO: Upload command in VSCode. The command compiled and uploaded my firmware binary to the device.

Update the device over the wifi

Now that my device is set up for the first time, I can use ESPHome to upload to the device over the wifi.

However, in my case, I had a problem: the ESPHome cannot find the device over the network using its name devicename.local. When I check the devices connected to my network, I can see my  device and I can ping it! But the ESPHome is still blind to it.

As a workaround, I had to use the property use_address to give an explicit IP address to the node. I gave it the same IP address the DHCP already has given it before, so the wifi section of the YAML file became like this:

wifi:
  ssid: "wifi-ssid"
  password: "wifi-password"
  use_address: 192.168.0.21

After doing that I managed to upload new changes over the wifi. (Please if you know a better solution let me know :)).

Adding the device to Home Assistant

In the Home Assistant dashboard, I navigated to Configuration menu item on the left, hit Integrations, and then at the bottom right corner hit ADD INTEGRATION. Once I am represented with a dialogue I searched for ESPHome.

I put the IP address of the device and magic happens! Under Devices I could see my device, and when I navigated to the details I saw the LED control Entity.

Connecting the moisture sensor

Ok great, so far so good, but we should not forget what we are here for: a moisture sensor!

I followed this video, but since my device is not Arduino, I had to figure out which pin I should use, and it was pin 33. Thanks to the form factor for the M5 Atom Lite, I only needed jumper wire.

Programming for the moisture sensor

Now, all that I have to do is to search for how to configure my YAML file and add an Entity to read data from the moisture sensor. When I searched ESPHome, I couldn’t find a straightforward way to do that, but I stumbled upon the Analog to Digital Sensor, and it appeared to be the answer.

So I added the following segment to the YAML file (Valeria is the name of our plant :D):

sensor:
  - platform: adc
    pin: 33
    name: "Valeria"
    update_interval: 500ms
    attenuation: 11db
    filters:

Of course, the update interval is too excessive, but it is good for debugging purposes when you dip the sensor in a cup of water.

Important Note: depending on the voltage of the sensor, you need to tune the attenuation property, the default is 0db, and I had to change it to 11db. Read the documentation of the Analog to Digital Sensor above for more information.

In ESPHome, I compiled and uploaded the new code, and managed to see the voltage readings next to the LED Entity, success! However, it was basic readings, and I needed a percentage. I found this post on Reddit when I was trying to figure out the Entity, and they already solved it for me :).

So the code below uses the Filter attribute. It takes the raw value of the readings, and passes it as a parameter to the subsequent function to return the result accordingly:

sensor:
  - platform: adc
    pin: 33
    name: "Valeria"
    update_interval: 500ms
    attenuation: 11db
    filters:
    - lambda: |-
          if (x > 3.74) {
            return 0;
          } else if (x < 1.53) {
            return 100;
          } else {
            return (3.74-x) / (3.74-2.85) * 100.0;
          }

Of course, you have to find your lowest and highest raw readings to get the right formula for your sensor. In this case, the highest was 3.74, and the lowest was 1.53. (Update: these are not really accurate values, which explains why I get more than 100 in my video above, I also didn’t remove the label “v”, embarrassing!)

Too much power consumption, let’s use Deep Sleep

The M5 Atom Lite is a small microcontroller, but this doesn’t mean that it doesn’t consume a lot of power. Putting this in a plant pot powered by battery will not last long.

The good thing is that we can use the Deep Sleep mode, once the device is put in deep sleep mode, it will reduce power consumption and the battery will last longer depending on how long you put the device in this mode. For more information about ESP deep sleep, check the following article.

To put the device in deep sleep using ESPHome, we will update our YAML to include the Deep Sleep component:

deep_sleep:
  id: deep_sleep_1
  run_duration: 10s
  sleep_duration: 2min

This will put the device into deep sleep mode for 2 minutes, and then will wake up for 10 seconds to allow the other components to do their job, and then will sleep again for 2 minutes.

But Deep Sleep has a problem…

There is a small problem, though, when you put the device in deep sleep mode: the device will shut down a lot of its capabilities, including CPU and wifi.

This means that the device will not be reachable for two minutes, and will only stay awake for 10 seconds. So if we want to update the firmware, this will be challenging.

So how can we solve this problem? well, if we can prevent the deep sleep mode the FIRST thing when the device wakes up, then we can update its firmware. Once we update the firmware, we re-enable deep sleep (not my genius idea, this is a common practice :))

How to achieve this I hear you say? The answer is MQTT. MQTT is a lightweight protocol to transmit messages between devices. The biggest advantage of this protocol is the persisted message concept: a client can push a message to the broker (server), and the message will stay there until another client (or same client) sends a new value to overwrite it. (also not my idea :D)

So if we push a message to the broker with a value like “turn off deep sleep”, and we configure the device to read from this broker the first thing when it wakes up, then we can achieve our goal!

Luckily this is easy with ESPHome, we need to update our YAML file to use the MQTT component (God I love ESPHome!):

mqtt:
  broker: 192.168.0.231
  port: 1883
  on_message:
    - topic: ota_mode
      payload: 'ON'
      then:
        - deep_sleep.prevent: deep_sleep_1
    - topic: sleep_mode
      payload: 'ON'
      then:
        - deep_sleep.enter: deep_sleep_1

The above segment will program the device so that it will read a message from the server 192.168.0.231, specifically from the Topic “ota_mode” (the name of the topic can be anything you want). In the case there is a message, we check the payload, if it equals to ON, then we prevent the deep sleep component we configured above. However, if there is another message under the topic sleep_mode, then go back to sleep mode.

Ideally, you don’t want two messages to represent one state, but let’s just go with this flow for now. Check the documentation of the MQTT Component to see how you can use Lambdas for tighter control (not AWS Lambda!).

Oops, but we don’t have an MQTT server!

Did I mention I love Docker? We run an MQTT server in a container on NAS just like we did for Home Assistant and ESPHome above. For that, I chose the Mosquitto server, which already has a container image.

The only thing I want to bring your attention to is that I mapped a file to the container on the path /mosquitto/config/mosquitto.conf to host the configuration. And I had the following configuration content:

allow_anonymous true

listener 1883

If you don’t put the second line, the server will only accept messages from clients on the same machine. Please note as well that this is not a secure setup, so please be careful with your choices.

Now, once I want to put my device OUT of sleep mode, I just send a message to the topic “ota_mode” with the value ON. And make sure that the topic “sleep_mode” doesn’t have the value ON. To do that I use the MQTT client “MQTT Explorer“, but you can also run the following command on your NAS through SSH:

docker exec -it [nameOfMosquittoContainerOnNas] mosquitto_pub -V mqttv311 -h localhost -d -t ota_mode “ON”

Conclusion

That was actually a lot of fun, and it’s just astonishing how good Home Assistant and ESPHome is. I am usually suspecious of the quality and efficiency of any product that generates code to achieve something, especially from a DSL-like language. In this case, things look pretty solid!

Let me know if you have any questions about this setup, I’d love hear from you, and I hope this helps you in your journey.

The post M5 Atom Lite, Home Assistant, ESPHome, and Capacitive Soil Sensor appeared first on Emad Alashi.

This post explains my latest experience in contributing to open source software with project KEDA.
Last week, the KEDA team have accepted and merged a Pull Request I created to support Pod Identity in the Event Hub scaler. @Simon Wight, an amazing community person, encouraged me to write about my experience (cheers for the nudge Simon :)), and this post is that.

I will explain how it all started, what I did to get it done, and all the fun and challenges that happened during that.

How it all started

My relation with KEDA started in mid 2019, I think it was a presentation by Jeff Hollan. At that time I was already working with Kubernetes, and I was already familiar Azure Functions. I have always been curious about scaling, whether it was Azure Functions or Containers, and KEDA came to complete this missing piece.

This explains the “interest” part of the contribution story, because no one can contribute to open source project, and give from their own time, unless the project is of interest to them. You need this spark to keep you going, because it’s not easy and at some point you WILL suspect yourself and say: “Oh man why am I doing this?!”.

Later in the year, I also was interested in learning Go, and this brought me closer to the idea that I can contribute to a project like KEDA.

One Step at a Time

The Pull Request (PR) that sparked this post was not my first contribution to KEDA, but it is definitely the biggest up until the moment of writing these words. I had to go by smaller steps, otherwise I would have been frustrated and left the thing from the beginning. So how did it start?

Understanding The Code, and Contributing Docs

Before I had the intention to contribute KEDA, I was curious to understand how it works. I have always believed that in order to use a tool efficiently, you have to go one level of abstraction deeper, and if I wanted to understand how KEDA works then I need to check out the code.

So I did that! I cloned the code, and I started examining it, trying to understand it. This didn’t only give me an advantage of understanding KEDA, it was a very good way to learn Go itself. It’s worth mentioning here that I didn’t even try to build it; I didn’t have the intention to contribute at that moment and I didn’t want to go through the hassle of installing the prerequisites.

Ater I became comfortable with how it works, I noticed that there are gaps in the documentation that are key to using KEDA in its full potential. So I decided to contribute to the documentation. I created my first PR to the docs, explaining how to write your own External Scaler.

This had two major benefits that helped in my future PRs:

1. I became familiar with the maintainers

I joined the KEDA Slack channel and had a couple of conversations with the maintainers on what is missing and what I intended to cover. This created a connection with the maintainers and allowed them, and myself, to understand the expectations, their method, and way of thinking. In my case the maintainers were: Tom Kerkhove, Zbynek Roubalik, and Ahmed ElSayed who are FANTASTIC people; they are helpful, encouraging, and appreciating, kudos to them!

2. I know more about the code now

After that, I was in a much better position to contribute code, I know how things work, in general, I know where to find code and how to navigate it.

This also put me in a good position to speak about it in user groups and conferences, and even give workshops.

In addition to that, I started writing an External Scaler live on Twitch. External Scalers are not part of KEDA’s binaries; they are an independent deployments that uses gRPC to communicate with KEDA.

All of this increased my bond with the project.

Contributing The First Lines of Code

This is where things get harder, but also more exciting!

Let’s just build it for now!

I already have cloned the repository, and already navigated it, now it’s time to run it. I didn’t want to do anything except a successful build and deployment; if I do these two successfully, I would have finished more than two thirds my way towards the first commit!

The reason is that there can be a lot of things that go wrong, even for such a simple task. You are setting a new environment from scratch, and making sure all the prerequisites and dependencies are installed the right way with the right version.

Reading the Contribution guide

So I read the contribution guide, this is where you should start with any OSS you want to contribute to. Rather than fighting your way by “discovering” code, reading the contribution guide will show you at least the entry point. (Dah! but we also don’t read manuals to set up a washing machine, do we now!)

However, the documentation was not complete at that time, and I had to do some discovery in order to have a successful build and deployment, and the right place for that discovery was the Makefile (or the build script in other repos).

Following the Contribution guide, and wiggling my way through reading the Makefile, I managed to have a successful build and local deployment. This was another opportunity to contribute to the documentation, and I did (along with other stuff).

Finding the smallest code contribution possible

Ok, now I am ready! I want to find the smallest contribution that doesn’t require a lot of effort or knowledge. The easiest way to find that out is to search for bugs in the repo’s Issues.

Why a bug? because solving bugs is like solving a jigsaw puzzle that is already very close to being completed; a lot of the pieces are already there, you just need to fit the last couple of ones. Also, the expectation is very clear; this is why maintainers classify it as a bug, so achieving the result should be straight forward.

For my luck, when I was trying to run my deployment locally, I was using an sample that had a bug already! I thought it was something I did, so I searched for it in the repo’s Issues, and I found that someone already reported it, perfect! It seems it’s a legitimate bug, and maybe I should try solving it.

So I rolled my sleaves, opened the code, searched for that error string in the code, and found it. Traversing it back, I understood what the problem was, and thus the solution was relatively simple.

Code contribution workflow, and creating the PR

So what does the code change workflow look like?

This will depend on how comfortable you are with Git, but I do the following:

1. Fork the repo to my GitHub account
2. Clone my new fork to my disk (name the remote “eashi”)
3. Add the original repo as another remote “origin”. So by now I have two remotes: “origin” from the remote repo, and “eashi” that is my fork. (you can swap the names, up to you)
4. Create a branch for the new code
5. Push the new branch to my repo
6. Create a PR from the new branch in “eashi” to the “main” branch in the “origin” repo.
7. After the maintainers merge the PR to the “main” branch in the project, I pull from the “main” in “origin” to the “main” branch in “eashi”.

One step I didn’t mention it above: I pull from the “origin main” to my repo’s “main” regularly if necessary.

Creating the PR

After I ran the code and validated that my code indeed solves the problem, I created a PR.

The description of the PR should make the goal of the code change clear, should describe how this PR achieves this goal, and it’s best if it includes reference to the issue that the PR is established upon.

Most of the projects these days provide a check-list of prerequisites that have to be met before the PR can be merged and accepted. Maintainers try to make this easy by providing a template, when the contributor creates the PR the initial description of the PR would explain how the PR should be structured.

In KEDA’s case, it’s 4 items:

• Commits are signed with Developer Certificate of Origin (DCO)
• Tests have been added
• A PR is opened to update the documentation on https://github.com/kedacore/keda-docs
• Changelog has been updated

I made sure that I ticked all the boxes in the PR.

In addition to this manual check-list, there is another automated check-list that is PRs go through: the code needs to compile, the tests need to run successfully, and the code should be scanned.

Of course these checks also different from project to project, and with the help of GitHub Actions this can run on every PR created, or every update to the PR.

Be patient now!

After you have created the PR, it will take time: days, sometimes weeks! DON’T be pushy, hasty, rude, or disappointed. Maintainers are humans, and they have families and priorities. This might not be their fulltime job, and this PR might NOT be the top of their priority.

It’s good that you have given from your time to contribute to the project, maintainers will really appreciate it, trust that if it takes time before they merge or comment on your PR doesn’t mean that they don’t value your contribution.

Receiving feedback on PR, and actioning on it

It’s very rare that the PR will be merged without comments or feedback, most of the time maintainers will have questions, at least. Any line of code is added to the repo is a responsibility, and it’s good for all parties, including you, to only allow code that is of good quality and of a good reason.

Don’t take the feedback personally, if you don’t agree with their comments try to have a good conversation about it, assume their best intentions and try to convince them why things should be done your way. If they are not convinced don’t be frustrated, after all it’s their responsibility to be that quality gate.

Just a reminder here, that we are still talking about a small contribution like a bug for example. In theory, this should have very little debate.

But I want to contribute a bigger and more important code

This brings us to the PR that sparked this post. After I got more comfortable with the project as explained above, I felt I can take a bigger change.

So I started looking for Issues that are little bigger than a bug, and I found one that is perfect for me! At that time, I was interested in Azure’s Pod Identity, I read a couple of articles about it and I got the basic concepts, but it was not too clear in my head.

The issue I found was “support AAD Pod Identity authentication for azure event hubs“. It’s something I am already interested in, it’s a little bit bigger than a bug, and it’s something I THINK I can deliver. Was I confident that I could do this? Not really, and that’s alright! If you find yourself in such a situation don’t worry, try to embark on the mission and you will learn your way through.

Sometimes there isn’t a clear Issue that makes things easier for contributors, in this case I urge you to reach out to the contributors on chat/Twitter or whatever means to express your interest in helping, they will guide you.

So I put a comment on the Issue to express my interest in doing it; there is a possibility that someone is already working on this Issue, and I don’t want to offend anyone, and I don’t to waste my effort.

How much effort was it?

From the minute I showed interest up until the PR was merged, it was 40 days. The change wasn’t big, it was mainly in two significant files, and around 60 lines of code. So where did the time go?!

1. Investigating and researching.

   A lot of my time was investigating and researching; trying to understand the libraries I am depending on, and trying to understand how Pod Identity REALLY works.

2. Setting the dev environment

   I already have mentioned above that I contributed small code changes before, but when I wanted to do this code change I have messed up my dev environment by installing different versions of the dependencies.

   This caused some disruption and urged me to go the “Remote Containers” path, for which there was already some guidance in the Contribution guide. However, for some reason things didn’t work for me, and I had to wiggle my way through again to set things up.

   I wanted to have a separate docs contribution for that part, but magically the bad behaviours stopped appearing . Keep in mind that this might happen to you too :).

3. Designing

   This might be a little bit too mouthful for such a change, but I was trying to achieve the goal of the issue with the least amount of disruption to the code, and still adhere to the general spirit of the code.

4. Troubleshooting

   This one was really hard, because debugging in Kubernetes is not too straight forward. I had to fill the code with logging statements and re-deploy everytime I figured out that I am in a blind spot. Contrary to the traditional “put a break point” way.

5. Code, deploy, and test cycle

   The cycle of introducing the change, deploy it, run it, check the result, and then change code again was time consuming. Especially that it involved deploying to Azure AKS because the feature was Azure specific.’

This is where most of my time went, all from late nights and weekends.

Having an Azure Subscription

I am fortunate enough to have a subscription that I am not paying money for from my own pocket, this allowed me to really contribute to this feature. If I didn’t have such a subscription it would have been expensive contribution to OSS for me.

Thankfully as well, Microsoft announced in Ignite that you can shut down an AKS cluster “az aks stop/start..“, that made a good difference :).

Congratulations, The PR is Merged, Now What?

This needs a celebration ! But this also means responsibility, I still worry at some point that my code might have a bug, but this comes with the package, and I believe that I have to keep an eye on the Issues to make sure I can fix whatever is reported.

Conclusion

It was a long journey, but it doesn’t have to be that way. Everybody is different; their priorities, capabilities, interest, time…etc.

So this isn’t necessarily a guidance, but it’s my experience in contributing to OSS, and I hope it will help you navigate your way through. 🙂

The post Contributing to Open Source Software with Project KEDA appeared first on Emad Alashi.

Below is the recording of the session, make sure to follow their channel because they post regularly. Also all feedback is welcome :).

The post “Azure Functions on Kubernetes” Talk With Integration Down Under Meetup appeared first on Emad Alashi.

• Hardware
• Software (this post)
• Humanware

As I have said before, I have learned a lot from amazing streamers like @noopkat and @csharpfritz so you will find a lot of this content matches theirs.

OBS Streaming Configuration

OBS is the main software that streams to the streaming service (Twitch in my case). I used Streamlabs at the beginning, but because it’s just an abstraction over OBS, I faced some limitations when I wanted to try different plugins. So I preferred to play directly with the OBS itself.

Media is not really my expertise, and I don’t like to stray away from the default configuration OBS has for streaming, so I let OBS do the test and suggest the best configuration. The following is the suggested configuration:

• Output
  • Video Bitrate: 2500 Kbps
  • Encoder: Software (x264)
  • Audio Bitrate: 160
• Video:
  • Base (Canvas) Resolution: 1920×1080
  • Output (scaled) Resolution: 1280×720 (I am not sure if this is the best configuration, but this is what I am using now ).
  • Common FPS Values: 60
• Audio:
  • Sample Rate: 44.1 kHz
  • Channels: Stereo

OBS Scene Setup

In OBS, the way you construct the scene is by creating layers of Sources, each Source can be an image, web page, video…etc. It needs a little bit of time getting used to, but you can check their website for more detailed guides.

Below is the scenes setup I use, and here it is in an exported JSON format.

The Starting Soon scene

When it’s time for my stream to start, I don’t start the meaty content of the stream instantly. Instead, I display “starting soon” scene to give a chance for the people to join it, this only lasts for a couple of minutes not more.

In this scene, I only show a small video in a loop, without exposing my microphone audio. Some, however, make it really cool like @csharpfrits and @BaldBeardedBuilder displaying their shadows as they move to prepare for the stream.

The Me Talking scene

After displaying the Starting Soon scene for about two minutes, I bring the Me Talking scene up. It’s a focus on my face with a display of the chat overlay next to it. In the scene, I also display my twitter handle for people who don’t know me, they can instantly look me up on Twitter (I stole this from @davidwengier :D).

I use this scene to establish a connection with the viewers, without them being distracted by code. It feels more direct and clear. I usually do a recap here, I explain what we went through in the last session, and what is our plan for that day’s session.

In these scenes, I utilise my greens screen. My office is not the best office in the world, and having a green screen means I can put a soothing blue background. I haven’t gone crazy with my backgrounds but will experiment with this in the future.

The biggest trick in this one is to remember to switch to the Code scene; it happened twice when I jumped into the coding part while the scene displayed was still at my big face only, no code, *facepalm*! There are some plugins that allow automatic switch, but I haven’t checked them out.

I also display the chat-box using the Chat-box extension from Streamlabs. I display this on all my scenes (ops! except for the Secret scene below, I just remembered while typing this ). The reason why I display the chat is that Twitch doesn’t allow you to keep the videos as an archive, so I offload them to YouTube. Once the video is on YouTube, there will be no capture for the chat unless I record it part of the video.

I know if I stream on YouTube directly, the chat messages will be replayed alongside the recording of the stream, I think this is a beautiful feature. I might consider broadcasting on YouTube in the future, but I’m focusing on one platform for now.

The Coding scene

The coding scene is the one I show most of the stream. I have a part (on the left) where I show the code editor/desktop/web pages, a part I where show the chat messages (top right), and a part I show my camera (bottom right).

You will notice that I totally separate the chat and the camera from the code editor, unlike other streamers who show their cameras and chat on top of the code editor. My main reason for this setup is that in rare cases I have to show something at the right corner of the screen, and my camera or the chat will obstruct it otherwise.

The Secret scene

In this scene, I show a funny video in a loop for people juggling knives. I show the video, my camera, and my audio.

Sometimes I need to display secret tokens or passwords on my screen. I have three monitors but I stream one of them only, so I could just simply move my editor to the other screen. However, moving the windows around is a little bit tricky because they don’t fill my window (check the “The Main Monitor” section below for why). Thus, I decided to have this Secret scene, it’s also funny 😀

I have streamed before writing a Visual Studio Code extension that hides YAML nodes that are secrets, but it uses regular expressions, and it was never published. My next stream by god willing will be a new extension that is built better using proper YAML parser (did I just say “build it again but better”?! we developers never change!).

Audio

I use Soundflower to create the right audio setup. It is necessary to convert the audio output of your machine to be another audio source for the stream. Here is a video on YouTube I found useful on how to set it up.

I use my H2N microphone mentioned in part 1 as the main microphone. Music is too distracting to me during coding, so I don’t play music at all.

The Pilot’s View of the Setup

This is how is my view when I am streaming, I have two external monitors and the laptop’s screen:

The Main Monitor

This is where I write the code, and you can see that I have left some room on the right to fit the chat overlay so that it’s captured with the video, this will allow the chat to be a permanent part of the recording so that people who watch the video later on a different medium can relate to my comments on the chat.

It needed some time to get used to because I am usually a full-screen guy, but after awhile you just don’t see the void.

The OBS monitor

This is where I leave my OBS open, and where I click to switch scenes whenever I need to.

I also use it sometimes to read the chat from the chat overlay. This is not optimal as I often find myself squeezing my eye to read the small font. And because it’s away from the camera, it seems I “look away” from the audience to read what they are saying.

The only good thing about reading the messages from OBS itself is that I am reading from the same view source the audience read from. This way, if for some reason the overlay is not working I wouldn’t be reading something the audience is not seeing.

In my future streams, I will grab the link of the chat overlay and put it in a browser, and then squeeze that browser window next to my VSCode. This way I won’t look away from the audience to read their messages. (I tried this before when I was proof-reading this post, didn’t work because the chat overlay had a minimum width :(. Will update you when I find a better solution).

The laptop monitor

I use this screen as an auxiliary monitor if required, recently I start opening the native Twitch app to see how things are going on the other streaming end. (I have no idea why at that time Twitch App was complaining about the internet :D).

AppleScript:

I have been a Windows user for so man years of my life, I wish there is an equivalent for AppleScript in Windows (Maybe there is, let me know if there is). It automates many aspects of the MacOS: opening applications, resizing windows, changing settings, and a lot more.

I got this tip from @noopkat (like many other tips). Just before I start my streaming session I run my script and it opens OBS, opens a clean browser window, resizes my VSCode and place it in the right position…everything! You can have a look at it here

Visual Studio Code

After .NET Core, and MacBook becoming my main PC, my main IDE has become Visual Studio Code. I love its reliability, lightness, and extensibility.

There is a feature in VSCode called “ScreenCast Mode” (thanks @ShahidDev for the tip), it prints the keys pressed down on the keyboard. This is a very good way to share with the audience the love of shortcuts.

There are tools that work on the OS level, but I haven’t used it yet. I have my eye on Keycaster, haven’t tried it yet though.

Zsh as terminal

I can’t really speak a lot about this one, it’s a feature-rich terminal that is also extensible, however, I haven’t really utilised a lot of its features. I hope @tkoster is not disappointed as he is my guide in anything related to Unix and Linux!

Twitch channel setup

Truly, Twitch does NOT have the best UX for their platform, it’s confusing to say the least. However, they are constantly changing things around and try their best to improve.

The customisable page in your channel (at least at the time of writing these words) is the About page, you can add panels and custom content. Below are the panels I have.

About Me panel

This where the bio goes, I like to use the first singular on Twitch because it’s a direct conversation with the audience, it makes sense to say “hi, welcome to my channel” than “Emad is a developer…”.

After I had this panel for a while, Twitch decided to display the About Me info you are in your profile in the About page, this somehow made this panel is redundant. However, the About Me in the profile is limited to 300 characters, this one you can write more, hence I renamed mine to “More About Me”!

Stream Schedule panel

This is an extension from Streamlabs to show the visitors when your next stream is. Twitch has introduced their own Schedule page in the channel, so this one might be redundant.

I am not sure if this is really benefiting the audience though, at some point I was thinking of creating my own Google calendar for the streaming and let people subscribe to it so they know when I cancel one. I might still do it so stay tuned, and let me know if you would like that too.

Twitter feed panel

There is nothing like Twitter to tell people who you really are (not really, it’s only 140 characters of text!). This extenstion provides a list of your most recent tweets.

Chat Bot

Up until writing this post, I didn’t have a chat bot. But in the last stream one of the audience sent “!theme”, and I didn’t have a bot to answer, so I stopped whatever I was doing and started showing my theme. If I had a chat bot setup this wouldn’t have happened.

The next step for after posting this is to set up a new Bot. There are so many bots out there, this video shows several of them. I am already leaning towards nightbot, but I also know that @chahrpfritz has been working on one, so let’s see how it goes, probably will be another post so stay tuned or ping me if I don’t write about it ;).

Summary

As you can see, this is a space of a continuous change; you will find yourself keep changing setups, and tweaking settings here and there until you find the final setup, which soon will change after :D.

I hope this was beneficial, let me know if you need more information.

The post My Twitch Streaming Setup – Part 2 Software appeared first on Emad Alashi.

While writing this post, I realised that it’s going to be a long one, so I will break it down into three posts:

1. Hardware (this article)
2. Software
3. Humanware

Laptop

At the beginning of my journey, I streamed from a Surface Pro 4 (Intel Core i7-6650U 2.2 to 3.4 GHz and 16G RAM). This worked fine when I streamed while working on pure Azure tasks that didn’t involve any CPU consumption. But when I started doing local development and compiling code, my frames started dropping and my audience started complaining about the quality of my stream.

So when it was time for my Toolkit Allowance renewal (thanks Telstra Purple!), I decided to bump to the best machine I could afford. I read many confusing opinions on the internet about the role of GPU in a stream, and I couldn’t decide whether to get a good GPU machine or good CPU machine. Since I don’t buy a machine every day, and I had some space to bump the budget in addition to the allowance, I decided to get both, GPU AND CPU :).

Now I stream from a Macbook Pro (32G RAM 2.3 GHz 8-Core Intel Core i9, Radeon Pro Vega 20 4 GB).
The strongest voice I heard on the internet was that you want to concentrate on the CPU, but I will leave this homework for you. Needless to say, I don’t have a problem compiling, streaming, and recording at the same time now.

Microphone

Long before I got into streaming I started a podcast, and for some time I was looking for the right microphone, I needed a microphone that was of good audio quality AND has the functionality of recording in case I was on the go, and I found these in the Zoom H2N. I call this the awesome microphone, but I also believe that this is an overkill for most people who would like to start streaming or producing professional video content.

Microphone Mount

I got an unbranded microphone mount from eBay, you can see from the picture that I am hanging it to the bookshelf next to me, not to my desk; my desk is pretty thick and the base of the mount won’t fit on it. But as a nice outcome hanging it on the bookshelf, the noise coming from the keyboard is barely present.

I put the microphone’s gain to the highest; I stream in the night when the kids are asleep, and I am in a relatively quiet suburb. I position the microphone as close as possible to my face just before appearing in my camera. I am not 100% that the audience doesn’t get any pffff sound due to the high gain, but so far no one has complained :).

Camera

I have the Logitech C920, it’s pretty very common amongst streamers and for a very good reason. It’s very well balanced between prices and features, I love the angle it takes, the quality of the picture, and the auto-focus. Having said that, the only way I am using it to record my face; I don’t do close up reviews to products and I don’t need to move it off the top of my screen.

Keyboard

Late 2018 I bought one of the early models of Vortex Race 3 with Silver switch. This is absolutely not necessary for a successful stream, but mechanical keyboards are just luxurious nice feeling :D.

I kinda regret the Silver switch as I tend to make too many mistakes while touch typing. If time goes back I would get a Red switch instead.

USB Hub(s)

I would have loved to get a proper docking station, but the decent ones are expensive AND they don’t support my old VGA monitors, so I went with the hubs option instead.

Five Ports UGREEN USB-C hub

It’s funny that the hub is too old now that I couldn’t find it on their website to link it in this post, here is a picture for it.

This hub takes:

• One of the monitors
• an Ethernet cable (a must for streaming in my opinion)
• The microphone
• The mouse
• The camera

Generic USB-C VGA adapter

With extra USB-A peripheral to take my keyboard and the other monitor.

key-chain USB-A to USB-C adapter

I use this adapter to connect my USB Logitech headset. I don’t use this headset’s microphone, only the headphone. Having said that, I don’t really play any music or crazy sounds during the stream, so I don’t put the headset on my head most of the time.

Studio Setup

I was lucky enough to find a relatively cheap studio setup on eBay. In preparation for this post I searched for the item on eBay and it seems the price has gone up :).

The bundle had:

• Green, White, and Black backdrops. The material is very poor, I am not sure what it is called, but I tried to iron it once and it almost melted. It works 99% of the time, but I noticed recently that the wrinkles confused my chroma key, and I wouldn’t get a perfect removal. The good thing is that it’s not too noticeable, and most of the time during my stream the focus is on the code scene.
• Big stand to hold the green backdrop, it consists of two extensible mounts and a 4-pieces rod to sit across them. Then you put the green backdrop on it and tighten it with clippers came with the bundle (a little bit of a hassle really).
• Two mounts to hold the lights bulbs
• Two 135W 5500K light bulbs
• Two white umbrellas
• And some accessories I don’t use (two black umbrellas and reflectors)

If time goes back, I would have changed the tools a little:

• I would get the new shiny LED lighting that can be mounted to the desk behind the screens. There are many lighting setups like this but I think the newest one out there is the Elgato Key Light (Update: I have received mixed feedback about the quality of the Elgato Key Light, so this is NOT a recommendation. Please do your homework and assess before buying). The lightning itself is not a problem, but setting the lightning every time is just too tedious.
• I would get an easy-setup green screen, also Elgato has a collapsible green screen that can be easily setup/taken off. For the same reason, setting this thing up and tearing it down just takes an uncomfortable time. In addition to the wrinkles problem above.

Surface Pro 4 (Not necessary)

Sometimes during my stream, I’d like to explain something on a whiteboard, and using the mouse for that isn’t really natural. So I thought I can use my old Surface Pro 4 laptop. I tried at the beginning to use NDI to stream from two laptops, but it just wouldn’t work.

So instead I used the Microsoft Whiteboard app: I use my SP4 to draw on the whiteboard, and then connect to the same whiteboard from my Mac. There can be a small delay between drawing and appearing on the screen, but it wasn’t that much. However, this setup is a little tedious and I am thinking of alternatives.

Summary

So this is my Twitch streaming setup. It’s worth mentioning that I didn’t get all of this setup at once, I accumulated it over time. I had the microphone first, then the camera, then the green screen and lightning…etc, and this was over many months.

You can also slice the budget even more if you choose lower end microphone, and a normal keyboard.

I hope this was beneficial, if you have any questions please let me know in the comments, or ping me on Twitter at @emadashi, would love to hear from you. Stay tuned for the two coming sections: Software and Humanware.

The post My Twitch Streaming Setup – Part 1 Hardware appeared first on Emad Alashi.

minikube profile list, invalid profile

Last Saturday, I had the privilege to speak at GIB Melbourne online, where I presented about Self-hosted Azure API Management Gateway. In the presentation, I needed to demonstrate using Minikube, and I spent couple of days preparing my cluster and making sure everything is good and ready.

One day before the presentation, Minikube suggested to me to upgrade to the latest version, and I thought: “what is the worst thing that can happen”, but then then responsible part of my brain begged me not to fall into this trap, and I stopped. Thank god I did!

After the presentation I decided to upgrade, so I upgraded to version 1.8.1 (I can’t remember the version I had before) but then none of my clusters worked!

When I try to list them using the command “minikube profile list” I find it listed under the invalid profiles

Oh this is not good! Was this update a breaking change that hinders my clusters unusable? Or is it that the new Minikube version doesn’t understand the old Profile configuration? And is the only way I am supposed to solve the problem is by deleting my clusters?! I am not happy.

Can I fix the configs?

Before I worry about breaking changes, let me check what a valid profile looks like in the new update, so I created a new cluster and compared the two profiles.  You can find a cluster’s profile in .minikube/profiles/[ProfileName]/config.json.

The following are the differences that I have noticed:

• There is no “MachineConfig” node in the configuration, and that most of its properties are taken one level higher in the JSON path.
• The “VMDriver” changed to “Driver”.
• The “ContainerRuntime” property is removed.
• There are about 4 properties introduced
  • HyperUseExternalSwitch
  • HypervExternalAdapter
  • HostOnlyNicType
  • NatNicType
• The “Nodes” collection is added, where each JSON node represents a Kubernetes cluster node. Each node has the following properties:
  • Name
  • IP
  • Port
  • KubernetesVersion
  • ControlPlane
  • Worker
• In the KubernetesConfig, the Node properties are moved to the newly created collection “Nodes” mentioned above:
  • “NodeIP” moved to “IP”
  • “NodePort” moved to “Port”
  • NodeName moved to Name
  • A new property ClusterName is added

The Solution

So what I did is that I changed the old profile format to match the new format, and set the new and different properties to the values that made most sense just like above. All was straight forward except for the Node IP address; It’s missing!

Digging a little deeper I found the IP address value (and other properties)  in the machine configuration “.minikube/machines/[clustername]/config.json”. I copied these values from there and then ran my cluster to be resurrected from the dead!

I would have loved if Minikube itself took care of fixing the configs rather than suggesting to delete the profiles. Or maybe that can be a Pull Request :).

I hope this helps.

The post How to Fix Minikube Invalid Profile appeared first on Emad Alashi.

After 4 years of using Windows, I am coming back to using a Mac. And there are so many things in Windows I am missing already. One of these things is posh-git; I loved how in one glance to your prompt you know the status of your git repo: how many files changed, how many added, how many deleted, how many indexed… just love it!

Once I moved to Mac, I changed my shell to use zsh using Oh My Zsh due to the rich experience it brings to the terminal. I was delighted to see all these themes and plugins, and then started looking for a theme that provided the same information posh-git prompt provided. To my surprise, there was none! So I started my quest to see how I can change zsh, the theme, or the plugin to have such prompt.

Being lazy, I wanted change an existing theme I like with the least amount of investment. I looked in the documentation to see how I could do that, and found the customisation wiki page:

Should I override the theme?

Overriding the theme seemed to be the perfect solution, however, there were couple of drawbacks:

• When you override a theme, you override the theme, period! This means that if the author changes something after you have overridden it, you will not get these new changes.
• It was a little bit too much for me to grasp! When I looked at avit theme as an example, I had questions like what is PROMPT and PROMPT2? What are all these special characters? Where is the reference/documentation to all of these? Are they theme-specific, or are they part of zsh theme reference?

Remember I wanted to put the least amount of effort, and I surely didn’t want to learn the whole thing! But while looking into avit theme, one thing grasped my attention: there was a clear reference to what seemed to be like a function git_prompt_info. And I thought this should be it, if I could find where this function is and how to override it.

To my luck, it was mentioned as an example in the customisation wiki page as an example!

Override the internals it is!

Ok great, now I know that I can customise git_prompt_info, all what I need is to mimic whatever posh-git does in that function!

So I hit google duckduckgo again on the hope that someone already did this, and oh my! I found that there is already a port of it on bash. That’s great, now what should I do? Replace the call of prompt_git_info in the theme with a call to __posh_git_ps1? Or should I call it from prompt_git_info? Since prompt_git_info is an internal lib function, it is probably used in many themes, thus it will make sense to just call __posh_git_pst from within. And to my good surprise, there is a GitHub issue in the posh-git-bash repo that discusses integrating with zsh, it’s even referenced in the main README.md file of the repo.

Initially I mistakenly called the __posh_git_ps1 function, but I soon realised that I need to print (echo) the git info just like prompt_git_info did rather than changing any variables, for that I should use the __posh_git_echo.

And thus I ended up with a file called emad-git-prompt.zsh under the path ~/.oh-my-zsh/custom with the content of posh-git-bash here, and at the end of the file I wrote the following code:

git_prompt_info () {

__posh_git_echo

}

I hope this helps you

The post Posh-git on Mac using Oh My Zsh Themes appeared first on Emad Alashi.

This post explains why and how I learned the Go language. Hopefully this will help you to learn it quickly, or will inspire you on how to learn new languages.

The Reason to Learn a New Language

There can be many reasons why someone would want to learn a new language, the main ones to me are: 1) To solve a current business problem 2) Learn concepts to adapt to current tools 3) For fun and passion. Of course, you can have a mix of these reasons to push you to learn a new language, or maybe just one strong enough of these reasons.

For a very long time in my career, C# was my main programming language, I used JavaScript a lot too, but it has always taken a back seat until TypeScript came about, and SPA became the de facto front-end development model. So for 16 years, it has been two languages and a half for me, and I have never felt the need to learn another language (Java in university doesn’t count).

Why not Haskell or F#?

When functional programming became a thing again, I tried to find the right reason to learn F# (or Haskell), but with the explosion of technical information in our industry, time became even more scarce (I have three kids under 5!) and I really needed a stronger reason to spend my time learning a new language. Unfortunately, even with @DanielChambers continuous efforts in converting me :P, I didn’t jump to the wagon.

It’s funny that the reason why I couldn’t put the effort was exactly the reason why functional programming itself is compelling; it’s the paradigm shift. The paradigm shift was so big that organisations in the I spend most of my time helping couldn’t afford to embrace it; 20+ years of OOP meant a lot of investment in education, solutions and patterns, frameworks, and staffing that made it hard to embrace such a change.

In my experience with these organisations, there might have been situations where functional languages could have solved a problem better than an OOP one, but the return of investment would have been little in the light of the legacy of these organisation.
Of course, I am not promoting that organisations should not invest in learning and adopting new technologies; that would be the path to failure! But I’m just describing the situation of most of the organisation I worked with.

This ruled out the business-need reason for me, and I am left with “learning concepts to adapt to current tools” since passion was not just enough :P. Luckily, I am surrounded by friends who are passionate about functional programming, and I managed to learn from them enough about its benefits and how to bring that to my OOP world. Conversations with these friends and colleagues like Daniel Chambers, Thomas Koster, and attending lectures by professionals like Joe Bahari, have helped me a lot in adopting functional concepts to my C#.

I Found The Reasons in Go

So I stayed on two languages and a half, until last year when I got the chance to work on a project in which we used Kubernetes. Once you step in the Kubernetes world you will realise that Go is the hero language; Kubernetes is written in Go, Helm is written in Go, and the templates Helm uses is based on the Go template engine. Although you can use Kubernetes without learning the Go language, once you want to get a little deeper it feels that learning Go would be an advantage.

In addition to that, with Cloud being my main interest, I have been seeing Go used more and more as the language of choice for many of the cloud-native project, products and services.

During the same time, many of my colleagues and Twitter friends have been porting their blogs from database-driven engines like WordPress to static website generators like Jekyll. I have two websites that could benefit from that, 1) my blog emadashi.com 2) and dotnetarabi.com podcast, which I built on ASP.NET and Subsonic ORM of Rob Conery’s. My friend Yaser Mehraban kept teasing me and applying his peer pressure until I surrendered, and I finally started looking into it moving my blog and my podcast to a static website generator.

My choice was Hugo; to me, it seemed the most mature static site generator with the least amount of churn and learning curve. And guess what, Hugo is written in Go! And the templating engine is based on Go’s. Same as Kubernetes, you don’t need to learn Go if you want to use Hugo, but it’s just another compelling reason to be familiar with the language.

So by now, it feels I am surrounded by problems that are being solved with Go, and it’s more evident that there is a greater possibility for me to work in Go in the future, even professionally.

All this, in addition to the low barrier of entry due to familiarity with C#, encouraged me to jump to the waters.

Where did I Start?

There are so many ways a person can start learning a language, to me I wanted to learn the language fast and learn just enough to get me going. For this reason, I didn’t pick up a book that would take me a while to learn, even though a book is probably the most profound way.

Instead of picking up a book, I went to https://golang.org and checked what the website has to offer; most of the modern projects and languages have documentation that includes tutorials and Getting Started guide. If these guides are well crafted it would be a great learning boost, and to my luck Go had great content.

Set-up

The first thing I wanted to do is to set up the environment and run the most basic example (the hello world of Go), for that I followed the Getting Started guide. Setting up the environment as a basic step for learning a language is very important; it will give you an understanding of the requirements of the language and will set up some expectation on how friendly the experience is to you, it breaks the ice. Also, it paves the way to the Hands-On step coming soon; I will explain this step later in this article.

Foundation

Now that my environment is setup and I ran my hello world example, I needed to understand what is really going on: how the code compiles, how it runs, how it is packaged, how it is hosted; I needed the foundational concepts to establish a firm ground to base my learning on. Learning the syntax and the various Go features will come along, and it will take time, but you can’t postpone the foundations. For this, I followed the “How to Write Go Code” guide. The article’s title might not sound too foundational, but the content lays the concepts.

Cruise as you need

If this is NOT your first programming language to learn, then you are already familiar with the concepts of structure flow: functions, loops, if clauses,…etc. This gives you a very good advantage to sweep through these swiftly; it’s unlikely that these are too different from other languages. A fast run through should be enough to capture anything standing out.

For this I used the Tour; there are two great things about the tour: 1) it has a simple navigatable structure 2) it is associated with an online playground where you can experiment and confirm your understanding on the spot. There is a wide range of topics covered in the Tour, some of which I would go through fast, and some I would take my time to comprehend; e.g. Slices can be little confusing compared to arrays in C#.

Note: Everyone’s experience is different, so it will not make sense to list the topics I went through swiftly and the ones I spent time on, use your own experience to judge that for yourself.

As for the advanced topics I left out a little until I had a better grasp on the basics of the language; overwhelming yourself with advanced topics at this stage might have a counter effect on your learning.

Hands-On

After understanding the basics from the How to Write Go code, and sweeping through the Tour, it was time to have my hands on the language; this is the only way you can really understand and learn a language.

I needed a problem to solve so I can have a driving purpose. The problem I chose is to import the existing records of DotNetArabi from the database (guests and episodes) to create corresponding Markdown files for the Hugo website, so this was my first program.

It’s important to understand here that I wasn’t 100% on top of things yet (neither now :P), but it was the practical experience that I relied on to grasp the concepts and gain the experience. If you leave the practical side for too long you will find yourself forgetting the basics, or that you are learning too advanced topics that you will rarely use. An iterative approach is very good here.

So I gradually built the application; each time I am stuck I’d either refer back to the Tour, or google it if it is not covered there (e.g. connecting to a database). In each of these stuck-and-solved situations, I take a moment to make sure I understand the solution and the technique behind it. Copy and paste is absolutely fine as long as you pause and comprehend.

Advanced Topics

Ok now at this stage I feel like I know the basics, and I am comfortable writing a program without big issues. But at this stage, writing a program in Go would give me very little advantage (if any) over writing it in another language; I am not getting the best out of the language. It’s the advanced features that make the difference, things like goroutines and channels by which we achieve concurrency with the least amount of maintenance overhead.

Don’t be afraid of the advanced topics; avoiding it the advanced topic because they might be complicated will jeopardise the value we are getting from learning a language in the first place!

So for this, I continued the Tour above for the advanced topics. The playground was of tremendous value as you will need to change things around to confirm your understanding. Also, the Tour has some exercises that will poke your thoughts, I highly advise trying these out! This will not just push you to comprehend the concepts, but it will also expand your horizons for the use cases that you might need these advanced features.

It would be great fun and value if you can go back to your pet project and try to implement some of these advanced concepts, and this is what I did. I went back to my application and utilised goroutines to extract the data to the markdown files.

Unit Testing

Leaving unit tests to the end wasn’t undermining their value, rather I wanted to focus on the language itself first; test frameworks and push the complexity and the learning curve high enough. My experience from JavaScript stings until now :P.

The Best of Go

Finally, Go website has a section called “Effective Go“. This section is not really a referential documentation, but it can be very valuable so that you write the Go code as the language has intended it to be like. It provides further context and rounded styling to writing the language in the best form.

I also here advise to pick and choose the topics, reading the whole thing might be counter-productive.

Close the Loop, Complete the Picture

By now you’d think you finished, but this is just the beginning; now is the time to tie things together by revising the language’s main characteristics, philosophy, and the greatest advantages.

If we look specifically at Go, as our example, this might be things like the simplicity of Go, where there no classes, no inheritance, or generics. Or things like concurrency and how Go deals with State in asynchronous code execution. At this stage, it will be valuable to check the videos, like Sameer Ajmani’s talk, and the literature out there that discuss “Why Go“.

I also found the FAQ in golang.org a valuable resource for some of the justifications and explanations. You should not read this as an article though, pick and choose the topics of interest.

But isn’t this backward? Shouldn’t I learn about these things at the beginning? True, you can learn these at the beginning, but you will not value the claims until you try and put your hands on the problem in practice, until then it will be merely claims in the air. So even if you start with these, you should also revise them and make sure you tie the loop.

Conclusion

In my journey to learn Go, I did the following:
• I had a good reason
• I established the core concepts
• I installed the tools and ran the “hello world” program
• I scanned through the structure flow
• I put my hands on the code and wrote the first program
• Read the advanced topics, and used the playground to confirm my understanding
• Watched more videos on why to use Go and its advantages

It’s important to say here that choosing a language to adopt for in an organisation involves more than just learning it. If you are in a position to influence a decision just be mindful of that.

I hopes this helps you out, enjoying coding :).

The post Learning a New Programming Language (Go language as an Example) appeared first on Emad Alashi.

For a long while, I’ve been thinking about streaming live development to Twitch or YouTube. Having spent some time behind the microphone while making DotNetArabi podcast, I can say there is a satisfiying feeling in producing content in a media format through which you can connect with the audience.

Why not just offline video?

I could just record an offline video and host it on YouTube, and it’s definitely a valuable medium. The problem with educational videos, specifically, is that it is a one-way communication channel, and without the entertainment factor, unlike movies, these videos can be daunting, imprisoning, and hard to follow.

The magic of live streaming

But with live streaming magic happens; it adds additional dimensions that make it more appealing:

1. It’s LIVE! It’s happening NOW, and this means couple of things: it implicitly has the anticipation factor; things are still happening and it might take interesting turns, just like live sports. In addition to that, by sharing the time span during which the event is happening, the audience gets the feeling of involvement and “I was there when it happened”, even if the audience didn’t directly interact with the broadcaster.
2. It’s real and revealing: When I was doing my homework preparing for this, I talked to my colleague Thomas Koster, and when I asked him about what could interest him in live streaming, his answer was:
   

   …it’s probably more the real time nature of it that appeals – to see somebody’s thought processes in action, as long as the broadcaster doesn’t waste too much time going around in circles.

   For example, watching somebody figure out a puzzle solution in the game The Witness in real time is much more interesting and valuable than watching a rehearsed, prepared performance of only the final solution.

   This is the ultimate stage for a developer broadcaster; it requires a lot of bravery and experience. I’d love to be able to do this soon, but it’s really the 3rd reason below that drew me to streaming.

3. It’s two-way communication: the interactive communication between the broadcaster and the audience brings the video to life. It provides timely opportunity to get the best out of this communication, whether it was by the audience correcting the broadcaster, or the broadcaster being available for immediate inquiries.

Specifically for this last reason, I became interested in live streaming; I want this relation with my audience; to have a collaborative experience where value is coming from everyone and going in all directions.

So, I am doing my first stream!

I have been following Jeff Fritz @csharpfritz and Suz Hinton @noopkat and greatly inspired by their amazing work! Also @geoffreyhuntley have started his journey and gave me the last nudge to jump into this space. I’ve learned a lot from Suz’s post “Lessons from my first year of live coding on Twitch“, and recently Jeff’s “Live Streaming Setup – 2019 Edition” (don’t let it scare you,  you don’t have to do it all!).

My next stream will be about Role Based Access Control (RBAC) in Azure Kubernetes AKS, I will walk you through RBAC, OAuth2 Device Flow, and how this works within Azure AKS, with hands-on live deployments and configuration.

What is my goal, and what is not?

What I am trying to achieve here is two-way communication through the session I have with my audience, that’s it.

Am I going to do this constantly now?

Actually, I don’t know! To me this is an experiment; I might keep doing it, or this might be my first AND LAST stream, let’s see what the future brings.

The post RBAC in Azure Kubernetes Service AKS on Twitch! appeared first on Emad Alashi.