Similar to the situation in Part 2 for onboarding a project into SonarCloud, onboarding a user into an organization is also not supported via the public API.

In the API documentation there is no entry for api/organizations

This was confirmed by a community posting and unfortunately did not appear on the roadmap:

Back to Chrome Developer Tools

So in the same way as in Part 2 I used Chrome Developer Tools to monitor what happens in the SonarCloud UI when adding a member to an organization:

We can see that a URL of /api/organizations/add_member is used:

The payload contained two items:

• login – the GitLab user id
• organization – the SonarCloud organization

Consequently the Add-SonarCloudOrganizationMember function was born and we can now fully onboard a user to an organization with the following:

The user is successfully onboarded to the organisation:

• Automating SonarCloud with PowerShell – Part 1 Introduction
• Automating SonarCloud with PowerShell – Part 2 Onboarding a Project
• Automating SonarCloud with PowerShell – Part 3 Onboarding a User

Warning: this post contains unsupported API calls, so use at your own risk

One of the main reasons I started looking at the SonarCloud api was to be able to automate the onboarding of new GitLab projects into SonarCloud. We didn’t want to have to do that manually via the UI, rather via a pipeline job. In part 1 of this series I demonstrated the basics of the API and the PSSonarCloud PowerShell module, in part 2 we’ll look at how to onboard a project into SonarCloud.

False Start

Initially this looked like it would be a very straightforward job, the SonarCloud Web Api has documentation for creating a project:

So as part of the PSSonarCloud module I added a function “`New-SonarCloudProject . You could use it as the following to create a project, all you need is a SonarCloud API key with permission to create a project, then the SonarCloud organisation, GitLab project name and project id:

The project will appear in SonarCloud and all looks good:

However, an issue will arise when you attempt to run some analysis against the project, it will fail and complain that the SonarCloud project is not integrated with your repo.

I discovered via a community forum post and an observation in the UI that the API call above does not include the integration with the code repo – compare that project with a project which was onboarded via the UI and it displays integration with GitLab:

A New Hope

The forum post suggested an unsupported option would be to examine the API call through browser development tools and use that:

So off to Google Chrome development tools we go……

By looking at the API call used when adding a project via the UI, we can see that a URL of /api/alm_integration/provision_projects is used

The payload contained two items:

• installationKeys – the GitLab project id
• organization – the SonarCloud organization

Consequently the New-SonarCloudProjectALMIntegrated function was born. So we can now fully onboard a project with the following:

The resultant project has been fully onboarded:

• Automating SonarCloud with PowerShell – Part 1 Introduction
• Automating SonarCloud with PowerShell – Part 2 Onboarding a Project
• Automating SonarCloud with PowerShell – Part 3 Onboarding a User

SonarCloud has a Web API which facilitated a lot of what I needed to achieve, however with some significant gaps which initially appeared to mean some of the onboarding would need to still be manual. Coming up in Part 2 and later of this tale will feature how I reverse engineered the Web API to fill in the missing parts of the puzzle, but in Part 1 I’ll focus on an introduction of the basics.

Pre-Requisites

The automation was carried out with PowerShell 7.x+. An API key for the SonarCloud Web API is required and can be generated from the security page of your SonarCloud account:

Initially I thought I would only need a few API calls, however it turned into many, so I ended up creating a PowerShell module, PSSonarCloud, to facilitate easy re-use of similar calls.

Getting Started

Install PSSonarCloud from the PowerShell Gallery:

Supply your SonarCloud API key to create a connection variable:

Retrieve a list of projects belonging to your SonarCloud organization:

Retrieve a list of users belonging to your SonarCloud organization:

Going Further

The module currently contains 15 functions for working with the API:

This only covers a relatively small part of the API, however as per most of my other PowerShell modules the functions are wrappers around a single function Invoke-SonarCloudRestMethod which means should you need to work with an API call that is not currently covered by PSSonarCloud then it should be straightforward.

For example the following API call to list the branches of a project is not part of PSSonarCloud yet:

The following code would get you there though:

• Automating SonarCloud with PowerShell – Part 1 Introduction
• Automating SonarCloud with PowerShell – Part 2 Onboarding a Project
• Automating SonarCloud with PowerShell – Part 3 Onboarding a User

to existing repos on GitHub.com using an existing Personal Access Token or authenticating to a new repo using a different account on an instance of GitHub Enterprise using a new Personal Access Token, I would receive a combination of the following errors:

repository not found

login failed use ctrl+c to cancel basic credential prompt

Turns out the GitHub recently deprecated basic authentication using a username and password in favour of a personal access token. I couldn’t see why this was causing me an issue since I was trying to use a personal access token.

This StackOverflow post cleared things up. A later version of git for Windows is required in order to use the new method. Initially I had v2.27.0.windows.1 so upgraded to the latest as of writing this post v2.30.0.windows.2. There’s a clue to some changes during the upgrade process that a new Git Credential Manager will be used:

After upgrade there is a different dialogue window when authenticating:

Using the Personal Access Tokens were now successful for me across all repos in GitHub.com and Enterprise.

As a side note, if your GitHub Enterprise uses SSO for authentication then you will need to Enable SSO for you Personal Access Token

You may then be prompted further when accessing a repo for the first time to navigate to a URL to authorise it for that repo via a browser.

A variation on this question is around Lego sets which contain multiple instruction booklets, since Get-BricksetSetInstructions will return a link to each of the PDFs for that set.

There are some examples in a previous article which may cover what you are looking for. However, in this article I’m providing a better example for the more complex scenario where there is a need for:

• Support for Lego sets with multiple PDFs
• Naming the downloaded files to match the Lego set number since the file name on the Lego site gives no indication which set it is for

This example will return all Lego sets with the Theme ‘Indiana Jones’ and download instruction PDFs for each one, including those with multiple booklets. You can though use the output from any query with Get-BricksetSet, depending on which sets you need the instructions for.

Note: it’s using Start-BitsTransfer which is Windows only. Replace with the file download command of choice if you’re on a different OS.

I’d had a quick look at the v3 API not long after it had been made available, and as far as I remember it was still only SOAP based. This had put me off updating the PowerShell module, since I had wanted to also support PowerShell core and there is still no SOAP support in PowerShell core, i.e. no New-WebServiceProxy available – see here and here. The previous version of the module was built entirely around using New-WebServiceProxy .

However, when I looked again in December 2020 following more external prompting I noticed on the API documentation page that there was support for REST as well as SOAP:

This meant that it was possible to use Invoke-RestMethod instead of New-WebServiceProxy and after some basic experimenting I went with the second option from the above screenshot to use HTTP POST and Content-Type set to application/x-www-form-urlencoded – which by the way I discovered as part of this exercise is the default when specifying the POST method with Invoke-RestMethod and no Content-Type set:

So the Brickset Module now contains a wrapper for Invoke-RestMethod, Invoke-BricksetRestMethod, which makes it easy to work with the API. For example to getThemes :

The rest of the functions are built using a similar pattern, some more complex where a more complicated body section is required to be sent.

There is also no more $Global:bricksetConnection to contain the API key and UserHash we need to make REST calls, it’s now available to the module via $Script:bricksetConnection. A security improvement I learnt about from an issue raised during PowervRA development – using script scope in a module is effectively module scope.

In addition I took the opportunity to re-use and update the Azure Pipelines CI/CD work Craig had done for us in PowervRA and apply it to this module which brings the following benefits:

• Builds the Module psd1 and psm1 files from the src folder
• Carries out PowerShell Script Analyzer tests
• Publishes the module to the PowerShell Gallery
• Creates a GitHub release

This means I can carry out any future updates to this module quicker and easier. Publishing a new version is as simple as creating a tag in the repo and the pipeline does everything else.

Version 2.0.1 of the Brickset module is now available from GitHub or the PowerShell Gallery via:

I used a combination of the online certification guide at Hashicorp and a few courses available from Pluralsight by Ned Bellavance:

• Terraform – Getting Started
• Terraform Deep Dive
• Implementing Terraform on Microsoft Azure

The guide at Hashicorp was excellent for covering the basics with a particular focus on the exam topics. I followed along using test AWS and Azure subscriptions I have access to in order to get more hands-on use. In addition, I found all three Pluralsight courses excellent not just in terms of preparing for the exam, but also had a wider scope and helped better prepare for some real world experience I needed for a work project.

Finally, I used some practice questions from this course on Udemy which gave a good impression of the type and level of difficulty of questions to expect on the exam.

The exam itself was online given the current issues with accessing physical test centres. Based off my previous experience with taking a Microsoft exam online and a good experience with this Hashicorp exam, I think any future exams I take will be online rather than the effort needed to get to a test centre, which for me is typically a 2 hour drive away.

Happy to report that the exam was a success

Consider two basic JSON files to use for an example:

It’s easy to start working with them as objects in PowerShell, but I always forget that you can’t then just concatenate them together since they are PSCustomObjects:

The solution is to use an array instead. Today I found a neat solution in this StackOverflow post which uses the Array subexpression operator, rather than more long winded efforts I’ve used before:

The resulting combinedfruit.json file:

I mostly used the AZ-400 course on Linux Academy by Tim Lawless to study for it and found it an excellent resource. I thought Tim did a great job of not just covering the topics as per the published curriculum, but also went further into illustrating many of the third-party integrations with Azure DevOps. To be fair it made a lot of sense to do this, since at a minimum at least an awareness of these integrations for common software build and deploy tools such as Jenkins, SonarQube, WhiteSource, Maven, Gradle etc is covered in many places in the exam.

In addition around 30 – 40% of the exam is based on Agile / DevOps theory, practice and general experience with building software in different programming languages, i.e. not technical questions around products available within the Azure DevOps suite. If that is not part of your day job or something you are unfamiliar with, then I would suggest the following books, for the Agile / DevOps part anyway:

– The Phoenix Project – https://itrevolution.com/book/the-phoenix-project/
– Google Site Reliability Engineering – https://landing.google.com/sre/books/

Since all Pearson VUE test centres are currently closed and my original on-site exam booking for April was cancelled, I switched over and took it via the online exam format. I was pleasantly surprised by the experience and would definitely consider it again over a test centre. After going through some initial security checks, ID and pictures of room you are taking the test in, it was nice to not be in a room with others testing at the same time and the inevitable distractions of noise and movement. I was then monitored by a proctor via my laptop’s webcam. My online real concerns were preventing one of my currently at home family members walking into the room and invalidating the exam or perhaps troubles with home broadband. Thankfully with some preparation and good fortune I experienced no issues. Full details about taking Microsoft exams online can be found here.

The predicted lab portion for this exam was not present, so only case studies and standard multiple choice questions. I haven’t seen anything official, but a few comments on blogs suggest this has been removed temporarily from all MS exams for performance reasons due to the greater demand for online rather than on-site exams at the moment.

I hope this helps you if you are headed down this certification path.

Today, we have released a new version of PowervRA, 4.0.0 with support for vRA Cloud and vRA on-premises version 8.0. The observant among you will notice that we’ve incremented the PowervRA version from 3.7.0 to 4.0.0. We’ve done this since there is a significant breaking change.

The API for vRA Cloud / 8.0 is completely different to that which exists for vRA 7.x and earlier. If you’ve been with the product for a while you’ll be aware of the pain challenge it has been to work with since inception due to the split brain nature of the product, more info here and here. Thankfully that has gone away with the latest release, however it left us as maintainers of PowervRA to decide whether to maintain support for both sets of APIs in the same PowervRA version.

Breaking Change

Ultimately, we took the decision to only support the API in vRA Cloud and 8.x going forward, consequently PowervRA 4.0.0 does not support vRA 7.x and earlier.

If you wish to continue to use PowervRA with vRA 7.x and earlier it is possible by using PowervRA 3.x, currently 3.7.0. You are able to install specific versions of PowerShell modules from the PowerShell Gallery by using the RequiredVersion parameter. So for example to install PowervRA 3.7.0:

We will continue to maintain a version of PowervRA that supports vRA 7.x, however to set a fair level of expectation please remember that this is a community based project with limited resources, both in terms of developers and lab environments and our focus going forward will be on PowervRA version 4.x and later.

We’ve done our best to communicate this in advance of release via Twitter and also the VMware Code Slack channel.

If you are aware of anyone likely to be impacted by this change, then we would be very appreciative if you would communicate it to them.

Context

Back when Craig and I first developed PowervRA for release in March 2016, it was primarily formed from our own needs since we were working on vRA projects where a lot of automation was required. Since that time, and particularly in the last couple of years, the career focus for both of us has moved away from vRA and our ability to significantly develop PowervRA and deal with logged issues has diminished somewhat. At the same time interest in PowervRA has increased, over 26k downloads currently on the PowerShell Gallery!

Acknowledging that, we recently sought out some extra assistance and that has been successful in helping us get this new version out and potentially can help support the previous version in the future. Also, by chance I happened to spend some time on a work project recently based around vRA Cloud which has meant I’ve had some hands on experience and been able to make some contributions.

TBH I think there is enough work for an individual to keep that person occupied full time on this project, for both PowervRA versions, for at least a number of months, but realistically that is not going to happen. We do of course welcome Pull Requests for either version and now have more resource on-board to have a better chance to turn them round into new releases.

With that said, here’s a few tips on getting started with the new version! Note these are all around vRA Cloud since that is what I’ve had access to. I don’t believe there should be that much difference to the on-premises version, other than around how to make the initial connection.

Getting Started

Install the latest version of PowervRA:

Observe the available functions, currently 36:

Getting Connected

In order to authenticate with vRA Cloud an API token is required (for vRA on-premises it is different, check the help for Connect-vRAServer for details). To generate one, navigate into the management portal and your account profile. On the API token page, generate a new token with the appropriate access requirements:

Save the token somewhere safe so that is is available for use with PowervRA.

Using the API token, make a connection to vRA Cloud:

Function Testing

The Get-* functions are typically a good way to get a feel for what is available. For example to get a list of all projects in vRA Cloud:

API Coverage

You’ll notice above that in the initial 4.0.0 release there are 36 functions, so there is obviously a lot of the API not currently covered, particularly in comparison to the previous release which has over 100 functions for vRA 7.x. We hope to build up this coverage over time, in the meantime remeber that Invoke-vRARestMethod is your friend.

We build all of the PowervRA functions around this helper function, so if there is a part of the API that is not covered by an existing function, you can use Invoke-vRARestMethod to fill in the gaps in the meantime.

For example we currently have no function coverage around Blueprints. However, by looking at the vRA API documentation there are examples for working with Blueprints. So the following code using Invoke-vRARestMethod would get you details about Blueprints:

Finally

Thanks for your interest in PowervRA. We hope you find the new version useful and we look forward to taking it further. Providing feedback via issues and code contributions via pull requests would be a great way to help us do that.