Pio@Work

Apache Superset Error 500 on first login

2025-03-02T22:04:00.006+00:00

It appears the docker image for Superset comes in an condition that still needs some tweaking.

Configuration inside the running container can be done with either docker exec, podman exec or an interactive console if a:ailable through the container management tools.

The first issue is a lack of an admin account:

superset superset fab create-admin

Then the database needs upgrading

superset superset db upgrade

Finally superset itself needs to configure some tables in the db:

superset superset init

Now logging into the management UI on port 9000/9443 should work

Running Apache Superset image on Rocky (Redhat) Linux 9

2025-03-02T17:10:00.002+00:00

Running containers on Rocky Linux 9 is pretty trivial via the Cockpit webui (available on <server ip>:9090)

The Superset container from docker.io is available for download using the container management page in Cockpit. Under the Podman menu, click the three dots and search for superset. The first entry should be the Apache Superset entry:

By default, when the container starts it looks for an environment variable called SUPERSET_SECRET_KEY which can be configured either inside the container configs or passed through as an environment variable, which is the more convenient method used here.

First generate a key with openssl

openssl rand -base64 42

Then simply add the environment variable and generated key under the integration section when create the container. A port mapping for 8088 is also required to access to the web gui inside the container

Proxmox web gui not responding after first installation

2025-02-28T21:53:00.001+00:00

A non responsive vanilla straight-off-the-iso deployment can be caused by certificate issues if part of the key pair is missing (possibly an installer bug?).

To confirm, log into the virtual machine console with the root user and the password that was entered during the setup process and check service status:

systemctl status pveproxy.service

Also check certificate information. If no certificate is present, the command will exit without any output.

pvenode cert info

You can try updating the certificates but this will probably fail.

pvecm updatecerts --force

There are a couple of locations that the relevant key files might be located:

/etc/pve/pve-root-ca.pem
/etc/pve/priv/pve-root-ca.key

Move files to a backup location:

mkdir ./backup
mv /etc/pve/pve-root-ca.pem ./backup
mv /etc/pve/priv/pve-root-ca.key ./backup

Updating certificates should regenerate a new public/private key pair

pvecm updatecerts --force

If successful, checking certificate information again with pvenode cert info should now show full details of the certificate and the web gui should now be responding, albeit with the usual warning around self signed certificates.

GRUB menu not displayed?

2024-04-26T20:58:00.002+01:00

GRUB is usually configured to show a menu for a few seconds on boot, but some distributions will set it to skip the menu.

The answer is to press a hotkey to enter the menu.

If BIOS is booting the system, press SHIFT

If UEFI is booting the system, press ESCAPE

When booking a VM, you get precious little time before it boots the system, so just hold the key down during POST!

Power Query authentication issues refreshing queries

2022-05-10T10:01:00.002+01:00

Power Query is an extremely handy tool, especially now that it's baked into Excel, but it has an annoying tendency to not show system dialog boxes when it can't authenticate to datasources.

A common error is "Exception of type 'Microsoft Mashup Engine.Interface.ResourceAccessForbiddenException' was thrown" although there are probably many more.

In order to do a turn-it-off-and-back-on-again reset of the stored (and failing) authentication credentials:

1. Go to Data -> Queries & Connections and right click on a query and edit it. T|his should bring up the query editor.

2. Then click on data source settings and Global Permissions. Select all the data sources listed and click edit permissions

3. Under credentials, click edit and select a different type of authentication and click save. Then go back in and re-select the right one, signing or entering username/password as appropriate.

4. Then select the appropriate Privacy Level. If it's not a confidential document you can just choose None.

5. You may need to repeat for the "data sources in current workbook" although Global Permissions to take precedence

Close the dialogue and refresh the queries.

Configuring Hashicorp repositories in Linux Mint (and other Ubuntu based distributions)

2022-03-07T22:45:00.007+00:00

Hashicorp distributes Packer and Terraform as both binaries and packages for popular Linux distributions.

They also have a handy apt repository for use with Ubuntu and Debian, but it generates a 404 error during apt-get update with other derivatives:

e.g.
Err: 9 https://apt.releases.hashicorp.com ulyssa Release
404 Not Found [IP: 151.101.2.49 443]

The reason is the respository only appears to contain branches for the Ubuntu and Debian codenames.

The solution is to edit the sources list and change it to use the equivalent Ubuntu/Debian codename instead:

Edit /etc/apt/sources.list.d/additional-repositories.list with your text editor of choice
The line that contains apt.releases.hashicorp.com will have the distribution codename in it

e.g. for Linux Mint 20.1 which has codeame ulyssa:
deb [arch=amd64] https://apt.releases.hashicorp.com ulyssa main
(use lsb_release -a to identify the codename)

Change the codename in the entry to the equivalent Ubuntu or Debian codename

deb [arch=amd64] https://apt.releases.hashicorp.com focal main

Then update the apt repository list and it should find the packages when you try to install Terraform or Packer with apt-get install

Diskspd - ERROR: Error adjusting token privileges for SeLockMemoryPrivilege (error code: 1300)

2021-12-14T11:17:00.003+00:00

This error is sometimes generated when choosing the -l option for DiskSpd (Use large pages for IO buffers)

I'm not sure what the benefit of testing with large pages is, so I usually just leave it off

Unable to install Powershell modules

2020-07-19T15:14:00.001+01:00

Installing powershell modules from powershell gallery on a machine with no direct internet access but which has access via a proxy just needs the proxy url / credentials supplied on the install-module command.

e.g. install-module vmware.vim -proxy http://<proxyip>:<proxy port>

However if you have no repositories registered with Powershell (such as the powershell gallery) you may still see errors relating to the NuGet provider:

NuGet provider is required to continue

PowerShellGet requires NuGet provider version '2.8.5.201' or newer to interact with NuGet-based repositories. The NuGet

provider must be available in 'C:\Program Files\PackageManagement\ProviderAssemblies' or

'C:\Users\administrator\AppData\Local\PackageManagement\ProviderAssemblies'. You can also install the NuGet provider by

running 'Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force'. Do you want PowerShellGet to install

and import the NuGet provider now?

And when attempting to install nuget, even specifying a proxy you may still get errors because the repository isn't registered.

The solution is to register the default repository via proxy, which will then allow modules to be installed

PS C:\> Get-PSRepository

WARNING: Unable to find module repositories.

PS C:\> Register-PSRepository -Default -Proxy http://<proxy ip>:<proxy port>

PS C:\> Get-PSRepository

Name InstallationPolicy SourceLocation

---- ------------------ --------------

PSGallery Untrusted https://www.powershellgallery.com/api/v2

You should now be able to install modules via the proxy with the -proxy parameter

Error after docker upgrade: "docker: Error response from daemon: OCI runtime create failed"

2020-07-02T08:48:00.002+01:00

After upgrading docker and docker cli (CentOS 7), a previously working container decided to bomb out with the following error:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"write /proc/self/attr/keycreate: permission denied\"": unknown.

It turns out that container-selinux also needs to upgraded to match.

e.g. run yum upgrade docker-ce.x86_64 docker-ce-cli.x86_64 container-selinux

Importing DFSR debug & Event logs info vRealize Log Insight

2018-03-28T09:35:00.000+01:00

If you're running a large DFS replication cluster with a large (millions) number of files, you probably have a very high turnover of both Windows Event logs and DFSR debug logs.

There are many log aggregation tools, from the freeware ELK stack and Greylog to the paid for Splunk and vRealize from VMware. The vRealize Log Insight tool doesn't have the per GB licencing model that Splunk has, but also comes up a little short on functionality in the way of pre-built "apps".

Neither have an app for DFSR but luckily it doesn't take much in Log Insight to import the DFSR logs. It's also (OVA) appliance based, so to add capacity you just deploy another appliance and join it to the cluster.

To import the DFSR logs you need to deploy the Windows Log Insight Agent to the file servers in question.

msiexec SERVERHOST=<log insight ip> /qn /i <agent msi filename>

Then once the agents are reporting into Log Insight, go to the administration page and choose management->Agents. Create a new group containing the File servers and enter this configuration

[filelog|DFSR_logs]
directory=C:\Windows\debug
include=Dfsr01000.log
event_marker=^\d\d\d\d\d\d\d\d\s\d\d:\d\d:\d\d.\d\d\d

[winlog|DFS_Replication]
channel=DFS Replication
tags={"ms_product":"activedirectory"}
parser=auto

This will ensure the DFSR debug logs are imported in multiline format as a datestamp is applied at the start of every new record.

Using Powershell values from the start of the pipeline in a table at the end.

2017-07-14T14:13:00.000+01:00

There are two ways to deal with large arrays of nested objects in powershell.

One is a bunch of nested foreach loops which is nicely readable in scripts.

However the beauty of the pipeline is that you can run output from one command into the input for another, which is often the way to drill down into dependent objects.

One such example is Vmware portgroups in Powercli.

The hierachy can be several levels deep

e.g.

cluster
-> host
-> vswitch
-> portgroup

I wanted to build a table showing portgroups on all hosts in a cluster, ignoring the default management kernel port

The command is quite simple.

Get-Cluster -name Cluster01 | Get-VMHost | Get-VirtualSwitch -name vSwitch0 | Get-VirtualPortGroup | where {$_.name -ne "Management Network" | select Name, VirtualSwitch | ft -autosize

This produces a nice table showing the portgourp name and the vswitch it's connected to.

However the hostname isn't a property of the portgroup, it's a property of the vswitch (or the host) so the question is how to show it in the table.

The answer is with the pipelinevariable command (introduced in PS4 and above) and a custom property expression.

Get-Cluster -name Cluster01 Get-VMHost | Get-VirtualSwitch -name vSwitch0 -PipelineVariable 'fi' | Get-VirtualPortGroup | where {$_.name -ne "Management Network"} | select Name, VirtualSwitch, @{Name = "VMHost";Expression = {($fi.vmhost)}} | ft -AutoSize

This returns a nice table of three columns, with the third column

Error scanning Dell hosts with VUM "Host cannot download files"

2017-07-12T14:49:00.001+01:00

This is one possible solution for the error "Host cannot download files from VMware vSphere Update Manager patch store. Check the network connectivity and firewall setup, and check esxupdate logs for details"

If you're running Dell hosts and Openmanage, there is an issue with the Openmanage VIB 8.2.0 - 8.5.0 (possibly other versions but these are the ones I've tested).

Unzip the file and have a look at the index.xml file

In my case the line containing the tag had a typo - you'd have thought Dell could spell their name, especially when it's only 4 characters long.

Edit this to be Dell and re-zip the files (making sure the files are in the root of the zipfile)

The bad news is that you can't manually remove VIBs from the VUM database once they've been imported, so you basically have to nuke your VUM config so first make a note of all your baselines / baseline groups.

At least you no longer have to uninstall and reinstall the database - there is a tool called VMwareUpdateManagerUtility.exe in the Update Manager application folder which will reset the database. Run it, point it at your VUM server, login and choose the Database settings -> re-initialize database option:

You should now be able to re-import the new VIB zipfile and perform a scan.

Unfortunately if you have alot of baselines or baseline groups, you'll need to recreate them.

Cannot create KDS Root Key - “Request is not supported”

2017-04-19T11:02:00.000+01:00

When trying to run the powershell command Add-KDSRootKey –EffectiveImmediately you get the error "Request is not supported" if the powershell console isn't running elevated as administrator.

Just right click on the powershell icon and run as administrator and rerun the command.

V2V error with vConverter standalone - Unable to obtain hardware information for the selected machine.

2015-03-26T10:31:00.001+00:00

When converting a VMWare workstation format VM (V2V) you might get this error if you're trying to read the image from a network path with a deep folder structure.

The simple resolution is to map a drive letter at the top level so the vmx and vmdk files are visible directly under the drive letter.

The converter wizard will then allow you to progress to specifying the destination system.

Windows 2012 - how to disable "run as administrator" requirement

2015-03-05T09:25:00.004+00:00

So you've just installed Windows 2012..

The first thing you do is turn off UAC just like in Windows 2008 to stop it prompting you every time java wants to update. Great! no more popups that you're just going to click ok to every time.

Then you remember you need to edit the hosts file, but what's this? You can't save it? What's going on?

The answer is you need to edit it with notepad that's been "run as administrator". But that's going to be a hassle isn't it, so how do you disable this and go back to Windows 2008 behaviour?

The answer is to edit a registry key which controls the EnableLUA setting:
HKLM:\Software\Microsoft\Windows\CurrentVersion\Policies\System\EnableLUA
Default value Windows 2008 = 0
Default value Windows 2012 = 1

You can do it via regedit or run the powershell below, in a powershell prompt that of course has been run as administrator:

Set-ItemProperty -Path "HKLM:\Software\Microsoft\Windows\CurrentVersion\Policies\System" -Name "EnableLUA" -Value "0"

The server needs to be rebooted which you can do from powershell too:

shutdown -r -t 0

Hey presto, no more prompts.

Credit to this post for the solution.

Network Policy Server Role won't install on Windows 2008

2015-03-04T10:04:00.001+00:00

I recently had a weird issue while trying to install TMG 2010 (Microsoft Threat Management Gateway) on Windows 2008R2 where the pre-requisites wouldn't install and kept giving an 0x80072afc error

I narrowed this down to the Network Policy Server role which wouldn't install.

Googling the error "windows could not start the network policy server services" along with the error code "0x80072afc" revealed quite a few articles, mostly relating to missing permissions on the IAS folder within the system directory and/or active directory issues.

I decided to take a clone of another known-working TMG as a workaround, but on this box the Network Policy Server service wouldn't start which led me to believe it wasn't permissions related.

The fix I eventually found was much simpler. I'd been trying to install TMG with the server in a workgroup and disconected from the network.

I plugged in the network card so the server picked up an IP from DHCP and Hey Presto! the TMG pre-req wizard then completed ok!

Understanding DFS replication processes and what to do if it stops working

2014-12-16T14:09:00.002+00:00

First of all, a quick introduction to Microsoft's DFSR - DFSR was introduced in Windows 2003 R2 and is the mechanism used to replicate files between servers. This is especially useful when you are using DFS namespaces to publish file shares as you want all possible targets in DFS to have the same files on them. If you're now lost, you should probably go and read up on DFS concepts on Microsoft's sites.

Now on to the nuts and bolts of how the DFS Replication service works.

The DFS Replication service maintains a database of filenames, paths and hashes in the system volume information\DFSR folder. It also holds a copy of the database in memory when it's running. There is only a single database *per drive letter*. Do not mess with this!

DFSRPrivate is a symlink in each folder configured for replication which points to the DFSR database.
When the first member is added to a replication group, it’s designated primary and builds it’s database of file hashes.
During the build process it is marked as primary (check with dfsradmin membership list /RgName:xxxxx /Attr:MemName,RfName,IsPrimary where xxxxx is your replication group name - use the full path including the fqdn if present)
Once the database build is complete, the primary flag goes away and an Event 4112 is logged in the event log. Also the replication state changes to 4.

e.g. (where dfsns is the namespace)

Before:
D:\>dfsradmin membership list /RgName:domain.local\dfsns\RepGroup1 /Attr:MemName,RfName,IsPrimary
MemName RfName   IsPrimary
SERVER1 FOLDER1 Yes
D:\>Wmic /namespace:\\root\microsoftdfs path dfsrreplicatedfolderinfo get replicationgroupname,replicatedfoldername,state | find /I "FOLDER1"
FOLDER1      domain.local\dfsns\RepGroup1   0

After:
D:\>dfsradmin membership list /RgName:domain.local\dfsns\RepGroup1 /Attr:MemName,RfName,IsPrimary
MemName RfName   IsPrimary
SERVER1 FOLDER1 No
D:\>Wmic /namespace:\\root\microsoftdfs path dfsrreplicatedfolderinfo get replicationgroupname,replicatedfoldername,state | find /I "FOLDER1"
FOLDER1 domain.local\dfsns\RepGroup1 4

If another server is introduced to the replication group while this process is happening, bad things ™ happen, so let it complete! This can take several hours on very large servers.
The database is built with something called fence value assigned in the database against each file. This is used in the event of conflicts (e.g. the same file being found in the same place on another server)

All files on the primary member are assigned the “Initial Primary” fence value. This guarantees that this server is considered the authoritative server during initial replication. If another server is introduced before the initial database build is complete, that second server considers itself primary too, so it will assign the same fence value, hence conflicts and the bad things ™.

Once the database has been built, a second member can be introduced. This will start building its database and assign the Initial Sync fence value to all files it finds (assuming there are existing files). The second server will then compare the database with the first server and start copying over any missing or different files.

•    If a file doesn’t exist on the secondary member, it will just be copied over and it will move to the next file
•    If a file exists already, then the fence values are compared for conflict resolution.
   o    The higher fence value wins and overwrites the lower. RDC (remote differential compression) is used to check the files for differences (comparing the file hashes) and changed blocks are copied to the second if required (usually nothing will be copied because the files are usually the same e.g. preseeded
   o    If they have the same fence value, the bad things ™ now occur as there is a conflict. Conflict resolution is invoked and uses first create time, last modified time to decide which file should win. Conflicting files are moved to the DrfsPrivate\ConflictedandDeleted folder.
This means that the live data on the primary server is moved out of live shares. It will stay in the ConflictedandDeleted folder until it runs out of quota at which time it will be flushed.
This isn’t necessarily the end of the world because the file is still on the second server and *should* eventually be replicated back, but if a User is looking for their file before this happens, they will not see it.

More information on the sequencing can be found on the MS page here:

As indicated below, the Initial Primary fence (2) is higher than the initial sync value (1) which are the two values assigned during the initial setup of replication.

0 Unfence This file or folder will lose all conflicts.
1 Initial Sync Initial fence value for non-primary member.
2 Initial Primary Initial fence value for primary member.
3 Default Default fencing value.
4 Fence Fence with current time stamp.

How to hide unknown devices in Dell OpenManage Essentials (or IT Assistant)

2014-12-16T13:57:00.001+00:00

When discovering a large number of devices, usually you'll pick up a few non-Dell bits of hardware, such as Virtual Machines or network switches.

Luckily Dell have an option to avoid cluttering up OME with these, although of course they have put it in a wierd place.

Instead of being with all the main configuration options under preferences, it's actually under the discovery schedule settings in Manage / Discovery and Inventory / Discovery Schedule

Force removal of vmkernel ports

2014-09-24T16:28:00.003+01:00

Trying to unbind a vmkernel port from an iSCSI adapter that is currently in use fails with the message
"Unable to unbind iscsi port."

Use the --force true parameter to force removal
esxcli iscsi networkportal remove -A vmhbaxx-n vmkx --force true

Equallogic setup.pl script not working with vCLI 5.5

2014-09-24T15:43:00.001+01:00

Dell supply (with the VMware Multipath module) a perl setup script for their Equallogic arrays to aid configuring VMware host networking (distributed switches are not recommended for storage).

This script relies on both perl, and the VMware command line interface (vCLI) to connect to hosts.

The vCLI version usually needs to match the host ESXi version you're trying to connect to, but the latest v5.5 vCLI appears to be missing some modules.

The usual way to install them would be to use ppm or cpan, but ppm as implemented in the vCLI doesn't appear to have all modules listed and if you've installed vCLI to the default path, dmake.exe will have issues with the spaces in the path when installing modules with cpan. The answer to this is to install it to a short directory name off the root (e.g. c:\vCLI)

Once you're able to install modules, from the vCLI run the following commands:
cpan
install CPAN
reload cpan
install SOAP::Lite
install MIME::Parser
install Data::UUID

This should install all the required components for the setup script.

Unfortunately the UUID module isn't installed into the default include (@INC) path so you need to manually specify it.

You can do this either by setting an environment variable or my preferred option of manually specifying the path in the perl command to run the script.
First locate the module in your installation
dir c:\UUID.pm /S

Then run perl with the -I (capital i) parameter:
perl -I C:\vCLI\Perl\site\lib\Data setup.pl

See this page for more information on non-default @INC paths: http://perlmaven.com/how-to-change-inc-to-find-perl-modules-in-non-standard-locations

Changing network settings in Linux

2014-03-12T16:30:00.001+00:00

Change IP
ifconfig eth0 192.168.0.10 netmask 255.255.255.0 up
or edit configs:
Debian / Ubuntu: /etc/network/interfaces
Redhat: /etc/sysconfig/network/network-scripts/ifcfg-eth0 or run command /usr/sbin/netconfig

Add a new IP
ifconfig - a 192.168.0.11 netmask 255.255.255.0

Set default gateway
route add default gw 192.168.0.1

Change dns servers
edit resolv.conf with:
nameserver 8.8.8.8 #google public dns
nameserver 208.67.222.222 #opendns

Dell Equallogic Virtual Storage Manager (VSM) reinstall fails to register in vCenter

2013-11-15T13:30:00.003+00:00

A nice little bug I've found in the Dell VSM appliance is that if you reinstall it (blowing away the original VM and deploying it from the OVF again) it won't register correctly in vCenter.

Even removing the old vCenter plugin entries through the vCenter web gui won't fix it

The simple workaround is to assign a new IP to the new VM and it'll register fine.

Microsoft product activation fails with error 0x80000048

2013-10-09T16:09:00.001+01:00

If you need to activate windows over the phone (e.g. for a server isolated from the internet in say, a DMZ) but get error 0x80000048, you'll need to enable (and start) the Telephony service which seems to be needed to pull up the phone numbers you need to call.

The updates you are trying to apply are not Dell-authorized updates

2013-10-01T14:39:00.004+01:00

If you get the error "The updates you are trying to apply are not Dell-authorized updates" on a Dell Poweredge server when trying to update firmware with the Lifecycle Controller (F10) option, you need to update the DRAC and LC firmware through the DRAC gui (or with RACADM).

You need:

iDRAC – at least version 1.85 or newer

Lifecycle Controller – at least version 1.5.2 or newer

VMware vCenter Support Assistant can't resolve vCenter server name

2013-09-26T11:41:00.004+01:00

The vCenter Support Assistant is a free Linux (SUSE) based virtual appliance to aid support requests in a vmware environment.

It's quite easy to setup but it has an odd quirk with DNS resolution.

By default, it will try to connect to the vCenter name you use in your vCenter vi-client, and it will ignore the name you use in the setup wizard to register with vCenter.

e.g. if you register with a FQDN for vCenter, but then use the short name in your Windows vi-client, it will report connectivity from virtual appliance to vCenter FAILS during it's diagnostics checks run at first login.

The solution is to manaually go into the linux console and etc the /etc/resolve.conf file and add a search suffic

1. Open Console
2. Login as root
3. edit the resolve.conf file
    vi /etc/resolve.conf
4. append the search suffix to the end of the file
    search yourdomain.com
5. save the file
    esc
    !wq!
6. rerun diagnostics

The last two checks should now pass.

Version of VCSA used was 5.1.1. YMMV with other versions.