This is just a brief overview. The installation process is pretty easy on these. Same as with most patches, IBM UpdateInstaller “update.sh” was used to install the service “pak” files. These patches must be done in order. Patch the WAS installation before patching WPG.

All WebSphere services must be stopped to install the WAS updates. On a standard installation, bcguser must be used to stop the service

[bcguser@WPGhost ~]$/opt/IBM/bcghub-simple/bin/./bcgStopServer.sh

We do not use ‘/opt’ for our WebSphere location, so change this if yours is different.

Next, use Update Installer to patch the WebSphere Application Server
[user@WPGhost ~]$ sudo /opt/IBM/WebSphere/UpdateInstaller/./update.sh

There is a gotcha here that had me “chasing my tail” for about 10 minutes. When going to install the WebSphere Partner Gateway fix pack, the Partner Gateway and WAS server must be started. Installation of the update will fail with error “user input validation”.

So before installing the WPG update, re-launch the Application Server and Partner Gateway

[bcguser@WPGhost ~]$ /opt/IBM/bcghub-simple/bin/./bcgStartServer.sh

Once done, launch IBM Update Installer again, passing the customized responce file for your environment. This needs to be executed as the root user, so sudo was used to allow xforwarding from a non-root account

[user@WPGhost ~]$ sudo /opt/IBM/WebSphere/UpdateInstaller/./update.sh -options /opt/IBM/bcghub-simple/responsefiles/bcgupdate_en_US.txt

Those are my miscellaneous notes about the update installation. Everything went fine here and I hope this fixes some of the SFTP issues we have been having.
This brings the WebSphere Partner Gateway Console form version 6.2.0.0.273 to 6.2.0.1.333

Notes: Here is the link to IBM’s website that lists the fixes that are provided in the update.

I have been wrestling around with IBM WebSphere Partner Gateway for a few weeks now.  There are so many tiny gotcahs out there that can affect the whole installation process.

The main one that got me was integration with Oracle.   An overview of the installation steps are shown below:

So the problem came down to the ‘Database owner name’ and ‘Schema owner login’ being the same.  This typically is not an issue.  The worst part is that the WAS (WebSphere Application Server) and WPG (WebSphere Partner Gateway) installation would both complete successfully. Not only that, they system would run with no errors.

That being said, once I started the patching process, it would always fail.   So as a last resort, I tried changing the ‘Database user name’ and ‘Schema owner login’ to be different.   Thanks to DBA Eric’s recommendation.  This worked!

I decided to put this blog up because I could not find any useful information for this when searching.  The patching process is a pain and I might go into more details on it in more blogs later.  Anyone else ran into this issue?

I am a firm believer in regular password rotation/change and Linux has a built in mechanism that makes it easy. The following is a brief overview of password and account ageing for Linux based systems.

The program that enables listing and modification on the expiration parameters is ‘chage’. Each individual user can view their account settings as shown below.
testuser@testServer:~$ chage -l testuser

Last password change					: Aug 07, 2009
Password expires					: Nov 05, 2009
Password inactive					: never
Account expires						: Aug 05, 1992
Minimum number of days between password change		: 90
Maximum number of days between password change		: 90
Number of days of warning before password expires	: 7

As you can see above, the last password change date is listed, as well as the expiration date for the current password. When executed from a non-privileged account, the user can only view their own account.

testuser@testServer:~$ chage -l root
chage: Permission denied.

Also, the non-privileged account can not change their settings either.

testuser@testServer:~$ chage -M 99 testuser
chage: Permission denied.

From the root account, you have to ability to modify all the settings for individual users.

root@testServer:~# chage
Usage: chage [options] [LOGIN]

Options:
  -d, --lastday LAST_DAY        set last password change to LAST_DAY
  -E, --expiredate EXPIRE_DATE  set account expiration date to EXPIRE_DATE
  -h, --help                    display this help message and exit
  -I, --inactive INACTIVE       set password inactive after expiration
                                to INACTIVE
  -l, --list                    show account aging information
  -m, --mindays MIN_DAYS        set minimum number of days before password
                                change to MIN_DAYS
  -M, --maxdays MAX_DAYS        set maximim number of days before password
                                change to MAX_DAYS
  -W, --warndays WARN_DAYS      set expiration warning days to WARN_DAYS

Before modification, I am going to turn off all expiration settings on the ‘testuser’ account. This is disabling password expiration on that individual account.

root@testServer:~# chage -E -1 -I -1 -m 0 -M 99999 testuser

No lets configure password aging for the test user. The first example below runs change in interactive mode.

root@testServer:~# chage testuser
Changing the aging information for testuser
Enter the new value, or press ENTER for the default

	Minimum Password Age [0]:
	Maximum Password Age [99999]: 90
	Last Password Change (YYYY-MM-DD) [2009-10-16]:
	Password Expiration Warning [7]:
	Password Inactive [-1]:
	Account Expiration Date (YYYY-MM-DD) [1969-12-31]: 2012-12-31

Verify that the settings took.

root@testServer:~# chage -l testuser
Last password change					: Oct 16, 2009
Password expires					: Jan 14, 2010
Password inactive					: never
Account expires						: Dec 31, 2012
Minimum number of days between password change		: 0
Maximum number of days between password change		: 90
Number of days of warning before password expires	: 7

The same can be accomplished using the command line, non-interactively.

root@testServer:~# chage -E 2012-12-31 -I -1 -m 0 -M 90 -W 7 testuser

With the above settings in place, the user should be warned 7 days before the password expires on their account. If the password is not changed before expiration day, on the next login the user will be forced to change their password.

A page is really virtual memory which is managed by the Translation Lookaside Buffers(TLB) in the CPU. The TLB controls the mapping of the virtual memory pages to physical memory addresses. In doing so, it bypasses the kernel virtual memory manager.

Per RedHat,

The TLB is a limited hardware resource, so utilising a huge amount of physical memory with the default page size consumes the TLB and adds processing overhead – many pages of size 4096 Bytes equates to many TLB resources consumed.

This is where Huge Pages come in. Pages are created at a larger size than the default 4096 bytes, and each page will consume only one TLB resource. So you can see this is a huge benefit. Using Huge Pages decrease the number of TLB resources required.

Side Affect
This is great, depending on what you are trying to accomplish. Once the physical memory is mapped to a Huge Page, it can no longer be used for “normal” memory allocation. This is because the memory is no longer mapped by the kernel virtual memory manager. The applications that you want to dedicate the Huge Pages to have to have support for them.

Benefit
So here is the best part of Huge Pages. It is dedicated memory to be used by only applications that request them. This dedicated memory is stored in physical RAM and will NEVER be swapped out! Thus, guaranteeing a level of performance. When memory is swapped to disk, it’s a lot slower than RAM and decreases the performance of the process(s)/program(s) gets pushed there.

Now knowing that Huge Pages are stored in RAM, this also means that the allocated RAM is dedicated. This is a little bit redundant to the above, but I want to make sure this point is clear.

Example: If a server has 8gigs of RAM and 5gigs are allocated to Huge Pages, that only leaves 3gigs for all other processes, programs, and underlining operating system to use.

Below shows my Linux desktop that has the default page size of 4096 set

user@workstation:~$ cat /proc/meminfo | grep Huge
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       4096 kB

So as you can see, I have no Huge Pages reserved or in use. The next example is from a production Oracle database server

[root@OracleServer1 ~]# cat /proc/meminfo | grep Huge
HugePages_Total: 12200
HugePages_Free:     85
Hugepagesize:     2048 kB

So to calculate the space dedicated to Huge Pages from above, it is 12,200 x 2048 kB which gives us

24 985 600 kilobytes = 23.828125 gigabytes

In the 2.6x Linux kenel, Huge Pages are enabled using the CONFIG_HUGETLB_PAGE feature when compiling the kernel. Most “Enterprise” Linux OSs by default have this enabled. The ones that I know of are RedHat, CentOS, and possibly Fedora from version 4+.

Notes: Again, applications that you want to dedicate Huge Pages to must have support for them. Most memory intensive ones do, but check for this first.

So I am re-doing my external RAID 1 drive enclosure. I love this little thing. It has two 2.5 inch 160gig SATA drives in it. The enclosure is connected via USB 2.0 but it does have an eSATA interface as well. I will be configuring this to have a 10 gig non-encrypted partition. The remaining ~150 gigs will be an encrypted (LUKS) filesystem to be used on my linux machine.

All of this will not be detailed here but will be split up in 3 blogs. Below just shows the time it takes to use Linux to overwrite the disk device using /dev/urandom. This is done to make it just that much harder for a would be hacker to try and brute force the key on the encrypted partition. If this is not done, the un-used space would just show up as empty, allowing for a more targeted attack against the pseudo random filesystem. Being pseudo-random means that it is not truly random. This being the case, with a lot of time and computing power, an attacker might be able to either brute force or find a pattern in the encryption.

So why not use /dev/random? For me, this would take forever! I do not have any special hardware or scripts pulling information from the environment and adding to the entropy pool. The data on this drive not being national security grade, /dev/random will do the job.

I know that the drive is under /dev/sdb. With that information, it is as simple as using “dd” (built in Linux utility) to overwrite all blocks on the drive with pseudo-random data.

root@tstbox:~# dd if=/dev/urandom of=/dev/sdb
dd: writing to `/dev/sdb': No space left on device
312581810+0 records in
312581809+0 records out
160041886208 bytes (160 GB) copied, 40284.5 s, 4.0 MB/s

From above, it tool 40,284.5 seconds to overwrite the drive with urandom data. This equals ~11 hours and 19 minutes. Definitely still a long time, but a lot faster than if /dev/random was used.

This workstation is not an impressive computer. It is a single CPU dual core machine with 2 gigs of ram. Below is the info on one of the cores.

root@tstbox:~# cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Core(TM)2 CPU          6300  @ 1.86GHz
stepping	: 2
cpu MHz		: 1867.000
cache size	: 2048 KB
physical id	: 0
siblings	: 2

Notes: I wish I could use /dev/random and probably will eventually when I can sit a drive out for a week. Setting up external drives in this fashion is really geared towards data protection. Not only are the drives in a mirrored RAID (one can fail and everything would still run fine), important data is encrypted using a strong key. So who cares is the external enclosure walks away at a conference? I would be out ~190$ but the data will be safe.

This one will be quick and easy! Below is how to install a new port license on a Cisco MDS 9000 switch from the Cisco CLI (Command Line Interface). Doing this does not remove the current license, just adds it to the configuration. As always though, back up your configuration and make sure if there is a current license that you also have a backup copy of it.

First, make sure you put a copy of the license onto a tftp, ftp, or sftp server. The MDS switch supports all of those protocols. Here we will be using tftp.

Copy the license from TFTP server to bootflash (persistent storage)

mds9124# copy tftp://172.0.0.1/MDS20090209112333135513.lic bootflash:
Trying to connect to tftp server......
|
 TFTP get operation was successful

Install the license

mds9124# install license bootflash:MDS20090209112333135513.lic
Installing license .......done

Now that the new port license is installed we need to verify that it is working. Below shows the default licensing that came with the unit.

mds9124# show license default
Feature                               Default License Count
-----------------------------------------------------------------------------
FM_SERVER_PKG                         -
ENTERPRISE_PKG                        -
PORT_ACTIVATION_PKG                   8
10G_PORT_ACTIVATION_PKG               0
-----------------------------------------------------------------------------

The new one contained licensing for an additional 8 ports. Below you can see that now there are 16 ports licensed.

mds9124# show license usage
Feature                      Ins  Lic   Status Expiry Date Comments
                                 Count
--------------------------------------------------------------------------------
FM_SERVER_PKG                 No    -   Unused             -
ENTERPRISE_PKG                No    -   Unused             -
PORT_ACTIVATION_PKG           Yes  16   In use never       -
10G_PORT_ACTIVATION_PKG       No    0   Unused             -
-------------------------------------------------------------------------------

<pre>mds9124# copy tftp://172.0.0.1/MDS20090209112333135513.lic bootflash:
Trying to connect to tftp server......
|
TFTP get operation was successful</pre>
Install the license
<pre>mds9124# install license bootflash:MDS20090209112333135513.lic
Installing license .......done<pre>

<pre>mds9124# show license default
Feature                               Default License Count
-----------------------------------------------------------------------------
FM_SERVER_PKG                         -
ENTERPRISE_PKG                        -
PORT_ACTIVATION_PKG                   8
10G_PORT_ACTIVATION_PKG               0
-----------------------------------------------------------------------------<pre>

<pre>mds9124# show license usage
Feature                      Ins  Lic   Status Expiry Date Comments
Count
--------------------------------------------------------------------------------
FM_SERVER_PKG                 No    -   Unused             -
ENTERPRISE_PKG                No    -   Unused             -
PORT_ACTIVATION_PKG           Yes  16   In use never       -
10G_PORT_ACTIVATION_PKG       No    0   Unused             -
-------------------------------------------------------------------------------</pre>

I always use ’sudo su -’ when I need to get to a root shell. I have seen a few people before, and a new co-worker recently use ’sudo -s’. Since I could not remember off hand the actual differences between the two, I had to check. The following will run through the actual limitations.

The big difference when using ‘-s’ are listed below

[root@testServ01 ~]# printenv
HOSTNAME=testServ01.testDomain.com
SHELL=/bin/bash
TERM=xterm
HISTSIZE=1000
USER=root
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
INPUTRC=/etc/inputrc
PWD=/root
LANG=en_US.UTF-8
SHLVL=1
HOME=/root
LOGNAME=root
CVS_RSH=ssh
LESSOPEN=|/usr/bin/lesspipe.sh %s
DISPLAY=localhost:10.0
G_BROKEN_FILENAMES=1
_=/usr/bin/printenv

DISPLAY=localhost:10.0
HISTSIZE=1000
HOME=/home/testuser01
HOSTNAME=testServ01.testDomain.com
INPUTRC=/etc/inputrc
LANG=en_US.UTF-8
LOGNAME=root
MAIL=/var/spool/mail/testuser01
PATH=/usr/bin:/bin
PWD=/home/testuser01
SHELL=/bin/ksh
SUDO_COMMAND=/bin/ksh
SUDO_GID=500
SUDO_UID=500
SUDO_USER=testuser01
TERM=xterm
USER=root
USERNAME=root

# iptables -L
/bin/ksh: iptables: not found [No such file or directory]

# /sbin/iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere            

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

So you need to find the serial number on your Cisco MDS 9000 series fiber switch? This is easy enough, although “show serial number” would have been better.

Quick way to find your serial number.

tstSwitch01# show license host-id
License hostid: VDH=SOZ115568P9

The following will also get the information that you need. I truncated some of the output. The serial number under the “Common block” is what we need.

tstSwitch01# show sprom backplane 1
DISPLAY backplane sprom contents:
Common block:
 EEPROM Size     : 1024
 Block Count     : 5
 FRU Major Type  : 0x6003
 FRU Minor Type  : 0x0
 OEM String      : Cisco Systems, Inc.
 Product Number  : DS-C9124-K9
 Serial Number   : SOZ115568P9
 Part Number     : 73-10565-03
 Part Revision   : A9
 Mfg Deviation   : 0
 H/W Version     : 1.0
 Mfg Bits        : 0
Chassis specific block:
 Block Signature : 0x5601
 MAC Addresses   : 00-0g-tr-46-n3-u6
 Number of MACs  : 64

This is a little easier to read. Here, an include statement is passed to only return lines including “Serial”. We need the first, not second serial number.

tstSwitch01# show sprom backplane 1 | include Serial
 Serial Number   : SOZ115568P9
Second Serial number specific block:
 Serial Number   : JFH2486G4DR

Notes: All actual serial numbers were changed. This process should be the same for all Cisco MDS 9000 series. If using a chassis based MDS switch, make sure to verify if you need the serial of the unit or the actual blade module for licensing.

This will review the partitioning scheme that I am currently using for VMware vSphere (ESX 4). For information concerning partitioning for VMware ESX 3.x, please refer to the following link:

http://blog.colovirt.com/2008/10/31/vmware-esx-server-partitioning/

The majority of the partitioning structure is the same what was used for 3.x. The only real changes is the fact that the installation process auto-creates two of the partitions that were to be manually created on 3.x. Those two partitions are:

As in the 3.x structure, again I still maintain that creating a seperate mount point for /var/core should be used. For the reasons stated below:From the 3.x post

“I have had a few servers core dump and drop over 5 gigs of data to /var/core. Before, per “best practices” a vendor recommended around 4 gigs for /var. I upped that to 6 gigs originally, but after 2 servers had /var 100% utilized I and revising that. /var is still 6 gigs but /var/core has been broken out into its own mount point. 15 gigs is a little high, but these servers had raid 1 – 73 gig hard drives. At least now if the servers core dump it will affect only its mount point. I highly recommend doing this!”

Below is how I am partitioning vSphere 4 servers

Notes: Yellow-Bricks (Duncan Epping), as well as VMETC have good articles as well.

We all like blink lights.  Well, maybe not all, but they are great for remote assistance at off-site datacenters.  All engineers are not equal and trying to talk some through locating an interface and changing a cable can sometimes be as efficient as banging your head against the wall.

If a cable change or SFP swap is all that is needed, than beaconing can help out!  Below is a quick run-though of enabling and disabling beaconing “blinky mode”.
Before we enable beaconing, check the interface to verify the beacon status

mds9000# sh interface fc1/20 | inc Beacon
    Beacon is turned off

Connect to the Cisco switch and move into configuration mode.

mds9000# config t
Enter configuration commands, one per line.  End with CNTL/Z.

Once in config mode, go into the specific port that needs to be located.

mds9000(config)# interface fc 1/20

General sub-parameters are listed below.

mds9000(config-if)# switchport ?
beacon        Disable/enable the beacon for an interface
description   Enter description of maximum 80 characters
encap         Configure encapsulation for the port
fcbbscn       Disable/enable buffer-to-buffer state change notification
fcrxbbcredit  Configure receive BB_credit for the port
fcrxbufsize   Configure receive data field size for the port
ignore        Enter parameter to be ignored
mode          Enter the port mode
rate-mode     Configure the rate mode for an interface
speed         Enter the port speed
trunk         Configure trunking parameters on an interface

Easy enough, tell the system to enable beacon.

mds9000(config-if)# switchport beacon

Exit config mode.

mds9000(config-if)# end

Re-check the interface to verify that beaconing is on

mds9000# sh interface fc1/20 | inc Beacon
    Beacon is turned on

So now the LEDs below the port should be flashing pretty lights and easily be located by anyone.  Once the fiber/SFP swap is completed, go back into config mode and disable beaconing.

mds9000# config t
Enter configuration commands, one per line.  End with CNTL/Z.
mds9000(config)# interface fc 1/20
mds9000(config-if)# no switchport beacon
mds9000(config-if)# end

Or you can use the GUI.  The following image shows how to enable and disable beacon from Cisco Device Manager.  It's easy as right-clicking on the port and going to beacon.

Beacon

Notes:  Pretty easy and basic command, but extremely useful!

After spending a year using the a RamSan 400 which is a 128 gigabyte solid state DRAM system, I wondered how it could get better.  These things are pretty expensive and the only drawback I found to the 400 is the limited storage.  There’s not much that you can do with 128 gigs of storage!  Granted it served the initial function perfectly.  This was used to house our main production database.

About 4 months ago, we were looking to add another RamSan into our environment after deciding EMCs flash DAEs were not for us (at this time).  I was initially a little weary of flash based storage, but had faith in RamSan.  After reviewing the product literature more and getting some hands on with the unit, those worries are gone.  Our RamSan 500 was purchased with 2 terabytes of Flash storage and the DRAM write cache was expanded to 64 gigabytes.

With the write cache expansion, this new units cache was literally 1/2 the size of the current RamSan 400 DRAM capacity.  With this in place and the high IO that the flash disks can maintain, there has not been any noticeable speed differences between the two units.  The 500 also has an internal battery for power backup.  If power is lost, the battery will keep the unit going long enough to copy the cache’s un-written data to disk.

There are two main features that I love.  First is tha ability to lock a LUN on the 500 series into the cache, so even-though it is mainly a flash storage system, extreme IO can be accomplished.  By locking the lun in cache, the data will reside in RAM and never go to disk unless power is lost.  The second is the ability to centrally manage other RamSans on the network (auto discovered) with a specific license key.  Almost forgot to mention, the unit has quad 4 gigabit fiber connectivity.

Notes:  If you have any questions, please ask!  Again, these units are amazing.

Going over the solid state offerings for the EMC Clariion lines, Texas Memory RamSans came into the conversation.  This was due to the fact that we currently run 2 RamSans in our Environment and consider them the highest tier storage in our datacenters.  One is 128 gigs of solid state DRAM storage and the other is 2 terabyte solid state Flash storage with a 64 gig DRAM cache.

Per the title, this is really about the EMC Clariion, not RamSans.  Since the RamSan 500 was fronted with the DRAM cache, and the EMC CX4 series contains cache as well, I was curious.  I already knew that each Service Processor (SP) in the EMC has 4 gig of cache, and that a LUN can only be active on one SP at a time.  Also, per a previous blog post, each DAE has a theoretical max throughput of 8 gigabit per second, 4 gigabit if a single LUN stripes across the whole shelf.

CX4 DAE (general) information
http://blog.colovirt.com/2009/05/29/san-emc-cx4-dae-drive-shelf-information/

As the conversation continued, I was told that typically the Service Processors caching would be disabled on LUNS that reside on the solid state flash drives.  In all actuality, I think it should.  Being that the typical Clariion implementation will not be dedicated as a solid state SAN, they will have to co-mingle with spindle based hard drives(regular).  The throughput is a LOT faster and latency a lot lower on the flash drives.  Enabling SP caching on LUNs contained in flash could possibly have a really bad affect on the rest of the system.  Being that the cache really speeds up IO for spindle based disks.

Most vendors are toting a 20 physical disks(15kRPM) to 1 Flash disk consolidation

What I am getting at is there seems to be the possibility of flooding the Service Processors cache with flash based storage data, depending on change rate.  Since flash is a lot faster, their LUNs would typically be used for databases or high volume services.  Even with the ability to do QOS in the CX4 series, disabling Service Processor caching for the flash LUNS looks to be the best solution.

This is quick and a little basic, but most people do not actually read the “man pages” or documentation.  The majority of the time, requests for access comes in specifying IP address instead of hostnames (FQDN).  I actually prefer this, but when doing a typical “iptables -L”, the reverse DNS is automatically checked for all IPs.

Most of the time I do not actually know the hostname that is associated and makes it hard to confirm the rule without doing a dns lookup on my own.  Below is the typical output of the command.
[root@testserver ~]# iptables -L

Chain Firewall-INPUT
(2 references)
target     prot opt source               destination
ACCEPT     tcp  --  mail.asdf.com        anywhere            tcp dpt:ssh
ACCEPT     tcp  --  static.123.net       anywhere            tcp dpt:ssh
ACCEPT     tcp  --  private.9z.com       anywhere            tcp dpts:ftp-data:ftp
ACCEPT     tcp  --  nto.ntpgr.com        anywhere            tcp dpts:ftp-data:ftp

Iptables has a built in option to disable DNS resolution.  This is done by passing “-n” in conjunction with “-L” and shown below.

[root@testserver ~]# iptables -L -n
Chain Firewall-INPUT
(2 references)
target     prot opt source               destination
ACCEPT     tcp  --  10.1.129.119         0.0.0.0/0           tcp dpt:22
ACCEPT     tcp  --  172.168.22.87        0.0.0.0/0           tcp dpt:22
ACCEPT     tcp  --  172.33.100.2         0.0.0.0/0           tcp dpts:ftp-data:ftp
ACCEPT     tcp  --  10.90.15.104         0.0.0.0/0           tcp dpts:ftp-data:ftp

Above you can see how easy it would be to verify the rules now without knowing the hostname or performing a lookup on your own.

Notes:  The iptables output was edited to remove non-relevant information and all IPs/hostnames were changed.

I like running ESXi via booting from USB 2.0 memory sticks.  This makes it that much easier for my home lab.  Especially since I mainly use ISCSI VMFS datastores.  Not to mention that “in a pinch”, having ESXi on memory sticks can aide in disaster recovery (DR) scenarios for small businesses.  Of course the requirement here is that the server MUST be able to boot from USB!  Also, get a big memory stick.  Each time an upgrade is performed to ESXi, the version being upgraded from is still stored on the memory stick in case a “roll back” is needed.  At least this is my understanding.  Larger memory sticks are pretty cheap now.  Below outlines the steps to creating a bootable ESX4i memory stick.  The main reason for me writing this up is that the process turns out to be different from ESX3i.

Even though this is being done using Linux, the directory structure and location of data files will be helpful in doing this from Windows.

Below is a link to Yellow Bricks (Duncan Epping) VMware blog.  He goes through the process of creating the USB boot device for Windows.  If you are doing this from Linux, continue on from this section.
http://www.yellow-bricks.com/2009/06/09/vsphere-esxi-on-a-usb-memory-stick/

Create a temporary mount point where the downloaded ESX4i ISO will be placed

root@laptop:~# mkdir /mnt/temp

Next, “mount -o loop” is used to mount the ISO to the /mnt/tmp location.  Using the loop option, it allows us to view and traverse the ISOs structure.

root@laptop:~# mount -o loop /home/user01/Desktop/VMware-VMvisor-Installer-4.0.0-164009.x86_64.iso /mnt/temp

Change into the temp directory and check the contents

root@laptop:~# cd /mnt/temp
root@laptop:/mnt/temp# ls
boot.cat    ienviron.tgz  isolinux.bin  menu.c32    vmkboot.gz
cimstg.tgz  image.tgz     isolinux.cfg  README.txt  vmkernel.gz
cim.vgz     install.tgz   mboot.c32     sys.vgz

For ESX4i, the file that actually contains what we need is image.tgz.  I created the “/root/build/vi4″ subdirectory for building this configuration.  the tgz file is copied there shown below since /mnt/tmp is read only.

root@laptop:/mnt/temp# cp image.tgz /root/build/vi4

Move into the vi4 directory and verify the file was copied

root@laptop:/mnt/temp# cd /root/build/vi4
root@laptop:~/build/vi4# ls
image.tgz

Un-compress and extract the contents of image.tgz into the current directory

root@laptop:~/build/vi4# tar -xzvf image.tgz
usr/
usr/lib/
usr/lib/vmware/
usr/lib/vmware/installer/
usr/lib/vmware/installer/VMware-VMvisor-big-164009-x86_64.dd.bz2

Above lists the directories and files created when the contents were extracted.  So we see that the dd file was listed in the output.

Move into the directory containing the dd file and list the contents

root@laptop:~/build/vi4# cd usr/lib/vmware/installer/
root@laptop:~/build/vi4/usr/lib/vmware/installer# ls
VMware-VMvisor-big-164009-x86_64.dd.bz2

Here, bunzip2 is used to de-compress the VMware-VMvisor-big-164009-x86_64.dd.bz2 file.  This took about 10 minutes on a slow laptop.

root@laptop:~/build/vi4/usr/lib/vmware/installer# bunzip2 -d VMware-VMvisor-big-164009-x86_64.dd.bz2

Now we can see the fully decompressed dd file, which is almost 1 gig in size.

root@laptop:~/build/vi4/usr/lib/vmware/installer# ls -lah
total 901M
drwxr-xr-x 2 201 201 4.0K 2009-05-25 15:16 .
drwxr-xr-x 3 201 201 4.0K 2009-04-30 22:08 ..
-rw-r--r-- 1 201 201 900M 2009-04-30 22:11 VMware-VMvisor-big-164009-x86_64.dd

I already removed all of the partitions from the memory stick being used here.  I used gparted to accomplish this, but fdisk can be used as well.  Once the memory stick to be used is connected, check /var/log/messages to see what device has been assigned to the memory stick.

Here, I already knew that /dev/sdb was assigned to the USB memory stick.  The “dd” command thats standard on Linux is used to identically copy the VMware-VMvisor file to the memory stick.

root@laptop:~/build/vi4/usr/lib/vmware/installer# dd if=VMware-VMvisor-big-164009-x86_64.dd of=/dev/sdb
9-x86_64.dd of=/dev/sdb
1843200+0 records in
1843200+0 records out
943718400 bytes (944 MB) copied, 322.119 s, 2.9 MB/s

Once this was complete, the USB memory stick was ready to go.  I went over to the server, attached it, and ESX4i was immediately booted with no issues.  Configure the management IP and user from the console and you are ready to go! The same process needs to be repeated for each memory stick that needs to be created.  You could just use “dd” to copy the ESXi memory stick to another memory stick, but that would take longer.

Notes: When mounting an ISO in loop mode, you will not be able to write the the directory structure or modify files located there.  It is read only.

Comment or email me if you have any questions. kevin@colovirt.com

This will not get very detailed, but I figured I would share the following information.  In light of not being happy with the typical “each shelf has a 4 Gig interconnect” statement, I kept checking until there was a better answer.  So, anyone working with EMC SANs typically knows that every shelf is connected to each SP (Service Processor – 2 per SAN), daisy chained in a specific loop, and assigned a shelf id.  Next is the LCC.

Each DAE contains 2 LCC interfaces.  LCC is the acronymn for “Link Control Card”.  Typically one LCC goes to each Service Processor.  So that means that each DAE has two 4 gig links for a total of 8gb theoretical throughput.

This is where I will be reading more information.  Just because you have 2 LCC cards, each LUN can only be assigned to 1 Service Processor!  If my theory is correct, that means if all drives in one shelf is dedicated to a single LUN, and that LUN can only be active on one SP, does that mean that the actual throughput will be limited to 4gb instead of 8?  This being due to the other LCC loop being connected to the SP that is not assigned control of that LUN.

Any thoughts?  Feel free to comment!