unix - linux - storage

Lustre – cluster filesystem : quick setup guide

2009-11-08T09:09:00.000-08:00

The Lustre file system is a distributed high performance cluster filesytem that redefines I/O performance and scalability for large and complex computing environments. This is ideally suited for data-intensive applications which requires the high IO performance.

Lustre components

MDS – Metadata Server: The MDS server makes metadata stored in the metadata targets available to Lustre clients.

MDT – Metadata Target: This stores metadata, such as filenames, directories, permissions, and file layout, on the metadata server.

OSS – Object Storage Server: The OSS provides file I/O service, and network request handling for the OSTs. The MDT, OSTs and Lustre clients can run concurrently on a single node. However, a typical configuration is an MDT on a dedicated node, two or more OSTs on each OSS node, and a client on each of a large number of compute nodes.

OST – Object Storage Target: The OST stores data as data objects on one or more OSSs. A single Lustre file system can have multiple OSTs, each serving a subset of file data.

Client: The systems that mount the Lustre filesystem.

Steps to create Lustre FS

Configure Lustre Management Server (lustre-mgs.unixfoo.biz – Server 1)

Add the disk to volume manager
[root@lustre-mgs mnt]# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created

[root@lustre-mgs mnt]# pvs
PV VG Fmt Attr PSize Pfree
/dev/sdb lvm2 -- 136.73G 136.73G
Create lustre volume group
[root@lustre-mgs mnt]# vgcreate lustre /dev/sdb
Volume group "lustre" successfully created
[root@lustre-mgs mnt]# vgs
VG #PV #LV #SN Attr VSize Vfree
lustre 1 0 0 wz--n- 136.73G 136.73G
Create logical volume "MGS" (the management server)

[root@lustre-mgs ~]# lvcreate -L 25G -n MGS lustre
Create Lustre Management filesystem.
[root@lustre-mgs ~]# mkfs.lustre --mgs /dev/lustre/MGS

   Permanent disk data:
Target:     MGS
Index:      unassigned
Lustre FS: lustre
Mount type: ldiskfs
Flags:      0x74
              (MGS needs_index first_time update )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters:

checking for existing Lustre data: not found
device size = 10240MB
2 6 18
formatting backing filesystem ldiskfs on /dev/lustre/MGS
        target name MGS
        4k blocks     0
        options        -J size=400 -q -O dir_index,uninit_groups –F

mkfs_cmd = mkfs.ext2 -j -b 4096 -L MGS -J size=400 -q -O dir_index,uninit_groups -F /dev/lustre/MGS
Writing CONFIGS/mountdata
[root@lustre-mgs ~]#
Activate the lustre management filesystem using mount command.
[root@lustre-mgs ~]# mount -t lustre /dev/lustre/MGS /lustre/MGS/
[root@lustre-mgs ~]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2             54558908   5276572 46466144 11% /
/dev/sda1               497829     29006    443121   7% /boot
tmpfs                  8216000         0   8216000   0% /dev/shm
/dev/lustre/MGS       10321208    433052   9363868   5% /lustre/MGS
[root@lustre-mgs ~]#

Configure Lustre Metadata Server (In this guide, both the management & metadata server runs on the same host)

Create logical volume "MDT_unixfoo_cloud"

[root@lustre-mgs ~]# lvcreate -L 25G -n MDT_unixfoo_cloud lustre
Create Lustre Metdata filesystem for the filesystem “unixfoo_cloud”.

[root@lustre-mgs ~]# mkfs.lustre --fsname=unixfoo_cloud --mdt --reformat --mgsnode=lustre-mgs@tcp0 /dev/lustre/MDT_unixfoo_cloud
   Permanent disk data:
Target:     unixfoo_cloud-MDTffff
Index:      unassigned
Lustre FS: unixfoo_cloud
Mount type: ldiskfs
Flags:      0x71
              (MDT needs_index first_time update )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters: mgsnode=10.217.0.237@tcp mdt.group_upcall=/usr/sbin/l_getgroups
device size = 20480MB
2 6 18

formatting backing filesystem ldiskfs on /dev/lustre/MDT_unixfoo_cloud
        target name unixfoo_cloud-MDTffff
        4k blocks     0
        options        -J size=400 -i 4096 -I 512 -q -O dir_index,uninit_groups –F
mkfs_cmd = mkfs.ext2 -j -b 4096 -L unixfoo_cloud-MDTffff -J size=400 -i 4096 -I 512 -q -O dir_index,uninit_groups -F /dev/lustre/MDT_unixfoo_cloud
Writing CONFIGS/mountdata
[root@lustre-mgs ~]#
Activate the metdata filesystem using mount command.
[root@lustre-mgs ~]# mkdir /lustre/MDT_unixfoo_cloud
[root@lustre-mgs ~]# mount -t lustre /dev/lustre/MDT_unixfoo_cloud /lustre/MDT_unixfoo_cloud

Configure OSTs ( servers: oss1, oss2 .. )

Add /dev/md0 to volume manager

[root@oss1 ~]# pvcreate /dev/md0
Create volume group "lustre"

[root@oss1 ~]# vgcreate lustre /dev/md0
Create logical volume (ost) for unixfoo_cloud

[root@oss1 ~]# lvcreate -n OST_unixfoo_cloud_1 -L 100G lustre
Create OST lustre filesystem
[root@oss1 ~]# mkfs.lustre --fsname=unixfoo_cloud --ost --mgsnode=lustre-mgs@tcp0 /dev/lustre/OST_unixfoo_cloud_1
mkfs_cmd = mkfs.ext2 -j -b 4096 -L unixfoo_cloud-OSTffff -J size=400 -i 16384 -I 256 -q -O dir_index,uninit_groups -F /dev/lustre/OST_unixfoo_cloud_1
Writing CONFIGS/mountdata
[root@oss1 ~]#
Activate the OST by using mount command
[root@oss1 ~]# mkdir -p /lustre/unixfoo_cloud_oss1
[root@oss1 ~]# mount -t lustre /dev/lustre/OST_unixfoo_cloud_1 /lustre/unixfoo_cloud_oss1

Mount on the client:

Mount the lustre filesystem unixfoo_cloud

[root@lustreclient1 ~]# mount -t lustre lustre-mgs@tcp0:/unixfoo_cloud /mnt
[root@lustreclient1 ~]# df –h
Filesystem            Size Used Avail Use% Mounted on
/dev/sda2              52G 5.1G   44G 11% /
/dev/sda1             487M   29M 433M   7% /boot
tmpfs                 7.9G     0 7.9G   0% /dev/shm
lustre-mgs@tcp0:/unixfoo_cloud
                       99G 461M   93G   1% /mnt
[root@lustreclient1 ~]#
Done.

Netapp Deduplication – quick setup guide

2009-11-02T07:39:00.000-08:00

Deduplication refers to the elimination of redundant data in the storage. In the deduplication process, duplicate data is deleted, leaving only one copy of the data to be stored. However, indexing of all data is still retained should that data ever be required. Deduplication is able to reduce the required storage capacity since only the unique data is stored. Netapp supports deduplication where only unique blocks in the flex volume is stored and it creates a small amount of additional metadata in the dedup process. The NetApp deduplication technology allows duplicate 4KB blocks anywhere in the flexible volume to be deleted and stores a unique one.

The core enabling technology of deduplication is fingerprints. These are unique digital signatures for every 4KB data block in the flexible volume. When deduplication runs for the first time on a flexible volume with existing data, it scans the blocks in the flexible volume and creates a fingerprint database, which contains a sorted list of all fingerprints for used blocks in the flexible volume. After the fingerprint file is created, fingerprints are checked for duplicates and if found, first a byte-by-byte comparison of the blocks is done to make sure that the blocks are indeed identical. If they are found to be identical, the block’s pointer is updated to the already existing data block and the duplicate data block is released and inode is updated.

Netapp Deduplication commands:

Enable dedup (asis) license.

fractal-design> sis on /vol/demovol
If you have a new flex volume which was just created, follow this step to enable ASIS deduplication

fractal-design> sis on /vol/demovol
Deduplication for "/vol/demovol" is enabled.
Already existing data could be processed by running "sis start -s /vol/demovol”
If you have already existing flex volume with data in it, follow this step.

fractal-design> sis start -s /vol/demovol
Checking the status of deduplication.
fractal-design> vol status demovol
Volume          State   Status          Options
VolArchive      online raid_dp, flex   nosnap=on
                        sis
Containing aggregate: 'aggr0'
fractal-design>

fractal-design> sis status /vol/demovol
Path            State   Status      Progress
/vol/demovol    Enabled Idle        Idle for 00:02:12
fractal-design>
Check the storage space saved due to deduplication
fractal-design> df -s /vol/demovol
Filesystem used saved %saved
/vol/demovol/ 9316052 0 0%
fractal-design>
If you have to run deduplication at a later point of time on this volume, just do a “sis start /vol/demovol”.
The sis can be scheduled using “sis config” command.
Done.

More netapp blog posts at : http://unixfoo.blogspot.com/search/label/netapp

Netapp Snapmirror Setup Guide

2009-10-23T22:41:00.000-07:00

Snapmirror is an licensed utility in Netapp to do data transfer across filers. Snapmirror works at Volume level or Qtree level. Snapmirror is mainly used for disaster recovery and replication.

Snapmirrror needs a source and destination filer. (When source and destination are the same filer, the snapmirror happens on local filer itself. This is when you have to replicate volumes inside a filer. If you need DR capabilities of a volume inside a filer, you have to try syncmirror ).

Synchronous SnapMirror is a SnapMirror feature in which the data on one system is replicated on another system at, or near, the same time it is written to the first system. Synchronous SnapMirror synchronously replicates data between single or clustered storage systems situated at remote sites using either an IP or a Fibre Channel connection. Before Data ONTAP saves data to disk, it collects written data in NVRAM. Then, at a point in time called a consistency point, it sends the data to disk. When the Synchronous SnapMirror feature is enabled, the source system forwards data to the destination system as it is written in NVRAM. Then, at the consistency point, the source system sends its data to disk and tells the destination system to also send its data to disk.

This guides you quickly through the Snapmirror setup and commands.

1) Enable Snapmirror on source and destination filer

source-filer> options snapmirror.enable
snapmirror.enable on
source-filer>
source-filer> options snapmirror.access
snapmirror.access legacy
source-filer>

2) Snapmirror Access

Make sure destination filer has snapmirror access to the source filer. The snapmirror filer's name or IP address should be in /etc/snapmirror.allow. Use wrfile to add entries to /etc/snapmirror.allow.

source-filer> rdfile /etc/snapmirror.allow
destination-filer
destination-filer2
source-filer>

3) Initializing a Snapmirror relation

Volume snapmirror : Create a destination volume on destination netapp filer, of same size as source volume or greater size. For volume snapmirror, the destination volume should be in restricted mode. For example, let us consider we are snapmirroring a 100G volume - we create the destination volume and make it restricted.

destination-filer> vol create demo_destination aggr01 100G
destination-filer> vol restrict demo_destination

Volume SnapMirror creates a Snapshot copy before performing the initial transfer. This copy is referred to as the baseline Snapshot copy. After performing an initial transfer of all data in the volume, VSM (Volume SnapMirror) sends to the destination only the blocks that have changed since the last successful replication. When SnapMirror performs an update transfer, it creates another new Snapshot copy and compares the changed blocks. These changed blocks are sent as part of the update transfer.

Snapmirror is always destination filer driven. So the snapmirror initialize has to be done on destination filer. The below command starts the baseline transfer.

destination-filer> snapmirror initialize -S source-filer:demo_source destination-filer:demo_destination
Transfer started.
Monitor progress with 'snapmirror status' or the snapmirror log.
destination-filer>

Qtree Snapmirror : For qtree snapmirror, you should not create the destination qtree. The snapmirror command automatically creates the destination qtree. So just volume creation of required size is good enough.

Qtree SnapMirror determines changed data by first looking through the inode file for inodes that have changed and changed inodes of the interesting qtree for changed data blocks. The SnapMirror software then transfers only the new or changed data blocks from this Snapshot copy that is associated with the designated qtree. On the destination volume, a new Snapshot copy is then created that contains a complete point-in-time copy of the entire destination volume, but that is associated specifically with the particular qtree that has been replicated.

destination-filer> snapmirror initialize -S source-filer:/vol/demo1/qtree destination-filer:/vol/demo1/qtree
Transfer started.
Monitor progress with 'snapmirror status' or the snapmirror log.

4) Monitoring the status : Snapmirror data transfer status can be monitored either from source or destination filer. Use "snapmirror status" to check the status.

destination-filer> snapmirror status
Snapmirror is on.
Source                          Destination                          State          Lag Status
source-filer:demo_source        destination-filer:demo_destination   Uninitialized -   Transferring (1690 MB done)
source-filer:/vol/demo1/qtree   destination-filer:/vol/demo1/qtree   Uninitialized -   Transferring (32 MB done)
destination-filer>

5) Snapmirror schedule : This is the schedule used by the destination filer for updating the mirror. It informs the SnapMirror scheduler when transfers will be initiated. The schedule field can either contain the word sync to specify synchronous mirroring or a cron-style specification of when to update the mirror. The cronstyle schedule contains four space-separated fields.

If you want to sync the data on a scheduled frequency, you can set that in destination filer's /etc/snapmirror.conf . The time settings are similar to Unix cron. You can set a synchronous snapmirror schedule in /etc/snapmirror.conf by adding “sync” instead of the cron style frequency.

destination-filer> rdfile /etc/snapmirror.conf
source-filer:demo_source destination-filer:demo_destination - 0 * * * # This syncs every hour
source-filer:/vol/demo1/qtree destination-filer:/vol/demo1/qtree - 0 21 * * # This syncs every 9:00 pm
destination-filer>

6) Other Snapmirror commands

To break snapmirror relation - do snapmirror quiesce and snapmirror break.
To update snapmirror data - do snapmirror update
To resync a broken relation - do snapmirror resync.
To abort a relation - do snapmirror abort

Snapmirror do provide multipath support. More than one physical path between a source and a destination system might be desired for a mirror relationship. Multipath support allows SnapMirror traffic to be load balanced between these paths and provides for failover in the event of a network outage.

ZFS : Basic administration guide

2009-10-23T07:33:00.000-07:00

ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs.

Watch this video about ZFS overview and demo. A good one..

This ZFS guide provides an overview of ZFS and its administration commands that will be helpful for beginners.

ZFS Pool:

ZFS organizes physical devices into logical pools called storage pools. Both individual disks and array logical unit numbers (LUNs) that are visible to the operating system can be included in a ZFS pools. These pools can be created as disks striped together with no redundancy (RAID 0), mirrored disks (RAID 1), striped mirror sets (RAID 1 + 0), or striped with parity (RAID Z). Additional disks can be added to pools at any time but they must be added with the same RAID level.

ZFS Filesystem :

ZFS offers a POSIX-compliant file system interface to the Solaris/OpenSolaris operating system. ZFS file systems must be built in one and only one storage pool, but a storage pool may have more than one defined file system. ZFS file systems are managed & mounted through /etc/vfstab file. The common way to mount a ZFS file system is to simply define it against a pool. All defined ZFS file systems automatically mount at boot time unless otherwise configured.

Here are the basic commands for getting started with ZFS.

Creating Storage pool using "zpool create" :

bash-3.00# zpool create demovol raidz c2t1d0 c2t2d0
bash-3.00# zpool status
pool: demovol
state: ONLINE
scrub: none requested
config:

        NAME         STATE     READ WRITE CKSUM
        demovol      ONLINE       0     0     0
          raidz1     ONLINE       0     0     0
            c2t1d0   ONLINE       0     0     0
            c2t2d0   ONLINE       0     0     0

errors: No known data errors
bash-3.00#

"zfs list" will give the details of the pool and other zfs filesytems.

bash-3.00# zfs list
NAME                   USED AVAIL REFER MOUNTPOINT
demovol               1.00G 900G 38.1K /demovol
bash-3.00#

Creating File Systems : "zfs create" is used to create zfs filesytem.

bash-3.00# zfs create demovol/testing
bash-3.00# zfs list
NAME                   USED AVAIL REFER MOUNTPOINT
demovol               1.00G 900G 38.1K /demovol
demovol/testing       32.6K 900G 32.6K /demovol/testing
bash-3.00#

bash-3.00# ls /dev/zvol/dsk/demovol -- This should show you the disk file.

Setting Quota for the filesytem : Until Quota is set, the filesytem shows the total available space of the containter zfs pool.

bash-3.00# zfs set quota=10G emspool3/testing
bash-3.00# zfs list
NAME                     USED AVAIL REFER MOUNTPOINT
demovol                 1.00G 900G   39.9K /demovol
demovol/testing         32.6K 10.0G 32.6K /demovol/testing

Creating a snapshot :

bash-3.00# zfs snapshot demovol/testing@snap21
bash-3.00# zfs list
NAME                     USED AVAIL REFER MOUNTPOINT
demovol                 1.00G 900G   39.9K /demovol
demovol/testing         32.6K 10.0G 32.6K /demovol/testing
demovol/testing@snap21      0      - 32.6K -
bash-3.00#

Get all properties of a ZFS filesytem :

bash-3.00# zfs get all demovol/testing
NAME             PROPERTY         VALUE                  SOURCE
demovol/testing type             filesystem             -
demovol/testing creation         Mon Feb 9 9:05 2009 -
demovol/testing used             32.6K                  -
demovol/testing available        10.0G                  -
demovol/testing referenced       32.6K                  -
demovol/testing compressratio    1.00x                  -
demovol/testing mounted          yes                    -
demovol/testing quota            10G                    local
demovol/testing reservation      none                   default
demovol/testing recordsize       128K                   default
demovol/testing mountpoint       /demovol/testing       default
..

Cloning a ZFS filesystem from a snapshot :

bash-3.00# zfs clone demovol/testing@snap21 demovol/clone22
bash-3.00# zfs list
NAME                     USED AVAIL REFER MOUNTPOINT
demovol                 1.00G 900G   39.9K /demovol
demovol/clone22             0 900G   32.6K /demovol/clone22
demovol/testing         32.6K 10.0G 32.6K /demovol/testing
demovol/testing@snap21      0      - 32.6K -
bash-3.00#

Performance IO Monitoring the ZFS storage pool:

bash-3.00# zpool iostat 1
               capacity     operations    bandwidth
pool         used avail   read write   read write
---------- ----- ----- ----- ----- ----- -----
demovol     4.95M 900G      0      0      0     35
demovol     4.95M 900G      0      0      0      0
demovol     4.95M 900G      0      0      0      0
demovol     4.95M 900G      0      0      0      0

Please refer to the man pages, zfs and zpool, for more detailed information. Additional documentation may be found at docs.sun.com and OpenSolaris ZFS Community.

Dstat : Linux Performance analysis tool

2009-10-20T10:25:00.000-07:00

Dstat is a handy utility for monitoring systems during performance tuning tests, benchmarks or troubleshooting. It combines vmstat, iostat, ifstat, netstat information and more.

Dstat overcomes some of their limitations and adds some extra features, more counters and flexibility. One more great feature of Dstat is that it is written in python, modular & easy to extend, add your own counters as plugins. It also allows to export CSV output, which can be imported in Gnumeric and Excel to make graphs.

Combines vmstat, iostat, ifstat, netstat information and more
Shows stats in exactly the same timeframe
Enable/order counters as they make most sense during analysis/troubleshooting
Modular design
Written in python so easily extendable for the task at hand
Easy to extend, add your own counters (please contribute those)
Includes about 10 external plugins to show how easy it is to add counters
Can summarize grouped block/network devices and give total numbers
Can show interrupts per device
Very accurate timeframes, no timeshifts when system is stressed
Shows exact units and limits conversion mistakes
Indicate different units with different colors
Show intermediate results when delay > 1

You can download Dstat at : http://dag.wieers.com/home-made/dstat

Linux Network bonding

2009-10-13T04:34:00.000-07:00

Linux network Bonding or teaming or trunking is creation of a single bonded interface by combining 2 or more ethernet interfaces. This helps in high availability of your network interface and offers performance improvement.

Steps for bonding in Oracle Enterprise Linux and Redhat Enterprise Linux are as follows..

Step 1.

Create the file ifcfg-bond0 with the IP address, netmask and gateway. Shown below is my test bonding config file.

$ cat /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
IPADDR=192.168. 1.12
NETMASK=255. 255.255.0
GATEWAY=192. 168.1.1
USERCTL=no
BOOTPROTO=none
ONBOOT=yes

Step 2.

Modify eth0, eth1 and eth2 configuration as shown below. Comment out, or remove the ip address, netmask, gateway and hardware address from each one of these files, since settings should only come from the ifcfg-bond0 file above. Make sure you add the MASTER and SLAVE configuration in these files.

$ cat /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
BOOTPROTO=none
#HWADDR=00:12:17:5C:A7:9D
#IPADDR=192.168.1.12
#NETMASK=255.255.255.0
#TYPE=Ethernet
#GATEWAY=192.168.1.1
#USERCTL=no
#IPV6INIT=no
#PEERDNS=yes
ONBOOT=yes
# Settings for Bond
MASTER=bond0
SLAVE=yes

$ cat /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1
BOOTPROTO=none
#HWADDR=01:12:18:5C:A7:D9
#IPADDR=192.168.1.13
#NETMASK=255.255.255.0
ONBOOT=yes
#TYPE=Ethernet
USERCTL=no
#IPV6INIT=no
#PEERDNS=yes
# Settings for bonding
MASTER=bond0
SLAVE=yes

$ cat /etc/sysconfig/network-scripts/ifcfg-eth2

DEVICE=eth2
BOOTPROTO=none
#HWADDR=00:12:27:5C:A7:9D
#IPADDR=192.168.1.12
#NETMASK=255.255.255.0
ONBOOT=yes
#TYPE=Ethernet
#GATEWAY=192.168.1.1
#USERCTL=no
#IPV6INIT=no
#PEERDNS=yes
MASTER=bond0
SLAVE=yes

Step 3.

Set the parameters for bond0 bonding kernel module. Add the following lines to /etc/modprobe.conf

# bonding commands
alias bond0 bonding
options bond0 mode=1 miimon=100

Step 4.

Load the bond driver module from the command prompt.

$ modprobe bonding

Step 5.

Restart the network, or restart the computer.

$ service network restart # Or restart computer

When the machine boots up check the proc settings.

$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.0.2 (March 23, 2006)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eth2
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth2
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:13:72:80: 62:f0

Look at ifconfig -a and check that your bond0 interface is active. You are done!. For more details on the different modes of bonding, please refer to unixfoo’s modes of bonding.

Solaris LDOMs virtualization : setup guide

2009-10-11T10:33:00.000-07:00

Sun Logical Domains or LDoms is a full virtual machine that runs an independent operating system instance and contains virtualized CPU, memory, storage, console, and cryptographic devices. This technology allows you to allocate a system resources into logical groupings and create multiple, discrete systems, each with their own operating system, resources, and identity within a single computer system. We can run a variety of applications software in different logical domains and keep them independent of performance and security purposes. The LDoms environment can help to achieve greater resource usage, better scaling, and increased security and isolation.

Logical Domains

View more documents from Jarod Wang.

Logical & Control domain : The control domain communicates with the hypervisor to create and manage all logical domain configurations within a server platform. The Logical Domains Manager is used to create and manage logical domains. The Logical Domains Manager maps logical domains to physical resources. Without access to the Logical Domains Manager all logical domain resource levels remain static. The initial domain created when installing Logical Domains software is a control domain and is named primary.

Image from : http://www.sun.com/blueprints/0207/820-0832.pdf

You can download Logical Domain manager from http://sun.com/ldoms . Please read the release notes for system firmware requirements and patch requirements. By default, Ldoms software gets installed to /opt/SUNWldm/. Make sure the below commands works - and that confirms Logical domain manager is running.

solfoo23# /opt/SUNWldm/bin/ldm list
Name State Flags Cons VCPU Memory Util Uptime
primary active -t-cv SP 32 16128M 49% 90mm

Creating default services : You need to create the default virtual services that the control domain uses to provide disk services, console access and networking. The below commands explains them.

Create Virtual Disk server(vds) : Virtual disk server helps importing virtual disks into a logical domain from the control domain.

solfoo23# ldm add-vds primary-vds0 primary
Create Virtual Console concentrator Server(vcc) : Virtual Console concentrator server provides terminal service to logical domain consoles.

solfoo23# ldm add-vcc port-range=5000-5100 primary-vcc0 primary
Create Virtual Switch server(vsw) : Virtual Switch server enables networking between virtual network devices in logical domains.

solfoo23# ldm add-vsw net-dev=e1000g0 primary-vsw0 primary
List the default services created

solfoo23# ldm list-services primary
VDS
NAME VOLUME OPTIONS DEVICE
primary-vds0

VCC
NAME PORT-RANGE
primary-vcc0 5000-5100

VSW
NAME MAC NET-DEV DEVICE MODE
primary-vsw0 00:11:5a:12:dc:fc e1000g1 switch@0 prog,promisc

Control Domain Creation : The next step is to perform the initial setup of the primary domain, which will act as the control domain. You should specify the resources that the primary domain will use and what will be released for use by other guest domains. In this document, we are creating the control domain with 2 cpu's and 1gb RAM.

solfoo23# ldm set-mau 0 primary
solfoo23# ldm set-vcpu 2 primary
solfoo23# ldm set-memory 1024M primary

Now, set these modified configuration permanent using list-spconfig option.

solfoo23# ldm list-spconfig
factory-default [current]

solfoo23# ldm add-spconfig initial

solfoo23# ldm list-spconfig
factory-default [current]
initial [next]

Reboot the server and it will come up with initial configuration.

Networking between domains : Networking between control, service and other domains is disabled by default. To enable this, the virtual switch device should be configured as a network device. On the server console and perform the following network configuration steps.

Plumb the virtual switch(vsw0)

solfoo23# ifconfig vsw0 plumb
Bring down the primary interface

solfoo23# ifconfig e1000g1 down unplumb
Configure Virtual switch with the primary interface details

solfoo23# ifconfig vsw0 <ip> netmask <netmask> broadcast + up
Modify the hostname file to make this configuration permanent

solfoo23# mv /etc/hostname.e1000g1 /etc/hostname.vsw0
Enable Virtual Network terminal server daemon

solfoo23# svcadm enable vntsd

Now the setup is done. Run "ldm list-bindings primary" and make sure they are ok.

Logical Domain Creation : Now that the system is ready, prepare and plan for the logical domain configuration. In this document, we are creating a logical domain with 2 CPUs and 1GB memory and "domfoo" is the name.

solfoo23# ldm add-domain domfoo
solfoo23# ldm add-vcpu 2 domfoo
solfoo23# ldm add-memory 1G domfoo
solfoo23# ldm add-vnet vnet1 primary-vsw0 domfoo
solfoo23# ldm add-vdsdev /dev/dsk/c1t2d0s2 vol1@primary-vds0
solfoo23# ldm add-vdisk vdisk1 vol1@primary-vds0 domfoo
solfoo23# ldm bind domfoo
solfoo23# ldm set-var auto-boot\?=false domfoo
solfoo23# ldm start-domain domfoo

You will be able see the domain using "ldm list-domain"

solfoo23# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv SP 2 2G 0.2% 3h 4m
domfoo inactive ----- 2 1G

Connect to the logical domain console by telneting to the virtual console port.

solfoo23# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost....
Escape character is ’^]’.
Connecting to console "domfoo" in group "domfoo" ....
Press ~? for control options ..
{0} ok

Your LDom is up!. You can install it using jumpstart. Your LDoms environment is ready!

Refer Solaris LDOM virtualization document links for more information :

Overview : http://www.sun.com/servers/coolthreads/ldoms/wp.pdf
LDOMs Beginners guide : http://www.sun.com/blueprints/0207/820-0832.pdf
LDOMs Demo : http://blogs.sun.com/ash/resource/flashdemos/64-ldoms-on-t2.html
LDOMs presentation : http://www.snpnet.com/customer_pub/sun/isv_LDOM/
Admin guide : http://docs-pdf.sun.com/819-6428-11/819-6428-11.pdf

Pictorial description of DAS NAS & SAN

2009-10-01T09:53:00.000-07:00

DAS:

Direct-attached storage, or DAS, is the most basic level of storage which most people are familiar with, in which storage devices are part of the host computer, as with drives, or directly connected to a single server, as with RAID arrays or tape libraries. Network workstations must therefore access the server in order to connect to the storage device. DAS is ideal for localized file sharing in environments with a single server or a few servers - for example, small businesses or departments and workgroups that do not need to share information over long distances or across an enterprise. From an economical perspective, the initial investment in direct-attached storage is cheaper.

NAS:

Networked storage was developed to address the challenges inherent in a server- based infrastructure such as direct-attached storage. Network-attached storage, or NAS, is a special purpose device, comprised of both hard disks and management software, which is 100% dedicated to serving files over a network.

SAN :

A storage area network (SAN) is an architecture to attach remote computer storage devices (such as disk arrays, tape libraries, and optical jukeboxes) to servers in such a way that the devices appear as locally attached to the operating system. Although the cost and complexity of SANs are dropping, they are uncommon outside larger enterprises. With their high degree of sophistication, management complexity and cost, SANs are traditionally implemented for mission-critical applications in the enterprise space. In a SAN infrastructure, storage devices such as NAS, DAS, RAID arrays or tape libraries are connected to servers using Fibre Channel. Fibre Channel is a highly reliable, gigabit interconnect technology that enables simultaneous communication among workstations, mainframes, servers, data storage systems and other peripherals. Without the distance and bandwidth limitations of SCSI, Fibre Channel is ideal for moving large volumes of data across long distances quickly and reliably.

Xen : Bridging on bonded & trunked interfaces

2009-09-30T10:35:00.000-07:00

When you deploy some critical stuffs like Oracle DB, Oracle RAC DB etc on the Xen or Oracle virtual machine, you'll have to use bonded network interfaces and the vlan trunks. By default, Xen and Oracle VM doesn't support the xen bridges created on bonded-vlaned-interfaces. The below document gives you a clear idea on how to create and configure xen bridges on bonded and vlan trunked interfaces.

Xen wiki document on bonded-vlaned xen network configuration
Oracle VM server configuration for bonded & trunked interfaces

Netapp Snapvault guide

2009-08-26T21:23:00.000-07:00

Netapp SnapVault is a heterogeneous disk-to-disk backup solution for Netapp filers and heterogeneous OS systems (Windows, Linux , Solaris, HPUX and AIX). Basically, Snapvault uses netapp snapshot technology to take point-in-time snapshot and store them as online backups. In event of data loss or corruption on a filer, the backup data can be restored from the SnapVault filer with less downtime. It has significant advantages over traditional tape backups, like

Reduce backup windows versus traditional tape-based backup
Media cost savings
No backup/recovery failures due to media errors
Simple and Fast recovery of corrupted or destroyed data

Snapvault consists of major two entities – snapvault clients and a snapvault storage server. A snapvault client (Netapp filers and unix/windows servers) is the system whose data should be backed-up. The SnapVault server is a Netapp filer – which gets the data from clients and backs up data. For Server to Netapp Snapvault, we need to install Open System Snapvault client software provided by Netapp, on the servers. Using the snapvault agent software, the Snapvault server can pull and backup data on to the backup qtrees. SnapVault protects data on a client system by maintaining a number of read-only versions (snapshots) of that data on a SnapVault filer. The replicated data on the snapvault server system can be accessed via NFS or CIFS. The client systems can restore entire directories or single files directly from the snapvault filer. Snapvault requires primary and secondary license.

How snapvault works?

When snapvault is setup, initially a complete copy of the data set is pulled across the network to the SnapVault filer. This initial or baseline, transfer may take some time to complete, because it is duplicating the entire source data set on the server – much like a level-zero backup to tape. Each subsequent backup transfers only the data blocks that has changed since the previous backup. When the initial full backup is performed, the SnapVault filer stores the data on a qtree and creates a snapshot image of the volume for the data that is to be backed up. SnapVault creates a new Snapshot copy with every transfer, and allows retention of a large number of copies according to a schedule configured by the backup administrator. Each copy consumes an amount of disk space proportional to the differences between it and the previous copy.

Snapvault commands :

Initial step to setup Snapvault backup between filers is to install snapvault license and enable snapvault on all the source and destination filers.

Source filer – filer1

    filer1> license add XXXXX
    filer1> options snapvault.enable on
    filer1> options snapvault.access host=filer2

Destination filer – filer2

    filer2> license add XXXXX
    filer2> options snapvault.enable on
    filer2> options snapvault.access host=filer1

Consider filer2:/vol/snapvault_volume as the snapvault destination volume, where all backups are done. The source data is filer1:/vol/datasource/qtree1. As we have to manage all the backups on the destination filer (filer2) using snapvault – manually disable scheduled snapshots on the destination volumes. The snapshots will be managed by Snapvault. Disabling Netapp scheduled snapshots, with below command.

    filer2> snap sched snapvault_volume 0 0 0

Creating Initial backup: Initiate the initial baseline data transfer (the first full backup) of the data from source to destination before scheduling snapvault backups. On the destination filer execute the below commands to initiate the base-line transfer. The time taken to complete depends upon the size of data on the source qtree and the network bandwidth. Check “snapvault status” on source/destination filers for monitoring the base-line transfer progress.

    filer2> snapvault start -S filer1:/vol/datasource/qtree1 filer2:/vol/snapvault_volume/qtree1

Creating backup schedules: Once the initial base-line transfer is completed, snapvault schedules have to be created for incremental backups. The retention period of the backup depends on the schedule created. The snapshot name should be prefixed with “sv_”. The schedule is in the form of “[@][@]”.

On source filer:

For example, let us create the schedules on source as below - 2 hourly, 2 daily and 2 weekly snapvault . These snapshot copies on the source enables administrators to recover directly from source filer without accessing any copies on the destination. This enables more rapid restores. However, it is not necessary to retain a large number of copies on the primary; higher retention levels are configured on the secondary. The commands below shows how to create hourly, daily & weekly snapvault snapshots.

    filer1> snapvault snap sched datasource sv_hourly 2@0-22
    filer1> snapvault snap sched datasource sv_daily 2@23
    filer1> snapvault snap sched datasource sv_weekly 2@21@sun

On snapvault filer:

Based on the retention period of the backups you need, the snapvault schedules on the destination should be done. Here, the sv_hourly schedule checks all source qtrees once per hour for a new snapshot copy called sv_hourly.0. If it finds such a copy, it updates the SnapVault qtrees with new data from the primary and then takes a Snapshot copy on the destination volume, called sv_hourly.0. If you don’t use the -x option, the secondary does not contact the primary and transfer the Snapshot copy. It just creates a snapshot copy of the destination volume.

    filer2> snapvault snap sched -x snapvault_volume sv_hourly 6@0-22
    filer2> snapvault snap sched -x snapvault_volume sv_daily 14@23@sun-fri
    filer2> snapvault snap sched -x snapvault_volume sv_weekly 6@23@sun

To check the snapvault status, use the command "snapvault status" either on source or destination filer. And to see the backups, do a "snap list" on the destination volume - that will give you all the backup copies, time of creation etc.

Restoring data : Restoring data is as simple as that, you have to mount the snapvault destination volume through NFS or CIFS and copy the required data from the backup snapshot.

You can also try Netapp Protection manager to manage the snapvault backups either from OSSV or from Netapp primary storage. Protection manager is based on Netapp Operations manager (aka Netapp DFM). It is a client based UI, with which you connect to the Ops Manager and protect your storages.

Vmware - Netapp cloning : Rapid VM provisioning

2009-07-10T05:59:00.000-07:00

Checkout this demo of provisioning 5000 Vmware VMs using Netapp File cloning and volume flexclone technologies. Good one ..

Pocket guide for netapp commands

2009-07-05T00:09:00.001-07:00

This post contains the list of commands that will be most used and will come handy when managing or monitoring or troubleshooting a Netapp filer.

sysconfig -a : shows hardware configuration with more verbose information
sysconfig -d : shows information of the disk attached to the filer
version : shows the netapp Ontap OS version.
uptime : shows the filer uptime
dns info : this shows the dns resolvers, the no of hits and misses and other info
nis info : this shows the nis domain name, yp servers etc.
rdfile : Like "cat" in Linux, used to read contents of text files/
wrfile : Creates/Overwrites a file. Similar to "cat > filename" in Linux
aggr status : Shows the aggregate status
aggr status -r : Shows the raid configuration, reconstruction information of the disks in filer
aggr show_space : Shows the disk usage of the aggreate, WAFL reserve, overheads etc.
vol status : Shows the volume information
vol status -s : Displays the spare disks on the filer
vol status -f : Displays the failed disks on the filer
vol status -r : Shows the raid configuration, reconstruction information of the disks
df -h : Displays volume disk usage
df -i : Shows the inode counts of all the volumes
df -Ah : Shows "df" information of the aggregate
license : Displays/add/removes license on a netapp filer
maxfiles : Displays and adds more inodes to a volume
aggr create : Creates aggregate
vol create <volname> <aggrname> <size> : Creates volume in an aggregate
vol offline <volname> : Offlines a volume
vol online <volname> : Onlines a volume
vol destroy <volname> : Destroys and removes an volume
vol size <volname> [+|-]<size> : Resize a volume in netapp filer
vol options : Displays/Changes volume options in a netapp filer
qtree create <qtree-path> : Creates qtree
qtree status : Displays the status of qtrees
quota on : Enables quota on a netapp filer
quota off : Disables quota
quota resize : Resizes quota
quota report : Reports the quota and usage
snap list : Displays all snapshots on a volume
snap create <volname> <snapname> : Create snapshot
snap sched <volname> <schedule> : Schedule snapshot creation
snap reserve <volname> <percentage> : Display/set snapshot reserve space in volume
/etc/exports : File that manages the NFS exports
rdfile /etc/exports : Read the NFS exports file
wrfile /etc/exports : Write to NFS exports file
exportfs -a : Exports all the filesystems listed in /etc/exports
cifs setup : Setup cifs
cifs shares : Create/displays cifs shares
cifs access : Changes access of cifs shares
lun create : Creates iscsi or fcp luns on a netapp filer
lun map : Maps lun to an igroup
lun show : Show all the luns on a filer
igroup create : Creates netapp igroup
lun stats : Show lun I/O statistics
disk show : Shows all the disk on the filer
disk zero spares : Zeros the spare disks
disk_fw_update : Upgrades the disk firmware on all disks
options : Display/Set options on netapp filer
options nfs : Display/Set NFS options
options timed : Display/Set NTP options on netapp.
options autosupport : Display/Set autosupport options
options cifs : Display/Set cifs options
options tcp : Display/Set TCP options
options net : Display/Set network options
ndmpcopy <src-path> <dst-path> : Initiates ndmpcopy
ndmpd status : Displays status of ndmpd
ndmpd killall : Terminates all the ndmpd processes.
ifconfig : Displays/Sets IP address on a network/vif interface
vif create : Creates a VIF (bonding/trunking/teaming)
vif status : Displays status of a vif
netstat : Displays network statistics
sysstat -us 1 : begins a 1 second sample of the filer's current utilization (crtl - c to end)
nfsstat : Shows nfs statistics
nfsstat -l : Displays nfs stats per client
nfs_hist : Displays nfs historgram
statit : beings/ends a performance workload sampling [-b starts / -e ends]
stats : Displays stats for every counter on netapp. Read stats man page for more info
ifstat : Displays Network interface stats
qtree stats : displays I/O stats of qtree
environment : display environment status on shelves and chassis of the filer
storage show <disk|shelf|adapter> : Shows storage component details
snapmirror intialize : Initialize a snapmirror relation
snapmirror update : Manually Update snapmirror relation
snapmirror resync : Resyns a broken snapmirror
snapmirror quiesce : Quiesces a snapmirror bond
snapmirror break : Breakes a snapmirror relation
snapmirror abort : Abort a running snapmirror
snapmirror status : Shows snapmirror status
lock status -h : Displays locks held by filer
sm_mon : Manage the locks
storage download shelf : Installs the shelf firmware
software get : Download the Netapp OS software
software install : Installs OS
download : Updates the installed OS
cf status : Displays cluster status
cf takeover : Takes over the cluster partner
cf giveback : Gives back control to the cluster partner
reboot : Reboots a filer

If you are not aware of the complete details of these commands and need more information on these commands, refer the Netapp Data Ontap administration manual from now site.

More netapp blog posts at : http://unixfoo.blogspot.com/search/label/netapp

Configuring Persistent static route in Linux

2009-05-04T09:13:00.000-07:00

Static routes will be added usually through "route add" command. The drawback of 'route' command is that, when Linux reboots it will forget static routes. But to make it persistent across reboots, you have to add it to /etc/sysconfig/network-scripts/route-<eth> .

To add static route using "route add":

# route add -net 192.168.100.0 netmask 255.255.255.0 gw 192.168.10.1 dev eth0

Adding Persistent static route:

You need to edit /etc/sysconfig/network-scripts/route-eth0 file to define static routes for eth0 interface.

GATEWAY0=192.168.10.1
NETMASK0=255.255.255.0
ADDRESS0=192.168.100.0

GATEWAY1=10.64.34.1
NETMASK1= 255.255.255.240
ADDRESS1=10.64.34.10

Save and close the file. Restart networking:

# service network restart

Verify new routing table:

# route –n

# netstat –nr

Xen : Adding and removing NICs on virtual machines

2009-04-15T07:33:00.000-07:00

Virtual Network interfaces can be attached or detached from a guest virtual machine (domU) while the virtual machine is up and running. The below Xen networking steps gives details on those.

“xm network-list” is the command to display the virtual network interfaces attached to a the guest Xen virtual machine. In this example, the command lists 3 NICs for the virtual machine “vm455”. We’ll try to remove and add back the third vif.

[root@host203 ~]# xm network-list vm455
Idx BE MAC Addr. handle state evt-ch tx-/rx-ring-ref BE-path
0 0 00:16:3E:47:AA:FD 0 4 12 2304 /2305 /local/domain/0/backend/vif/1/0
1 0 00:16:3E:22:33:44 1 4 13 2306 /2307 /local/domain/0/backend/vif/1/1
2 0 00:16:3E:40:22:12 2 4 14 2308 /2309 /local/domain/0/backend/vif/1/2
[root@host203 ~]#

To remove a NIC card from the virtual machine, use “xm network-detach”, with the corresponding options. In this example, the interface 3 is removed from the virtual machine. This command has to be executed on the Xen server.

[root@host203 ~]# xm network-detach vm455 2
[root@host203 ~]# xm network-list vm455
Idx BE MAC Addr. handle state evt-ch tx-/rx-ring-ref BE-path
0 0 00:16:3E:47:AA:FD 0 4 12 2304 /2305 /local/domain/0/backend/vif/1/0
1 0 00:16:3E:22:33:44 1 4 13 2306 /2307 /local/domain/0/backend/vif/1/1
[root@host203 ~]#

Once the network is detached, you can only 2 virtual NICs using “xm network-list” on the Xen server. If you check the virtual machine, it will have only 2 NICS. Use ifconfig to check on the guest virtual machine.

To add a NIC to the virtual machine, use the command “xm network-attach”. Pass the MAC-id , bridge name and the script to create the VIF as arguments. The default vif creation script is “vif-bridge”. This is defined in /etc/xen/xend-config.sxp .

[root@host203 ~]# xm network-attach vm455 bridge=xenbr1 script=vif-bridge mac=00:16:3E:40:22:12
[root@host203 ~]# xm network-list vm455
Idx BE MAC Addr. handle state evt-ch tx-/rx-ring-ref BE-path
0 0 00:16:3E:47:AA:FD 0 4 12 2304 /2305 /local/domain/0/backend/vif/1/0
1 0 00:16:3E:22:33:44 1 4 13 2306 /2307 /local/domain/0/backend/vif/1/1
2 0 00:16:3E:40:22:12 2 4 14 2308 /2309 /local/domain/0/backend/vif/1/2
[root@host203 ~]#

The xm network-attach/detach command adds and removes NIC on a live machine. To make the changes permanent, edit the virtual machine configuration file in /etc/xen and edit the vifs. The same steps apply to Oracle Virtual machine also.

GSmartControl : GUI for smartctl

2009-03-08T08:14:00.001-07:00

GSmartControl is a graphical user interface for Linux smartctl, which is a tool for querying and controlling S.M.A.R.T. (Self-Monitoring, Analysis, and Reporting Technology) data on modern hard disk drives. You can read about smartctl at : http://unixfoo.blogspot.com/2007/11/disk-information-using-smartctl.html . It allows you to inspect the drive's S.M.A.R.T. data to determine its health, as well as run various tests on it.

GSmartControl features include

Automatically report and highlight any abnormal SMART information.
Ability to enable / disable SMART.
Ability to enable / disable Automatic Offline Data Collection - A short self-check that the drive will perform automatically every four hours with no impact on performance.
Ability to set global and per-drive options for smartctl.
Display drive identity, capabilities, attributes, error and self-test logs.
Perform SMART self-tests.
Ability to load smartctl output as a "virtual" device, which acts just like a real (read-only) device.
Works on most smartctl-supported operating systems.

The UI is very good and gives lot of information. Checkout the screenshots.

iperf : Network throughput measurement tool

2009-02-23T03:58:00.001-08:00

Iperf is a commonly used network testing tool that can create TCP and UDP data streams and measure the throughput of a network. Iperf allows the user to set various parameters that can be used for testing a network, or alternately for optimizing or tuning a network. Iperf has a client and server functionality, and can measure the throughput between the two ends, either unidirectonally or bi-directionally. You can download it in Sourceforce http://sourceforge.net/projects/iperf . Iperf is a client server program. In our example, let us start "server" iperf at location1 and let the "location2" be the client.

Server side : To start iperf as "server" service, use the "-s" flag.

[root@location1 ~]# iperf -s
---------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
---------------------------------------

To stop the server, send Ctrl+C.

Client side: For client side, iperf has to be initiated with "-c" option. There are lot of other options, which performs different kind of testing. Few examples are explained below.

1) Measuring throughput using default iperf option. This connects to the iperf server mentioned and finds out the network throughput.

[root@location2 ~]# iperf -c location1
---------------------------------------
Client connecting to location1, TCP port 5001
TCP window size: 16.0 KByte (default)
---------------------------------------
[ 3] local 201.87.4.3 port 41505 connected with 131.15.17.19 port 5001
[ ID] Interval       Transfer     Bandwidth
[ 3] 0.0-10.1 sec 27.4 MBytes 22.8 Mbits/sec
[root@location2 ~]#

2) Measuring bidirectional throughput usind -d option.

[root@location2 ~]# iperf -c location1 -d
---------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
---------------------------------------
Client connecting to location1, TCP port 5001
TCP window size: 16.0 KByte (default)
---------------------------------------
[ 5] local 201.87.4.3 port 41518 connected with 131.15.17.19 port 5001
[ 4] local 201.87.4.3 port 5001 connected with 131.15.17.19 port 59284
[ ID] Interval       Transfer     Bandwidth
[ 5] 0.0-10.1 sec 27.2 MBytes 22.7 Mbits/sec
[ 4] 0.0-10.0 sec 25.2 MBytes 21.1 Mbits/sec
[root@location2 ~]#

3) Measuring multi-thread throughput. Use -P option to specify the number of parallel threads to be started.

[root@location2 ~]# iperf -c location1 -P 3
---------------------------------------
Client connecting to location1, TCP port 5001
TCP window size: 16.0 KByte (default)
---------------------------------------
[ 5] local 201.87.4.3 port 41523 connected with 131.15.17.19 port 5001
[ 4] local 201.87.4.3 port 41522 connected with 131.15.17.19 port 5001
[ 3] local 201.87.4.3 port 41521 connected with 131.15.17.19 port 5001
[ ID] Interval       Transfer     Bandwidth
[ 5] 0.0-10.1 sec 27.9 MBytes 23.3 Mbits/sec
[ 4] 0.0-10.1 sec 27.9 MBytes 23.2 Mbits/sec
[ 3] 0.0-10.1 sec 27.8 MBytes 23.1 Mbits/sec
[SUM] 0.0-10.1 sec 83.6 MBytes 69.5 Mbits/sec
[root@location2 ~]#

3) Measuring throughput by disabling Nagle's Algorithm.

Nagle's algorithm, named after John Nagle, is a means of improving the efficiency of TCP/IP networks by reducing the number of packets that need to be sent over the network. [ http://en.wikipedia.org/wiki/Nagle%27s_algorithm ]

[root@location2 ~]# iperf -c location1 -P 3 -N
---------------------------------------
Client connecting to location1, TCP port 5001
TCP window size: 16.0 KByte (default)
---------------------------------------
[ 3] local 201.87.4.3 port 41527 connected with 131.15.17.19 port 5001
[ 4] local 201.87.4.3 port 41528 connected with 131.15.17.19 port 5001
[ 5] local 201.87.4.3 port 41529 connected with 131.15.17.19 port 5001
[ ID] Interval       Transfer     Bandwidth
[ 3] 0.0-10.1 sec 27.8 MBytes 23.1 Mbits/sec
[ 4] 0.0-10.1 sec 27.9 MBytes 23.2 Mbits/sec
[ 5] 0.0-10.1 sec 28.0 MBytes 23.2 Mbits/sec
[SUM] 0.0-10.1 sec 83.7 MBytes 69.6 Mbits/sec
[root@location2 ~]#

You can even run the client side on regular intervals using cron daemon and plot graphs (using mrtg or rrd) for the throughput. Hope this is helpful.

Vmware Performance tuning Tips

2009-02-12T01:14:00.001-08:00

These posts explain about Vmware performance tuning on Linux servers.

These links explains the tips to tune the disk io by setting read-aheads and memory based improvements by making changes to kernel parameters like vm.swapiness, vm.overcommit_memory, vm.dirty_background_ratio, vm.dirty_ratio. They also suggests some vmware config tweaks like mainMem.useNamedFile. Worth trying if you have Vmware performance issues. But dont try in production unless you are sure of what you are doing.

EMC CLARiiON AX Demo

2009-02-02T20:00:00.001-08:00

Gotitsolutions.org has a great demo or EMC CLARiiON AX. You can see the demo here.

More details of EMC CLARiiON AX can be found here.

Xen performance monitoring

2009-01-29T06:58:00.001-08:00

Xen’s Hypervisor does not have an easy collection of performance counters. The management machine - “Domain-0″ is actually a privileged virtual machine, and thus - get its own small share of CPUs and RAM. Collecting performance information on it will lead to collecting performance information for a single VM, and not the whole bunch. “xentop” allows collection of information, however, combining this with Cacti , or any other SNMP-based collection tool is a bit tricky. A great solution is provided by Ian P. Christian in his blog post about Xen montoring and there is a script that collects the performance details.

Reference

OpenSource Tools for AIX

2009-01-22T03:37:00.001-08:00

Here are several websites that has OpenSource Tools for AIX. The "IBM AIX Toolbox" has the most tools, but not necessarily the most current versions. The other links are also good to note.

IBM AIX Toolbox for Linux Applications – http://www.ibm.com/servers/aix/products/aixos/linux/download.html
AIX Open Source – http://www.perzl.org/aix/
Hudson Valley CC - http://pware.hvcc.edu
Bull Open Software archive for AIX - http://www.bullfreeware.com
IBM Developerworks in general - http://www.ibm.com/developerworks/downloads/

Reference : AIXpert Blog

Linux Kernel panic reboot

2009-01-22T02:14:00.001-08:00

By default after a kernel panic Linux just waits there for a sysadmin to hit the restart or powercycle button. This is because of the value set on "kernel.panic" parameter.

[root@linux23 ~]# cat /proc/sys/kernel/panic
0
[root@linux23 ~]# sysctl -a | grep kernel.panic
kernel.panic = 0
[root@linux23 ~]#

To disable this and make the Linux OS reboot after a kernel panic, we have to set an integer N greater than zero to the paramter "kernel.panic", where "N" is the number of seconds to wait before a automatic reboot. For example , if you set N = 10 , then the system waits for 10 seconds before automatic reboot. To make this permanent, edit /etc/sysctl.conf and set it.

[root@linux23 ~]# echo "10" > /proc/sys/kernel/panic
0
[root@linux23 ~]# grep kernel.panic /etc/sysctl.conf
kernel.panic = 10
[root@linux23 ~]#

Netapp Ontap Management SDK

2009-01-21T04:06:00.001-08:00

Netapp provides an SDK ( Netapp API ) that contains resources necessary to develop third-party applications which monitor and manage Netapp Filers. The SDK contains libraries, code samples and bindings in C and Perl for the Netapp Ontap management API programming.

The Ontap SDK contains ..

SDK Core API library bindings in C and C++, Java and Perl.
SDK Core API and Data ONTAP API documentation, sample codes, developer tools, Design guides.
Manage ONTAP SDK Help.

For more detailed information read : http://communities.netapp.com/docs/DOC-1110
You can download the SDK from : http://communities.netapp.com/docs/DOC-1365

Solaris : Auditing File attributes

2009-01-20T19:51:00.001-08:00

Solaris has a software registry which maintains information of software packages installed. The registry is invaluable for auditing the system to determine what software has been changed, installed, removed, or patched. The software registry contains a database of installed files. This database is physically located in the file /var/sadm/install/contents . Each file, special file, and directory installed on the system has an entry in this database. If some attributes of files are changed after installation, "pkgchk" command can find it out and report it. A good command for auditing. Here is an example..

solaris98# pkgchk
ERROR: /etc/apache/magic
    file size <12965> expected <12441> actual
    file cksum <8026> expected <33401> actual
ERROR: /etc/apache/mime.types
    file size <14987> expected <9957> actual
    file cksum <46595> expected <27635> actual
ERROR: /etc/auto_master
    file size <113> expected <395> actual
    file cksum <9773> expected <34676> actual
ERROR: /etc/default/dhcpagent
    file size <3394> expected <2826> actual
    file cksum <26394> expected <43621> actual

Some fiiles are expected to change such as /etc/system - which gets edited by sysadmin very often. pkgchk has a -n option that will bypass checking these files. Though this is a tempting option to use for reducing the amount of output from an audit, it is good to know what got changed.

solaris98# pkgchk -l -p /etc/system
Pathname: /etc/system
Type: editted file
Expected mode: 0644
Expected owner: root
Expected group: sys
Referenced by the following packages:
SUNWcsr
Current status: installed

solaris98#

If you want to check what got changed on a filesystem , you can use find & pkgchk to know it. Check the example below.

solaris98# find /usr -mount -exec pkgchk -p {} \;
ERROR: /usr
permissions <0755> expected <0775> actual
WARNING: no information associated with pathname </usr/platform/TSBW>
WARNING: no information associated with pathname <8000>
WARNING: no information associated with pathname </usr/platform/TSBW>
WARNING: no information associated with pathname <Ultra-2i>
..

Reference : http://www.sun.com/blueprints/1299/repairing.pdf

Linux Tips : Useful links

2009-01-19T07:38:00.001-08:00

Came across the below links and it has lot of useful linux tips - neat and great. Checkout them

Netapp Storage Commands

2009-01-07T19:27:00.001-08:00

Here are some of the useful functions of "storage" command in Netapp.

1) To show all disks on the system : Use "storage show disk -T" to display all the disks attached to the filer, the disk serial number, vendor, model, disk firmware version and type of disk (SATA/ATA/FCAL)

# rsh filer12 storage show disk -T
DISK                  SHELF BAY SERIAL           VENDOR   MODEL      REV TYPE
--------------------- --------- ---------------- -------- ---------- ---- ------
0d.16                   1    0 xxxxxxxxxxxxxxxx NETAPP   X276 NA07 FCAL
0d.17                   1    1 xxxxxxxxxxxxxxxx NETAPP   X276 NA07 FCAL
0d.18                   1    2 xxxxxxxxxxxxxxxx NETAPP   X276 NA07 FCAL
0d.19                   1    3 xxxxxxxxxxxxxxxx NETAPP   X276 NA07 FCAL
0d.20                   1    4 xxxxxxxxxxxxxxxx NETAPP   X276 NA07 FCAL
0d.21                   1    5 xxxxxxxxxxxxxxxx NETAPP   X276 NA07 FCAL
0d.22                   1    6 xxxxxxxxxxxxxxxx NETAPP   X276 NA07 FCAL
...

2) To see complete information of a particular disk : Use "storage show disk -a <disk-id>" to view complete information of a netapp disk. This command gives you the shelf, bay, serial number of disk, disk speed and many other.

# rsh filer12 storage show disk -a 0d.99
Disk:             0d.99
Shelf:            5
Bay:              13
Serial:           xxxxxxxxxxxxxxxxxxxx
Vendor:           NETAPP
Model:            X276
Rev:              NA07
RPM:              10000
WWN:              xxxxxxxxxxxxxxxxxxa
UID:              xxxxxxxxxxxxxxxxx:00000000:00000000:00000000:00000000
Downrev:          no
Pri Port:         B
Power-on Hours:   N/A
Blocks read:      0
Blocks written:   0
Time interval:    00:00:00
Glist count:      0
Scrub last done: 00:00:00
Scrub count:      0
LIP count:        0
Dynamically qualified: No
#

3) To list all storage adapters on the filer : Use "storage show adapter -a" command to display all the storage adapters (hba) on the filer.

# rsh filer12 storage show adapter -a

Slot:            0a
Description:     Fibre Channel Host Adapter 0a (Dual-channel, QLogic 2322 rev. 3)
Firmware Rev:    3.3.25
FC Node Name:    xxxxxxxxxxxxxxxxxxx
FC Packet Size: 2048
Link Data Rate: 2 Gbit
SRAM Parity:     Yes
External GBIC:   No
State:           Enabled
In Use:          No
Redundant:       Yes

Slot:            0b
Description:     Fibre Channel Host Adapter 0b (Dual-channel, QLogic 2322 rev. 3)
Firmware Rev:    3.3.25
FC Node Name:    xxxxxxxxxxxxxxxxxxx
..

4) To get shelf details of filer : Use "storage show shelf <shelf-id>" command to display the details of the shelf and its partner shelf.

# rsh filer12 storage show shelf 0c.shelf2
Shelf name:    0c.shelf2
Channel:       0c
Module:        A
Shelf id:      2
Shelf UID:     xxxxxxxxxxxxxxxxxxxxxxx
Term switch:   N/A
Shelf state:   ONLINE
Module state: OK

                               Loop Invalid Invalid Clock Insert Stall Util    LIP
Disk    Disk     Port            up      CRC     Word Delta   Count Count Percent Count
Id     Bay    State         Count    Count    Count
----------------------------------------------------------------------------------------
[IN ]          OK                0        0        0      8       0      0    71     0
[OUT ]          OK                0        0        0      0       0      0    52     0
[ 32]     0    OK                0        0        0     32       0      0     0     0
[ 33]     1    OK                0        0        0     32       0      0     2     0
[ 34]     2    OK                0        0        0     24       0      0     0     0
[ 35]     3    OK                0        0        0     24       0      0     1     0
[ 36]     4    OK                0        0        0      8       0      0     2     0
[ 37]     5    OK                0        0        0     24       0      0     4     0
...

More Netapp commands at : http://unixfoo.blogspot.com/search/label/netapp