Setting up an aggregated link between a Cisco switch and a VMware ESXi server

Goal

Increase bandwidth between the network and an ESXi host by setting up an aggregated link between the two.

These aggregated links are commonly refered to as an etherchannel, trunk, portchannel or teamed NIC’s.

Prerequisites

  • A recent Cisco switch that supports load balancing over an Etherchannel based on source and destination IP addresses. (For example, the popular Catalyst 2950 series do not support this loadbalancing method). You can check if your switch supports this load balancing by checking if the exec command
    port-channel load-balance src-dst-ip
    is available. If it’s not, you can’t use this switch for the purpose described in this post.

Caveats

  • ESXi does not support dynamic aggregated links with protocols like LACP. One must manually configure the link on both ends.
  • Available bandwidth to 1 single host will not increase, as this is the nature of the aggregation link technologies being used. If you have two 100 MBit links, the maximum attainable speed between a virtual machine and a single host on the network will still be 100 MBit. However, if two hosts were to connect to the virtual machine, chances are pretty good one host’s traffic will go via one physical link and the other host’s via the second physical link.
  • This example aggregates two physical links into one, you can use more. You can mix different port speeds, but recommended configuration is all links having the same speed.

Configure ESXi

Edit the network settings by going to Configuration -> Networking. Edit the virtual network properties which you want to create an aggregated link for, in this example this is vSwitch0.

Next, add the second network interface to the vSwitch in the Network Adapters tab:

Now we need to configure ESXi to bond the links on these to adapters together. Go back to the Ports tab and edit the vSwitch properties:

On the vSwitch properties window, go to the last tab NIC Teaming and set Load Balancing to “Route based on IP hash”:

That’s it for vmware. Now we need to configure the switch to create the aggregated link.

Configure the switch

In this example FastEthernet 0/23 and FastEthernet 0/24 are connected to my VMware ESXi server, so I’m going to use the interface range commands to apply the necessary configuration to both switchports.

It’s important you match the loadbalancing method ESXi uses to the one the switch uses. This is done using the port-channel load-balance command.

s2(config)#interface range FastEthernet 0/23 – 24
s2(config-if-range)#
s2(config-if-range)#channel-group 1 mode on
Creating a port-channel interface Port-channel 1
00:25:49: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up
00:25:50: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up
s2(config-if-range)#exit
s2(config)#port-channel load-balance src-dst-ip

Verify port-channel operation:

s2#show interface port-channel 1
Port-channel1 is up, line protocol is up (connected)
  Hardware is EtherChannel, address is 64d9.89ee.1234 (bia 64d9.89ee.1234)
  Description: To vwmare for VMs
  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s, link type is auto, media type is unknown
  input flow-control is off, output flow-control is unsupported
  Members in this channel: Fa0/23 Fa0/24
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:01, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 6000 bits/sec, 2 packets/sec
  5 minute output rate 3000 bits/sec, 3 packets/sec
     32312407 packets input, 33220875322 bytes, 0 no buffer
     Received 135526 broadcasts (71135 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 71135 multicast, 0 pause input
     0 input packets with dribble condition detected
     55382934 packets output, 67979911754 bytes, 0 underruns
     0 output errors, 0 collisions, 1 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out
s2#

References

SSH with key authentication on Cisco IOS devices

Goal

Connect to a Cisco switch, router, etc… using SSH with key authentication.

Prerequisites

  • Have putty and puttygen.
  • You have already configured your Cisco device to be able to accept SSH logins using usernames and passwords.

Caveats

Keep your SSH keys in a safe place, treat them like the keys to your house (unless you don’t particularly care about your house). If you suspect your keys have been stolen, make sure no devices will accept your stolen SSH key! Setting a passphrase on your key is a smart idea.

Generating a RSA key with PuTTYgen

Open puttygen.exe and set the parameters: the type of key must be SSH-2 RSA. You can vary the number of bits in the generated key, a higher number is more secure. Click the Generate button to generate a new key, move the mouse around the window to create additional randomness.

Appropriately comment your key.

You may wish to enter a passphrase, this passphrase will be asked every time you connect to a device using that key. It’s like a password.

Save your public key by clicking Save public key.  Create a folder to store your keys and name the file publickey.pub. Next, click Save private key, save it under the same folder as privatekey.ppk. Also, copy the “Public key for pasting into OpenSSH authorized_keys file” bit and save it to file.

The end result should look something like this:

Setting up your Cisco device to accept your key

Assuming you have a user called “admin” as which you want to perform key based authentication, we will associate this user with a key he’s allowed to login with.

Enter the following commands:

ip ssh pubkey-chain
 username admin
  key-string

Now, you need to copy the bit from authorized_keys. You only need the signature, you must leave away the “ssh-rsa” and the comment at the end bits. So, if you have this authorized_keys:

ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIBztRNLSqzuIDxATAl6zAhLcsTL40XHAANd+7ljTpbgvYX5IDJjYyD4jpDW9x8Qml553k0chDNFuW2ZE0gVL+MetDatI/DrgMIVRVcU9ZJLsVGqv6SuXeQI7UpvkP7ow+HS0hTd7GDw9sJ+OjAEIIcAhlBJ+4CPIeWjs98Z5ube6Q== my-cisco-device-key

You need to paste the

AAAAB3NzaC1yc2EAAAABJQAAAIBztRNLSqzuIDxATAl6zAhLcsTL40XHAANd+7ljTpbgvYX5IDJjYyD4jpDW9x8Qml553k0chDNFuW2ZE0gVL+MetDatI/DrgMIVRVcU9ZJLsVGqv6SuXeQI7UpvkP7ow+HS0hTd7GDw9sJ+OjAEIIcAhlBJ+4CPIeWjs98Z5ube6Q==

part. (Pasting is right-click in PuTTY). Press enter.

Exit a few times to leave the device’s configuration.

That’s it for configuring the Cisco device. You can add multiple keys per user and you can have multiple users each having their own keys for logging in.

Setting up a PuTTY profile

To make use of your key, you must specifically configure PuTTY to do so. Open PuTTY and go to Connection -> SSH -> Auth. On the Private key file for authentication field, browse to your privatekey.ppk file.

You may also wish PuTTY automatically tries to login with the username. Go to Connection -> Data and fill in the Auto-login username field.

Once you’ve completed your setup, you may wish to save your session in the Session dialogue, so you don’t have to repeat this every time you want to login to your device.

Testing

When you login using your key, the result should be like this:

Add iSCSI initiator to the Windows Server 2008/2008R2/SBS 2008/SBS 2011 installation DVD

Goal

These days, in large environments, iSCSI has become the defacto standard to make block storage readily available to other computers. A possible deployment scenario is attaching an iSCSI target to a Windows server to create backups using the built-in Windows backup software.

However, the Windows installation DVD (which is required for a BMR, Bare-Metal Restore) doesn’t come with the Microsoft iSCSI initiator. So we’re going to add it ourselves.

This works for all installation DVD’s starting with Windows Vista up to Windows 7 including all server versions.

Updates

25/01/2012: the registry entries are now a hyperlink to a file you can download. WordPress was leaving out the “\ 0” (without the space) which could lead to all sorts of errors including “initiator instance does not exist”.

Prerequisites

  • The Windows Automated Installation Kit, installed, specific for the operating system you wish to restore. If you want to restore a Windows 2008 server, you need the Windows Vista AIK, if you want to restore a Windows 2008 R2 server, you need the Windows 7 AIK.
  • A Windows Vista/7/Server 2008/Server 2008R2/SBS 2008/SBS 2011 DVD.
  • A clean Windows installation. Preferably install a new machine.
  • This also works with the Small Business Server 2008 or 2011 operating system. SBS 2008 is based on Windows 2008 Server (which, in turn, is based on Windows Vista) and SBS 2011 on Windows 2008R2 Server (which is based on Windows 7).

Caveats

  • The major release of Windows you wish to restore must match the DVD you’re creating. You cannot use a Windows 7 DVD to restore a Windows Vista/2008 Server and you cannot use a Windows Vista/2008 Server DVD to restore a Windows 7. So:
    • Restore Windows Vista/2008 Server -> Windows Vista AIK & DVD
    • Restore Windows 7/2008 R2 Server -> Windows 7 AIK & DVD
  • Basically we’re going to be ripping out the necessary components out of an existing Windows installation and put these components on the installation DVD.
  • We’re going to use the pre-installation environment from the Windows DVD because the pre-installation environment from the AIK, winpe.wim, doesn’t contain recenv.exe, what you need to restore a backup.
  • All systems involved should have the same architecture.

Stealing the iSCSI initiator components

For this you’ll need a clean installed Windows OS. You need to copy the following files from it:

%systemroot%\System32\drivers\msiscsi.sys
%systemroot%\System32\en-US\iscsicli.exe.mui
%systemroot%\System32\en-US\iscsicpl.dll.mui
%systemroot%\System32\en-US\iscsicpl.exe.mui
%systemroot%\System32\en-US\iscsidsc.dll.mui
%systemroot%\System32\en-US\iscsiexe.dll.mui
%systemroot%\System32\en-US\iscsilog.dll.mui
%systemroot%\System32\iscsicli.exe
%systemroot%\System32\iscsicpl.dll
%systemroot%\System32\iscsicpl.exe
%systemroot%\System32\iscsidsc.dll
%systemroot%\System32\iscsied.dll
%systemroot%\System32\iscsiexe.dll
%systemroot%\System32\iscsilog.dll
%systemroot%\System32\iscsium.dll
%systemroot%\System32\iscsiwmi.dll
%systemroot%\System32\oledlg.dll

Keep the tree intact.

Setting up the root

Open the Deployment Tools Command Prompt as Administrator and use the included copype script to set up the root:

In the command prompt:

C:\Program Files\Windows AIK\Tools\PETools>copype
Usage: copype [x86 | amd64 | ia64] destination

Example: copype x86 c:\windowspe-x86

C:\Program Files\Windows AIK\Tools\PETools>copype amd64 c:\tmp 
===================================================
Creating Windows PE customization working directory

    c:\tmp
===================================================

        1 file(s) copied.
        1 file(s) copied.
        1 file(s) copied.
        1 file(s) copied.
        1 file(s) copied.
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\bcd
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\boot.sdi
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\bootfix.bin
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\efisys.bin
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\efisys_noprompt.bin
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\etfsboot.com
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\fonts\chs_boot.ttf
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\fonts\cht_boot.ttf
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\fonts\jpn_boot.ttf
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\fonts\kor_boot.ttf
C:\Program Files\Windows AIK\Tools\PETools\amd64\boot\fonts\wgl4_boot.ttf
11 File(s) copied
C:\Program Files\Windows AIK\Tools\PETools\amd64\EFI\boot\bootx64.efi
C:\Program Files\Windows AIK\Tools\PETools\amd64\EFI\microsoft\boot\bcd
C:\Program Files\Windows AIK\Tools\PETools\amd64\EFI\microsoft\boot\fonts\chs_boot.ttf
C:\Program Files\Windows AIK\Tools\PETools\amd64\EFI\microsoft\boot\fonts\cht_boot.ttf
C:\Program Files\Windows AIK\Tools\PETools\amd64\EFI\microsoft\boot\fonts\jpn_boot.ttf
C:\Program Files\Windows AIK\Tools\PETools\amd64\EFI\microsoft\boot\fonts\kor_boot.ttf
C:\Program Files\Windows AIK\Tools\PETools\amd64\EFI\microsoft\boot\fonts\wgl4_boot.ttf
7 File(s) copied
        1 file(s) copied.

Success

Updating path to include peimg, cdimage, imagex

   C:\Program Files\Windows AIK\Tools\PETools\
   C:\Program Files\Windows AIK\Tools\PETools\..\AMD64

c:\tmp>

You should now have the following directory structure:

Copying boot.wim from the installation DVD

Insert your Windows installation DVD and copy the following file

\sources\boot.wim

to

C:\tmp\ISO\sources

Mounting the .wim image

We must mount the boot.wim image using imagex, this will allow us to edit the files inside the .wim file. A .wim file can contain multiple images (that’s how they create DVD’s that can install multiple editions of Windows 7), so it’s important we supply the correct image_number argument. We can check which images a .wim file contains using the same imagex tool:

C:\tmp>imagex /info /?

ImageX Tool for Windows
Copyright (C) Microsoft Corp. All rights reserved.
Version: 6.1.7600.16385

IMAGEX [FLAGS] /INFO img_file [img_number | img_name] [new_name] [new_desc]

Returns the stored XML descriptions for the specified WIM or image.

  img_file - The path of the WIM file to be queried for XML information.
  img_number - The number that identifies an image within the WIM file.
  img_name - The name that identifies an image within the WIM file.
  new_name - The new unique name for the specified image.
  new_desc - The new description for the specified image.

Accepted FLAGS:

  /BOOT
  Marks a volume image as bootable. Available for Windows PE images only.

  /CHECK
  If not provided, existing checks are removed during updates.

  /TEMP
  Specifies the path where temporary files are stored.

  /XML
  Returns the output as well-formed XML.

Example:
  imagex /info d:\imaging\data.wim

C:\tmp>imagex /info C:\tmp\ISO\sources\boot.wim

ImageX Tool for Windows
Copyright (C) Microsoft Corp. All rights reserved.
Version: 6.1.7600.16385

WIM Information:
----------------
Path:        C:\tmp\ISO\sources\boot.wim
GUID:        {868be8f0-f22c-494b-b20a-e9d997a921e5}
Image Count: 2
Compression: LZX
Part Number: 1/1
Boot Index:  2
Attributes:  0x8
             Relative path junction

Available Image Choices:
------------------------
<WIM>
  <TOTALBYTES>168641635</TOTALBYTES>
  <IMAGE INDEX="1">
    <DIRCOUNT>2182</DIRCOUNT>
    <FILECOUNT>9853</FILECOUNT>
    <TOTALBYTES>985750801</TOTALBYTES>
    <HARDLINKBYTES>334277841</HARDLINKBYTES>
    <CREATIONTIME>
      <HIGHPART>0x01CB88D1</HIGHPART>
      <LOWPART>0xDB7CCA61</LOWPART>
    </CREATIONTIME>
    <LASTMODIFICATIONTIME>
      <HIGHPART>0x01CB88D1</HIGHPART>
      <LOWPART>0xDBE0C44B</LOWPART>
    </LASTMODIFICATIONTIME>
    <WINDOWS>
      <ARCH>9</ARCH>
      <PRODUCTNAME>Microsoft® Windows® Operating System</PRODUCTNAME>
      <EDITIONID>WindowsPE</EDITIONID>
      <INSTALLATIONTYPE>WindowsPE</INSTALLATIONTYPE>
      <PRODUCTTYPE>WinNT</PRODUCTTYPE>
      <PRODUCTSUITE></PRODUCTSUITE>
      <LANGUAGES>
        <LANGUAGE>en-US</LANGUAGE>
        <DEFAULT>en-US</DEFAULT>
      </LANGUAGES>
      <VERSION>
        <MAJOR>6</MAJOR>
        <MINOR>1</MINOR>
        <BUILD>7601</BUILD>
        <SPBUILD>17514</SPBUILD>
        <SPLEVEL>1</SPLEVEL>
      </VERSION>
      <SYSTEMROOT>WINDOWS</SYSTEMROOT>
    </WINDOWS>
    <NAME>Microsoft Windows PE (x64)</NAME>
    <DESCRIPTION>Microsoft Windows PE (x64)</DESCRIPTION>
    <FLAGS>9</FLAGS>
  </IMAGE>
  <IMAGE INDEX="2">
    <DIRCOUNT>2413</DIRCOUNT>
    <FILECOUNT>10532</FILECOUNT>
    <TOTALBYTES>1072858042</TOTALBYTES>
    <HARDLINKBYTES>379422568</HARDLINKBYTES>
    <CREATIONTIME>
      <HIGHPART>0x01CB88D1</HIGHPART>
      <LOWPART>0xEA7C5A8E</LOWPART>
    </CREATIONTIME>
    <LASTMODIFICATIONTIME>
      <HIGHPART>0x01CB8951</HIGHPART>
      <LOWPART>0x471198E5</LOWPART>
    </LASTMODIFICATIONTIME>
    <WINDOWS>
      <ARCH>9</ARCH>
      <PRODUCTNAME>Microsoft® Windows® Operating System</PRODUCTNAME>
      <EDITIONID>WindowsPE</EDITIONID>
      <INSTALLATIONTYPE>WindowsPE</INSTALLATIONTYPE>
      <PRODUCTTYPE>WinNT</PRODUCTTYPE>
      <PRODUCTSUITE></PRODUCTSUITE>
      <LANGUAGES>
        <LANGUAGE>en-US</LANGUAGE>
        <DEFAULT>en-US</DEFAULT>
      </LANGUAGES>
      <VERSION>
        <MAJOR>6</MAJOR>
        <MINOR>1</MINOR>
        <BUILD>7601</BUILD>
        <SPBUILD>17514</SPBUILD>
        <SPLEVEL>1</SPLEVEL>
      </VERSION>
      <SYSTEMROOT>WINDOWS</SYSTEMROOT>
    </WINDOWS>
    <NAME>Microsoft Windows Setup (x64)</NAME>
    <DESCRIPTION>Microsoft Windows Setup (x64)</DESCRIPTION>
    <FLAGS>2</FLAGS>
  </IMAGE>
</WIM>

C:\tmp\>

Mount the image. The one we want is the “Microsoft Windows Setup (x64)”, which has number 2.

C:\tmp>imagex /mountrw /?

ImageX Tool for Windows
Copyright (C) Microsoft Corp. All rights reserved.
Version: 6.1.7600.16385

IMAGEX [FLAGS] /MOUNTRW [image_file image_number | image_name image_path]

Mounts a WIM image with read/write permission, to a specified path.

  image_file - The path of the WIM file containing the specified image.
  image_number - The number that identifies the image within the WIM file.
  image_name - The name that identifies the image within the WIM file.
  image_path - The path where the specified image will be mounted.

Without FLAGS:

  Lists mounted images.

Accepted FLAGS:

  /CHECK
  Enables WIM integrity checking. If not provided, existing checks are removed.

Example:
  imagex /mountrw d:\imaging\data.wim 2 c:\mounted_images

C:\tmp>imagex /mountrw c:\tmp\ISO\sources\boot.wim 2 C:\tmp\mount

ImageX Tool for Windows
Copyright (C) Microsoft Corp. All rights reserved.
Version: 6.1.7600.16385

Mounting: [c:\tmp\ISO\sources\boot.wim, 2] -> [C:\tmp\mount]...

[ 100% ] Mounting progress

Successfully mounted image.

Total elapsed time: 47 sec

C:\tmp>

If we now visit C:\tmp\mount, what we see is actually the content of boot.wim.

Integrating the iSCSI initiator components

Copying files

Copy the files you retrieved from your source installation to where you mounted the boot.wim tree. The files must match in location, so if you had for example

C:\Windows\System32\iscsicli.exe

This must become

C:\tmp\mount\Windows\System32\iscsicli.exe

Verify all files stated above are present.

Modifying the registry

Next up is modifying the registry to integrate the iSCSI initiator driver in the Windows setup environment. Open the Windows registry editor (regedit.exe) and select the HKEY_LOCAL_MACHINE root. This will allow you to load an external registry file. We have to load two hives. Select File -> Load Hive and browse to:

C:\tmp\mount\Windows\System32\config\SYSTEM

Key Name: PE_Sys

Next up, load:

C:\tmp\mount\Windows\System32\config\SOFTWARE

Key Name: PE_Soft

The result should look like this:

Right click and save the following .reg file:

http://users.telenet.be/redshift/iscsi.reg

Then select File -> Import and import the .reg file you just downloaded.

Unload the loaded hives by selecting File -> Unload Hive on PE_Soft and PE_Sys.

You’re done, all necessary components have been integrated. If you want you can extra drivers (storage, networking, etc…) using the dism tool.

Saving changes to the .wim file

Make sure there are no open explorer windows on C:\tmp\mount and the registry hives are unloaded. We use the same imagex tool to commit the changes and unmount the .wim file:

C:\tmp>imagex /commit /?

ImageX Tool for Windows
Copyright (C) Microsoft Corp. All rights reserved.
Version: 6.1.7600.16385

IMAGEX [FLAGS] /COMMIT mount_path ["image_name"]

Commits the changes made to a mounted image without unmounting the image.

  mount_path - The path of the mounted image to commit.
  image_name - If the /append flag is set, then a unique image name must be provided.

Accepted FLAGS:

  /APPEND
  Captures the changes made to the wim into a new image in the wim.

  /TEMP
  Specifies the path where temporary files are stored.

Example:
  imagex /commit c:\mounted_images
  imagex /commit /append c:\mounted_images new_image_name

C:\tmp>imagex /commit c:\tmp\mount 
ImageX Tool for Windows
Copyright (C) Microsoft Corp. All rights reserved.
Version: 6.1.7600.16385

Committing: [c:\tmp\mount]...

[ 100% ] Committing Image progress

Successfully committed image.

Total elapsed time: 19 sec

C:\tmp>imagex /unmount /?

ImageX Tool for Windows
Copyright (C) Microsoft Corp. All rights reserved.
Version: 6.1.7600.16385

IMAGEX /UNMOUNT [/COMMIT] [image_path]

Unmounts a WIM image from the specified path.

  image_path - The path to be unmounted.

Without FLAGS:

  Lists mounted images.

Accepted FLAGS:

  /COMMIT
  Saves changes to the mounted WIM file. If not specified changes are discarded.

Example:
  imagex /unmount /commit c:\mounted_images

C:\tmp>imagex /unmount c:\tmp\mount 
ImageX Tool for Windows
Copyright (C) Microsoft Corp. All rights reserved.
Version: 6.1.7600.16385

Unmounting: [c:\tmp\mount]...

[ 100% ] Mount cleanup progress

Successfully unmounted image.

Total elapsed time: 7 sec

C:\tmp>

You can combine these last two steps by using

imagex /unmount /commit c:\tmp\mount

Creating a bootable ISO

We use the oscdimg tool to create a bootable image:

oscdimg -u2 -bc:\tmp\etfsboot.com c:\tmp\ISO c:\tmp\image.iso

This will result in a file C:\tmp\image.iso. You can burn this ISO to disc.

Testing

After you’ve created the image, you should test it. Boot up a machine with it. What will first appear is the regular Windows Setup phase. You won’t be able to install Windows with it, as install.wim is missing. However, you can still use all the other tools present. After selecting languages, click “Repair your computer”. The system will offer to repair an existing Windows installation if there is one present in the machine’s disk. Be sure to deselect the OS it’s offering by clicking in the blank space. Then press Next >. Now we have a window with System Recovery Options.

First we need to initialize networking. Do this by selecting Command Prompt and enter:

 wpeutil initializenetwork

Verify network operating with ipconfig:

ipconfig

Continue on by starting the msiscsi service:

net start msiscsi

Now start the iSCSI initiator applet:

iscsicpl

Configure your iSCSI backup target in the initiator. Select System Image Recovery in the System Recovery Options menu to start restoring a backup.

References

Microsoft: Deployment Image Servicing and Management Technical Reference