Social Icons


Sunday, September 25, 2011

iSCSI Boot from SAN on Cisco UCS

Last week Cisco released the new 2.0 code for Cisco UCS.  I had some time to play with the update and tried out the new iSCSI Boot from SAN support feature to see how it works.  Here are some of the steps I used to get a VMware ESXi 5.0 host up and running via iSCSI storage on UCS.

Service Profile and vNIC Configuration

1)      In the UCSM, create a Service Profile.  This profile will be created as usual(except for vHBAs) but make sure that you create 2 vNICs that will be reserved for iSCSI traffic.  UCS will use these vNICs as “Overlay NICs” that will provide the VLAN connectivity details for the iSCSI vNICs in the Service Profile.
2)      Configure the VLANs on the Overlay vNICs by selecting the vNIC and clicking the Modify VLANs link in the working pane. 
3)      In the Modify VLANs window, Select your VLAN for iSCSI and select the “Native VLAN” radial button for the VLAN.  Making this the native VLAN for the Overlay vNICs will pass the frames for the iSCSI VLAN untagged and ease the configuration and installation of the OS.  Click OK, and then click the Save Changes button in the working pane for the vNIC.  Repeat these steps for the 2nd vNIC that will be used as an Overlay vNIC for iSCSI.

Create Boot Policy for iSCSI 
1)      Highlight the Service Profile in the left pane, click the Boot Order tab in the working pane and click “Modify Boot Policy”.
2)      In the Modify Boot Policy Window Click Create Boot Policy.
3)      Enter the Boot Policy name and Description, and check the “Enforce vNIC/vHBA/iSCSI name” box.
4)      Add a CD-ROM device.
5)      Expand iSCSI vNICs, Click Add iSCSI Boot, then enter the iSCSI vNIC name and click Ok.  Repeat for the 2nd iSCSI vNIC.  Click OK.
Creating the iSCSI vNICs
1)      Click the + sign next to the Service Profile, select “iSCSI vNICs” then click the Add button at the bottom of the working pane.  Enter the name for the iSCSI vNIC, select the first Overlay vNIC you created earlier, the iSCSI Adapter Policy if necessary and then select the VLAN for iSCSI traffic(or leave at default) then click ok.  Repeat for the 2nd iSCSI vNIC.

Note:  Do not configure the MAC Address Assignment for Palo cards

Configuring iSCSI Boot Parameters
1)      Select the Service Profile in the left pane click the Select the Boot Order in the working pane and then click the Modify Boot Policy link.
2)      Select the preconfigured Boot Policy that you have created to boot from iSCSI from the drop down then click OK.
3)      Back in the Boot Order tab, Select the first iSCSI object then click the Set ISCSI Boot Parameters button.
4)      Enter the Initiator Name that you have created for this server
Note:  The UCSM will not generate an iqn for the server/service profile.  While it may not be necessary, I recommend creating iqns similar to the iqn naming standard.  Make sure that the iqn for each server is unique!

5)      In the Initiator Address section, Select Static from the drop down then enter the IP Address information for this specific iSCSI vNIC.
6)      Next select iSCSI Target Interface, then click the + sign.
7)      In the Create iCSCI Target window, enter the iqn for the target volume you created earlier in LeftHand CMC.  Enter the IP address, and make sure the that LUN ID is zero.  Click OK, Then Click OK again.
8)      Repeat steps 3 through 7 of this section for the 2nd iSCSI vNIC.
Booting the Server
1)      With the iSCSI and Overlay vNICs configured and the Boot Parameters complete, you are now ready to boot the server
2)      Apply the Service Profile to a supported blade(Blade must be using a Palo Interface Card for this guide).
3)      Once the Service Profile has completed the association process, Boot the server
4)      When the Cisco splash screen appears, click the Esc key to see the devices load prompts during boot.
5)      When the Cisco VIC loads, make sure that the screen displays the information for your iSCSI volume.  
6)      When the server prompts for boot media, connect to the ISO file for VMware ESXi 5.0 and follow the installation prompts to complete the installation.  You should see the configured Lefthand volume when prompted for the installation location.
7)      Select the drive and continue the installation.
8)      Reboot the server once the installation is complete.


Mani said...

Full disclosure, I’m a Cisco employee

If you are using an SP template to clone service profiles, it is better to use the "Specific boot policy option". THis is available when you go and select boot policy.

When you create specific boot policy you don't need to reenter the target details in each of the cloned SP.

Harold Simon said...


Thanks for posting and good point! I usually do that for SP templates, but I just did a quick run through here on the iSCSI Boot config. Also for iSCSI, it appears that you will still have to manually configure the the iSCSI boot parameters per SP when using Static initiator IP address.
However using DHCP for initiator IPs should work fine with the SP template configuration that you mentioned.

Actually even using the DHCP setting you will need to enter the unique Initiator Name in the Boot Parameters for each SP that will be used for a VMware ESXi host(From what I have seen, UCS doesn't generate these dynamically). This initiator name will be used for LUN masking on the storage.

Mani said...

Hi Harold,
There are 2 dhcp options that UCSM and Palo support. Dhcp option 17 and DHCP option 43. With DHCP option 43, you can specify a vendorId (column) and you don't need to specify any initiator name.

With Option 17 a single target name is obtained from the DHCP server.

With Option 43, you can specify intiator name, 2 targets in the dhcp config file

Harold Simon said...


Thanks for the info, I will have to set this up on the DHCP server and see how it plays with our iSCSI storage.


arlthree said...

Release Notes for 2.0(3a)

UCS Manager can now automatically auto-generate IQN identifiers for iSCSI, and validate the IQN format. (CSCtr62641)

August Belly said...

Quick question: how are you setting up the Initiator Name for that second iSCSI object?

If I try to give it the same name as the first object, UCS will, of course, tell me that initiator name is not available because it's in use.

If I give it a different name, LeftHand won't see it and allow iSCSI access until I set up a second server with the CMC. Of course, that just give me two servers within LeftHand that point back to the same physical blade.

Is the purpose of the second iSCSI connection for redundancy? If so, why don't we just simply tick the failover option in UCS when setting it up and keep it to one iSCSI NIC?

Or, is the second NIC meant for load balancing or otherwise larger pipe to the SAN?

Cheers, and thanks.

Mani said...

Hi August, We will provide an option for IQN @ a service profile level (pool or static) in a future release.
Customers want it to be flexible, abilty to choose same or different iqn for the same server.

Enabling fail over is not recommended per Cisco white papers ....

Also the RFC dosen't seem to strictly mandate it ....

Hope you find the iqn pool feature helpful ...