Wanting to expand the number of NICs from 4 to 8, I had an Intel Gigabit ET quad port adaptor laying around, so put it to use.
After physically installing the PCI express x4 card in the server and booting ESX 4.0 U1, I found it had not picked up the NICs. It appeared since this is avery new card, the driver is not in the default drivers of ESX 4.0 U1, so I looked closer...
Displaying the NICs only showed only the builtin Broadcom NICs, not the Intel.
/usr/sbin/esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 01:00.00 bnx2 Up 1000Mbps Full 00:24:e8:4a:15:20 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic1 01:00.01 bnx2 Up 1000Mbps Full 00:24:e8:4a:15:22 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic2 02:00.00 bnx2 Up 1000Mbps Full 00:24:e8:4a:15:24 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic3 02:00.01 bnx2 Up 1000Mbps Full 00:24:e8:4a:15:26 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
I checked to see if the intel gigabit module was loaded, which it wasnt.
/usr/sbin/esxcfg-module -l | grep igb
Listing the PCI devices showed the Intel card was found but not fully recognized:
/sbin/lspci
00:00.0 Host bridge: Intel Corporation Unknown device 3406 (rev 13)
00:01.0 PCI bridge: Intel Corporation Unknown device 3408 (rev 13)
00:03.0 PCI bridge: Intel Corporation Unknown device 340a (rev 13)
00:04.0 PCI bridge: Intel Corporation Unknown device 340b (rev 13)
00:05.0 PCI bridge: Intel Corporation Unknown device 340c (rev 13)
00:06.0 PCI bridge: Intel Corporation Unknown device 340d (rev 13)
00:07.0 PCI bridge: Intel Corporation Unknown device 340e (rev 13)
00:09.0 PCI bridge: Intel Corporation Unknown device 3410 (rev 13)
00:14.0 PIC: Intel Corporation Unknown device 342e (rev 13)
00:14.1 PIC: Intel Corporation Unknown device 3422 (rev 13)
00:14.2 PIC: Intel Corporation Unknown device 3423 (rev 13)
00:1a.0 USB Controller: Intel Corporation Unknown device 2937 (rev 02)
00:1a.1 USB Controller: Intel Corporation Unknown device 2938 (rev 02)
00:1a.7 USB Controller: Intel Corporation Unknown device 293c (rev 02)
00:1d.0 USB Controller: Intel Corporation Unknown device 2934 (rev 02)
00:1d.1 USB Controller: Intel Corporation Unknown device 2935 (rev 02)
00:1d.7 USB Controller: Intel Corporation Unknown device 293a (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801BA/CA/DB/EB PCI Bridge (rev92)
00:1f.0 ISA bridge: Intel Corporation Unknown device 2918 (rev 02)
00:1f.2 IDE interface: Intel Corporation 2 port SATA IDE Controller(ICH9) (rev 02)
01:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
01:00.1 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
02:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
02:00.1 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
03:00.0 RAID bus controller: LSI Logic / Symbios Logic LSI MegaRAID SAS 1078 Controller (rev 04)
04:00.0 Fibre Channel: QLogic Corp ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
05:00.0 Fibre Channel: QLogic Corp ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
06:00.0 PCI bridge: Integrated Device Tech Unknown device 8018 (rev 0e)
07:02.0 PCI bridge: Integrated Device Tech Unknown device 8018 (rev 0e)
07:04.0 PCI bridge: Integrated Device Tech Unknown device 8018 (rev 0e)
08:00.0 Ethernet controller: Intel Corporation Unknown device 10e8 (rev01)
08:00.1 Ethernet controller: Intel Corporation Unknown device 10e8 (rev01)
09:00.0 Ethernet controller: Intel Corporation Unknown device 10e8 (rev01)
09:00.1 Ethernet controller: Intel Corporation Unknown device 10e8 (rev01)
0b:03.0 VGA compatible controller: Matrox Graphics, Inc. Unknown device0532 (rev 0a)
To fix this download VMware ESX/ESXi 4.0 Driver CD for Intel 82575 and 82576 Gigabit Ethernet Controller
http://downloads.vmware.com/d/details/esx_esxi40_intel_82575_82576_dt/ZHcqYmR0QGpidGR3
Burn the ISO to a CD.
On the ESX server insert the CD and on the ESX service console mount the CD:
mount /mnt/cdrom/
Check the CD is mounted:
mount
/dev/sdk5 on / type ext3 (rw)
None on /proc type proc (rw)
None on /sys type sysfs (rw)
None on /dev/pts type devpts (rw)
/dev/sda1 on /boot type ext3 (rw)
/dev/sdk1 on /var/log type ext3 (rw,errors=panic)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/scd0 on /mnt/cdrom type iso9660 (ro,nosuid,nodev,_netdev)
Change the directory to the location of the update/driver zip file:
cd /mnt/cdrom/offline-bundle/
Run ESXUPDATE to install the driver:
/usr/sbin/esxupdate --bundle=INT-intel-lad-ddk-igb-1.3.19.12.1-offline_bundle-185976.zip update
INT-intel-lad-ddk-igb-1.3.19.12.1-offline_bundle-185976.zip
################################################################# [100%]
Unpacking vmware-esx-drivers-net-igb-400.1.3.19.12.1-1.0.4...
################################################################# [100%]
Installing vmware-esx-drivers-net-igb
################################################################# [100%]
Cleaning up vmware-esx-drivers-net-igb
################################################################# [100%]
unning [/usr/sbin/vmkmod-install.sh]...
ok.
The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot the server (Note: Ensure in maintenance mode with no VMs running):
reboot
After the server boots check the update is installed:
/usr/sbin/esxupdate query
-----------Bulletin ID----------- -----Installed----- --------------------------Summary---------------------------
ESX400-Update01a 2009-12-29T08:34:22 VMware ESX 4.0 Complete Update 1
ESX400-200912401-BG 2010-02-01T16:54:22 Updates vmx, vmkernel etc
ESX400-200912402-SG 2010-02-01T16:54:22 Updates OpenSSL
ESX400-200912403-SG 2010-02-01T16:54:22 Updates NSS and NSPR
ESX400-200912404-SG 2010-02-01T16:54:22 Updates DHCP
ESX400-201002401-BG 2010-03-04T08:42:15 Updates vmkernel64,vmx,hostd etc
ESX400-201002402-BG 2010-03-04T08:42:15 Updatesinitscripts
ESX400-201002404-SG 2010-03-04T08:42:15 Updates glib2
ESX400-201002405-BG 2010-03-04T08:42:15 Updates megaraid-sas
ESX400-201002406-SG 2010-03-04T08:42:15 Updates newt
ESX400-201002407-SG 2010-03-04T08:42:15 Updates nfs-utils
ESX400-201002408-BG 2010-03-04T08:42:15 Updates Enic driver
INT-intel-lad-ddk-igb-1.3.19.12.1 2010-04-07T12:53:39 igb: Intel(R) Gigabit Ethernet Network Driver for VMware ESX
The module should now be loaded correctly:
/usr/sbin/esxcfg-module -l | grep igb
igb 0x418029e76000 0x17000 0x417feacc8340 0x1000 1d Yes
The PCI devices (1-4 depending if single, dual or quad port) should now be named correctly:
/sbin/lspci
00:00.0 Host bridge: Intel Corporation Unknown device 3406 (rev 13)
00:01.0 PCI bridge: Intel Corporation Unknown device 3408 (rev 13)
00:03.0 PCI bridge: Intel Corporation Unknown device 340a (rev 13)
00:04.0 PCI bridge: Intel Corporation Unknown device 340b (rev 13)
00:05.0 PCI bridge: Intel Corporation Unknown device 340c (rev 13)
00:06.0 PCI bridge: Intel Corporation Unknown device 340d (rev 13)
00:07.0 PCI bridge: Intel Corporation Unknown device 340e (rev 13)
00:09.0 PCI bridge: Intel Corporation Unknown device 3410 (rev 13)
00:14.0 PIC: Intel Corporation Unknown device 342e (rev 13)
00:14.1 PIC: Intel Corporation Unknown device 3422 (rev 13)
00:14.2 PIC: Intel Corporation Unknown device 3423 (rev 13)
00:1a.0 USB Controller: Intel Corporation Unknown device 2937 (rev 02)
00:1a.1 USB Controller: Intel Corporation Unknown device 2938 (rev 02)
00:1a.7 USB Controller: Intel Corporation Unknown device 293c (rev 02)
00:1d.0 USB Controller: Intel Corporation Unknown device 2934 (rev 02)
00:1d.1 USB Controller: Intel Corporation Unknown device 2935 (rev 02)
00:1d.7 USB Controller: Intel Corporation Unknown device 293a (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801BA/CA/DB/EB PCI Bridge (rev 92)
00:1f.0 ISA bridge: Intel Corporation Unknown device 2918 (rev 02)
00:1f.2 IDE interface: Intel Corporation 2 port SATA IDE Controller (ICH9) (rev 02)
01:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
01:00.1 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
02:00.0 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
02:00.1 Ethernet controller: Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T (rev 20)
03:00.0 RAID bus controller: LSI Logic / Symbios Logic LSI MegaRAID SAS 1078 Controller (rev 04)
04:00.0 Fibre Channel: QLogic Corp ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
05:00.0 Fibre Channel: QLogic Corp ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
06:00.0 PCI bridge: Integrated Device Tech Unknown device 8018 (rev 0e)
07:02.0 PCI bridge: Integrated Device Tech Unknown device 8018 (rev 0e)
07:04.0 PCI bridge: Integrated Device Tech Unknown device 8018 (rev 0e)
08:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
08:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
09:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
09:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
0b:03.0 VGA compatible controller: Matrox Graphics, Inc. Unknown device 0532 (rev 0a)
Finally listing the NICs with ESXCFG-NICS shows the devices and can now be used for mapping to vSwitches:
/usr/sbin/esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 01:00.00 bnx2 Up 1000Mbps Full 00:24:e8:4a:15:20 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic1 01:00.01 bnx2 Up 1000Mbps Full 00:24:e8:4a:15:22 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic2 02:00.00 bnx2 Up 1000Mbps Full 00:24:e8:4a:15:24 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic3 02:00.01 bnx2 Up 1000Mbps Full 00:24:e8:4a:15:26 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic4 08:00.00 igb Down 0Mbps Half 00:1b:21:47:0a:f0 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic5 08:00.01 igb Down 0Mbps Half 00:1b:21:47:0a:f1 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic6 09:00.00 igb Down 0Mbps Half 00:1b:21:47:0a:f4 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic7 09:00.01 igb Down 0Mbps Half 00:1b:21:47:0a:f5 1500 Intel Corporation 82576 Gigabit Network Connection
Share this blog post on social media:
TweetAll advice, installation/configuration how to guides, troubleshooting and other information on this website are provided as-is with no warranty or guarantee. Whilst the information provided is correct to the best of my knowledge, I am not reponsible for any issues that may arise using this information, and you do so at your own risk. As always before performing anything; check, double check, test and always ensure you have a backup.