2016-07-28

Enabling 10Gb networking on RHEL7 in AWS cloud (some instances types)

Even when you order recent RHEL 7.2 in some beefy Amazon EC2 instance type (like "c4.8xlarge") which have "enhanced networking capabilities", you still have to do some manual steps described in Amazon's docs (note there are actually two ways - based on the instance type you will choose). Basically you need newer network driver than what is in default installation. First check your current driver:

We would like to have "ixgbevf" here:

# ethtool -i eth0 | grep '^driver'
driver: vif

And here we would like to have at least "2.14.2" (ignore my actual version):

# modinfo ixgbevf | grep '^version'
version:        2.12.1-k-rh7.3

So lets go on (as root on the instance) - we will take newest version in Intel Ethernet Drivers and Utilities:

curl -o ixgbevf-3.2.2.tar.gz 'http://netcologne.dl.sourceforge.net/project/e1000/ixgbevf%20stable/3.2.2/ixgbevf-3.2.2.tar.gz'   # URL might be different for you, follow the download button on the SourceForge site
yum -y install kernel-devel gcc rpm-build   # we will need these to compile
rpmbuild -tb ixgbevf-3.2.2.tar.gz   # specfile is inside of the tarball
rpm -ivh /root/rpmbuild/RPMS/x86_64/ixgbevf-3.2.2-1.x86_64.rpm   # install resulting rpm
cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.$(date +%m-%d-%H%M%S).ORIG   # backup current initrd
dracut -f -v   # rebuild initrd so it contains our module
shutdown -h now   # stop the instance

Now we will need to run one command via aws command line tool. If you do not have it installed, like me (I'm on Fedora 24), you can install it with:

mkdir aws
cd aws
virtualenv .
. bin/activate
pip install aws
aws configure

And now we can finally run the command which enables some fancy attribute for your instance (they say in the docs this can not be undone):

aws ec2 modify-instance-attribute --instance-id <instanceID> --sriov-net-support simple

Now start the instance. Warning: instance had different public IP (and DNS hostname) now in my case, so do not blindly attempt to connect to previous hostname. Lets check what we have on the system now:

# modinfo ixgbevf | grep '^version'
version:        3.2.2
# ethtool -i eth0 | grep -e '^driver' -e '^version'
driver: ixgbevf
version: 3.2.2

This looks good, although I have not tested real performance yet. Going to turn off that expensive machine now :-)

No comments:

Post a Comment