Working local DNS for your libvirtd guests

Update 2017-12-25: possibly better way: Definitive solution to libvirt guest naming

This is basically just a copy&paste of commands from this great post: [Howto] Automated DNS resolution for KVM/libvirt guests with a local domain and Automatic DNS updates from libvirt guests which already saved me a lots of typing. So with my favorite domain:

Make libvirtd's dnsmasq to act as authoritative nameserver for example.com domain:

# virsh net-dumpxml default
<network>
  <name>default</name>
  <uuid>2ed15952-d1c0-4819-bde5-c8f7278ce3ac</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:a4:40:a7'/>
  <domain name='example.com' localOnly='yes'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>

And restart that network:

# virsh net-edit default   # do the edits here
# virsh net-destroy default
# virsh net-start default

Now configure NetworkManager to start its own dnsmasq which acts like your local caching nameserver and forwards all requests for example.com domain to 192.168.122.1 nameserver (which is libvirtd's dnsmasq):

# cat /etc/NetworkManager/conf.d/localdns.conf
[main]
dns=dnsmasq
# cat /etc/NetworkManager/dnsmasq.d/libvirt_dnsmasq.conf
server=/example.com/192.168.122.1

And restart NetworkManager:

# systemctl restart NetworkManager

Now if I have guest with hostname set (check HOSTNAME=... in /etc/sysconfig/network on RHEL6 and below or hostnamectl set-hostname ... on RHEL7) to "satellite.example.com", I can ping it from both virtualization host and another guests on that host by hostname. If you have some old OS release on the guest (like RHEL 6.5 from what I have tried, 6.8 do not need this), set hostname with DHCP_HOSTNAME=... in /etc/sysconfig/network-scripts/ifcfg-eth0 (on the guest) to make this to work.

Enabling 10Gb networking on RHEL7 in AWS cloud (some instances types)

Even when you order recent RHEL 7.2 in some beefy Amazon EC2 instance type (like "c4.8xlarge") which have "enhanced networking capabilities", you still have to do some manual steps described in Amazon's docs (note there are actually two ways - based on the instance type you will choose). Basically you need newer network driver than what is in default installation. First check your current driver:

We would like to have "ixgbevf" here:

# ethtool -i eth0 | grep '^driver'
driver: vif

And here we would like to have at least "2.14.2" (ignore my actual version):

# modinfo ixgbevf | grep '^version'
version:        2.12.1-k-rh7.3

So lets go on (as root on the instance) - we will take newest version in Intel Ethernet Drivers and Utilities:

curl -o ixgbevf-3.2.2.tar.gz 'http://netcologne.dl.sourceforge.net/project/e1000/ixgbevf%20stable/3.2.2/ixgbevf-3.2.2.tar.gz'   # URL might be different for you, follow the download button on the SourceForge site
yum -y install kernel-devel gcc rpm-build   # we will need these to compile
rpmbuild -tb ixgbevf-3.2.2.tar.gz   # specfile is inside of the tarball
rpm -ivh /root/rpmbuild/RPMS/x86_64/ixgbevf-3.2.2-1.x86_64.rpm   # install resulting rpm
cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.$(date +%m-%d-%H%M%S).ORIG   # backup current initrd
dracut -f -v   # rebuild initrd so it contains our module
shutdown -h now   # stop the instance

Now we will need to run one command via aws command line tool. If you do not have it installed, like me (I'm on Fedora 24), you can install it with:

mkdir aws
cd aws
virtualenv .
. bin/activate
pip install aws
aws configure

And now we can finally run the command which enables some fancy attribute for your instance (they say in the docs this can not be undone):

aws ec2 modify-instance-attribute --instance-id <instanceID> --sriov-net-support simple

Now start the instance. Warning: instance had different public IP (and DNS hostname) now in my case, so do not blindly attempt to connect to previous hostname. Lets check what we have on the system now:

# modinfo ixgbevf | grep '^version'
version:        3.2.2
# ethtool -i eth0 | grep -e '^driver' -e '^version'
driver: ixgbevf
version: 3.2.2

This looks good, although I have not tested real performance yet. Going to turn off that expensive machine now :-)

Verify that a programme is communicating through proxy only

I had to verify, that a programme on a remote server is communicating through proxy only, while there were lots of other services on the server running (and communicating over the network). While I could watch proxy's (squid) logs, setup firewall to log access to and from certain hosts or use iftop. In my case these had various down-sides (e.g. iftop is more to track amount of traffic and I had to check that even the smallest packet wont bypass mine http proxy - even if you can pass filters mentioned below to iftop as well - see -f option). I have chosen tcpdump, and this post is to save exact command I have used:

tcpdump -i any "tcp and host not proxy.example.com and host not my-workstation.example.com and not ( dst localhost and src localhost ) and not ( dst $( hostname ) and src $( hostname ) )"

• tcp says that I'm interested in TCP traffic only
• host not proxy.example.com instructs tcpdump to ignore (should not log) any traffic to/from my proxy server
• host not my-workstation.example.com asks tcpdump to ignore traffic to/from my workstation as I'm connected via ssh from there (it could be hardened to only ignore ssh traffic - port 22, but this is good enough for me)
• not ( dst localhost and src localhost ) ignore traffic going from localhost to localhost (some other services on the system are talking to each other and I'm not interested in it)
• not ( dst $( hostname ) and src $( hostname ) ) same as above, but some services are using my external IP for their internal discussions and again, I do not need to know about that

This way tcpdump only logs communication from/to parts of external world I'm interested about.