Proxmox VE and Shorewall Part 2
Shorewall and Proxmox VE Cluster Configuration
This is a follow up article describing how to use Proxmox VE and Shorewall together. This article focus on using Shorewall within your Proxmox cluster. If you have not read the first article I recommend that you do so, it will aid your understanding with what is going in this one.
Network Layout and Shorewall Configuration
We are going to be using a bridging configuration. This is what Proxmox VE uses with by default. Bridging allows for easy migration of hosts without having to re-configure the firewall each time a machine is migrated.
The default Proxmox network configuration looks similar to this:
iface eth0 inet manual auto vmbr0 iface vmbr0 inet static address 192.0.2.10 netmask 255.255.255.0 gateway 192.0.2.1 bridge_ports eth0 bridge_stp off bridge_fd 0
Shorewall 4.4.3 will only support one bridge device. This is not a huge drawback and add several advantages. The first advantage that having a single bridge device adds is a more abstracted network layout. You have cluster nodes that have a different number of network interfaces and still be free to migrate between them and know that your virtual machine will continue to work. With a cluster setup this is a very important property. Proxmox allows you to bond multiple NICs together so that they look as one logical interface. I personally use 802.3ad bonding mode because my switches support it. This bonding mode gives you increased throughput and redundancy. Other bonding modes will work as well. If you think you need multiple bridges you don't; there is a solution using VLANs. We will discuss VLANs more later in this article.
In the last article we just used the Shorewall version that was in the repos; Shorewall version 4.0.15. That version of Shorewall does not have all the features we need. Thankfully the package maintainer has provided an up to date package on his personal site. Intrusions on how to install that can be found here and work perfectly with Proxmox VE.
The configuration files for Shorewall are stored in the /etc/shorewall/ directory. To get started, copy the example configuration files from the ones installed with the Shorewall package.
cp /usr/share/doc/shorewall/default-config/* /etc/shorewall/
cp /usr/share/doc/shorewall/examples/two-interfaces/* /etc/shorewall/
These files give you a good base to begin customizing the firewall to meet your needs.
First thing is to edit the shorewall.conf file. You want to enable IP forwarding. Change it from Keep to On. Also make sure that startup is enabled.
IP_FORWARDING=On STARTUP_ENABLED=Yes Then edit the zones file: fw firewall world ipv4 net:world bport dmz:world bport vz ipv4
The zones file defines what zones are available so that you can make rules and policies based on them.
Next edit the interfaces file:
world vmbr0 detect bridge net vmbr0:bond0 dmz vmbr0:vmtab+ vz venet0 detect routeback
This file just tells Shorewall what interfaces are connected to what zones. This also allows you to make policies based where traffic is traveling to or from. The policy file is where you define what those policies are. In this article the following polices will be defined.
Traffic from the firewall:
1.To the Internet is permitted
2.From the Internet is prohibited
Traffic from the DMZ:
1.To the Internet is permitted
2.To the firewall is prohibited
# Internet Connections dmz net ACCEPT vz world ACCEPT # Allow FW to use internet $FW world ACCEPT net all DROP info # THE FOLLOWING POLICY MUST BE LAST all all REJECT info
Next we need to define rules. Rules are the exceptions to the policies defined above. We are just going to create a very basic set. You can easily expand on these on your own.
# Accept SSH connections for administration # SSH(ACCEPT) net $FW SSH(ACCEPT) dmz $FW SSH(ACCEPT) vz $FW # Permit access to Proxmox Manager and Console ACCEPT net $FW tcp 5900:5999 HTTPS(ACCEPT) net $FW HTTP(ACCEPT) net $FW # # Allow Ping from the local network # Ping(ACCEPT) dmz $FW Ping(ACCEPT) vz $FW Ping(ACCEPT) net dmz Ping(ACCEPT) net vz # # Drop Ping from the "bad" net zone.. and prevent your log from being flooded.. # Ping(DROP) net $FW # # # VMID: 120 # Type: KVM # Name: test0.example.com # IP: 192.0.2.11 # HTTP(ACCEPT) net dmz:192.0.2.11 # VMID: 113 # Type: OpenVZ # Name: test1.example.com # IP: 192.0.2.12 # HTTP(ACCEPT) net vz:192.0.2.12
This set of rules allows SSH and Ping to work from the net zone. The last part of the file shows a web server running at 192.0.2.11 and at 192.0.2.12. As you can see the syntax is slightly different for a KVM machine versus a OpenVZ machine.
We then need to make some changes from the default two interface setup. We need to edit the masq file and change it to fit your environment. I am not using masquerading so I removed the entries in that file. We also need to edit the routestopped file again customize this to your environment. See http://shorewall.net/Documentation.htm#Routestopped for more information.
The last things to do are to check your syntax using: shorewall check and to try your firewall out using: shorewall try /etc/shorewall 60 The first command just does basic checks of your syntax and shows any errors. The second command apply your Shorewall configuration for 60 seconds for you try out make sure that everything works as expected. If you lock yourself out you just have to wait until the 60 seconds is over and then you are able to get back in. Once you are satisfied that everything is working edit the /etc/default/shorewall file and change startup=0 to startup=1.
The issue with having more than one bridge with Proxmox VE and Shorewall is all the bridges generated have names that all start off the same way. There is a patch to Shorewall that allows you to define multiple bridges as discussed here: http://tinyurl.com/ykb33jr However this is not an ideal solution. The ideal solution would be to change the way Proxmox names the generated bridge ports to reflect what bridge on the host they are connecting to. Another workaround is to just define VLANs and use those VLANs within your virtual machine. Defining VLANs is distro dependent so I won't go into it but by using VLANs you are able to migrate your virtual machine to any cluster node and have it work as long as your switch port is part of those VLANs. Also as of Shorewall version 4.4.4 it is possible to specify multiple bridges.
Applying the Firewall Cluster Wide
Keeping your configuration files in sync cluster wide can easily be done with an rsync script or even just with scp. If there is interest I will write part 3 that keeps your Shorewall files updated with Puppet (http://reductivelabs.com/trac/puppet/). So leave me a comment to see that.
The final gotcha is you need to restart the firewall every time a virtual machine is migrated. I have not found a good way to do this yet but for right now just doing it by hand is working. If anybody has ideas on a way to have it happen automatically I would be interested to hear.
Linux Systems Administrator
Rocky Mountain College