**How to get HTCondor and a NAT Firewall to Cooperate**
-For an HTCondor machine on which you want to do useful work either as a submit node or as an execute node, you also would like to share your new computer resources with others outside your department or division. But, you do not want crooks using your systems to wreak havoc. So, you install a firewall between your HTCondor resource and the Internet. In this description, we assume that the firewall is a separate host from the HTCondor resource. We further assume explicitly that the firewall is a bastion host running Linux, and the firewall is iptables, with Network Address Translation (NAT); this assumption allows the description to contain explicit commands to run. You or your firewall administrator should be able to translate these instructions as needed to your particular firewall installation. We are also assuming that you are not going to be using CCB.
+The setup here is that you have a HTCondor machine on which you want to do useful work either as a submit node or an execute node. You also would like to share your new computer resources with others outside your department or division, but you do not want crooks using your systems to wreak havoc. So, you install a firewall between your HTCondor resource and the Internet. In the setup below, we will assume that the firewall is a separate host from the HTCondor resource. Indeed, will assume explicitly that it is a bastion host running Linux, and the firewall is iptables, with Network Address Translation (NAT); this is to be explicit about which commands to run. Either you, or your firewall admin, should be able to translate these instructions to your firewall installation. We are also assuming that you are not going to be using CCB.
+CCB allows communication between HTCondor daemons in a private network (outgoing connections only) with daemons in a public network (bidirectional connections allowed). It therefore is not a complete solution for a case where you need daemons in two separate private networks to communicate---one or the other network must allow bidirectional connections for CCB to help. In the case described below, we want a submit node, which is in a private network, to communicate with execute nodes in other private networks, such as OSG. This is the private-to-private case that cannot be solved with CCB alone. The solution below uses port-forwarding to make the submit node effectively public. This allows the execute nodes in private networks in OSG to use CCB to have bidirectional connectivity with your submit node. Bidirectional connectivity could also be achieved without CCB by also applying the port-forwarding solution below to the execute nodes of the remote private network, which may not be possible, either because of your own security concerns or because you do not administer machines on the remote network.
-Assume that HTCondor is installed and running with the following set up:
+Let us first assume that you have HTCondor installed and running. For this, you should follow the instructions in {link: http://research.cs.wisc.edu/htcondor/manual/v7.9/3_2Installation.html section 3.2 of the HTCondor manual}.
-The submit machine, which runs the condor_schedd is installed on a machine named S with IP address 192.168.0.1.
-
-The execute machine, which runs the condor_startd is installed on a machine named E with IP address 192.168.0.2.
-
-The firewall has an Internet-facing, external IP address of 10.0.0.1.
-Its internal, ---that is, facing toward your local network---IP address of 192.168.0.250; we will call this machine F. We know 10.0.0.1 is actually not a routable address, but pretend that it is for the duration of this document. S and E are in the domain =mydomain.net=.
+Let us assume you have the following setup. The HTCondor schedd is installed on a machine (named S) with IP address 192.168.0.1; this is your submit machine. An HTCondor startd is installed on a machine (named E) at 192.168.0.2; this is your execute node. The firewall has an external---that is, facing the Internet---IP address of 10.0.0.1, and an internal---that is, facing toward your local network---IP address of 192.168.0.250; we will call this machine F. We know 10.0.0.1 is actually not a routable address, but pretend that it is for the duration of this document. S and E are in the domain =mydomain.net=.
Then we will make the following changes to =condor_config.local= on S (the schedd). To find your HTCondor configuration files, the command =condor_config_val -dump= will be a big help, as the files are listed in the header of the output
@@ -30,7 +26,7 @@
PRIVATE_NETWORK_INTERFACE = eth0
TCP_FORWARDING_HOST = 10.0.0.1
{endcode}
-The use of =PRIVATE_NETWORK_NAME= on S and E allow them to communicate _directly_ without going through the firewall F.
+The use of =PRIVATE_NETWORK_NAME= on S and E allow them to communicate _directly_ without going through the firewall F. The port choice of 9616 is arbitrary.
Now, on the firewall F, we run the following commands to redirect connections from the Internet to ports 9617 and 9616 on F to the corresponding ports on S and E:
{code}
@@ -42,8 +38,8 @@
Finally, run =condor_reconfig= on S and E to incorporate the configuration changes.
-One may well ask how this works for HTCondor. Simply put, HTCondor wraps all the information in a "sinful string". =condor_status -schedd -l= will return a Schedd ClassAd, which will contain a line that looks something like:
+One may well ask how this works for HTCondor. Simply put, HTCondor wraps all the information in an string with all the address information. =condor_status -schedd -l= will return a Schedd ClassAd, which will contain a line that looks something like:
{code}
MyAddress = "<10.0.0.1:9617?PrivAddr=%3c192.168.0.1:9617%3fsock%3d936_480b_8%3e&PrivNet=mydomain.net&noUDP&sock=936_480b_8>"
{endcode}
-This string contains the needed information for another HTCondor client or daemon to contact the schedd and begin using high throughput computing.
+This string contains the needed information for another HTCondor client or daemon to contact the schedd and begin using high throughput computing. Examining this string may be helpful in debugging if you are unable to connect to remote HTCondor services.
