How to Proxy Git Connections
using socat to ...Git... through a corporate firewall
One of the first things you will notice when you start developing open
source software for a larger corporation is the company firewall or
proxy blocking all outgoing traffic not routed through their network
appliance for inspection and review first. This sadly includes all of
our git fetch
traffic.
Before we can do anything upstream we first need to configure Git to pull in updates through these corporate firewalls. This is where the multipurpose relay tool, socat, shines.
Proxy http(s) Remotes
Git uses the Smart HTTP protocol to download updates from a remote
that starts with https://*
URI. These type of connections have
become, by far, the most popular way to use Git today and is the default
protocol used by many popular forges like GitHub or GitLab.
Internally Git relies on the libcurl library to handle these HTTP connections which means Git comes with the ability to proxy these requests built-in.
It also means that Git will respect your system's http_proxy
and
https_proxy
environment variables. Simply add something like
this to your ~/.bashrc
or execute these lines in your shell
before you need to pull updates from your remote project and Git will
proxy the requests through your corporate firewall appliance
automatically.
$ grep 'https\?_proxy' ~/.bashrc
export http_proxy=http://mega.co.proxy.example.com:8080/
export https_proxy=http://mega.co.proxy.example.com:8080/
Alternatively, if you want to keep all your Git configurations in one
place, you can add this to our ~/.gitconfig
to configure Git's
proxy settings:
[http]
proxy = http://mega.co.proxy.example.com:8080
If you only need to proxy the repositories hosted outside of the
company network, you can use a pattern like this to filter the domains
that use the proxy
configuration settings.
For example any repository hosted on kernel.org servers, use:
[http]
[http "https://git.kernel.org"]
proxy = http://mega.co.proxy.example.com:8080
For more details on proxy-ing http based Git projects, check-out the
--proxy
option in the curl(1) man page, or the
http.proxy
entry in the git-config documentation.
Proxy Git Remotes
Another common way to fetch updates from a remote tree is by using their
git://*
URI which uses the Git Server Protocol. Often used to
share projects that do not require any user authentication as it is the
fastest of the three transfer protocols available.
To proxy these types of requests we will need Git to run a simple shell
script, like the example below, any time we fetch a remote repository.
This example script will proxy all repositories outside of the
example.com
domain.
$ cat ~/bin/gitproxy
#!/usr/bin/sh
if [ $1 == *example.com ]; then
# the repository is outside the company network, proxy it
socat tcp:mega.co.proxy.example.com:8080:$1:$2
else
# no need to proxy internal network traffic
socat tcp:$1:$2
fi
Then, to have Git run this script, we can set the gitProxy
setting in our ~/.gitconfig
like this:
[core]
gitProxy = ~/bin/gitproxy
Because this is just a simple shell script, we can make this as complicated or as simple as we want. For example, you can add more logic if you use a laptop that may need work outside of the office.
Proxy SSH Remotes
Finally, if the remote starts with ssh://*
or uses a typical ssh
connection like git@git.sr.ht:*
we are connecting via the SSH
protocol. These connections are typically used when you maintain the
remote repository or otherwise need to authenticate yourself before you
can git push
changes to the server.
In these situations, Git relies on SSH to handle the authentication and
connection to the remote, which means we will need to edit our
~/.ssh/config
file to proxy these types of connections.
Inside OpenSSH's configuration file, we can use the ProxyCommand
option for all remote excluding any URI inside our company's network.
Host * !*.mega.co.example.com
User git
ProxyCommand socat tcp:mega.co.proxy.example.com:8080:%h:%p
Or we can manually add the remotes we want proxied. For example any remote at git.kernel.org:
Host git.kernel.org
User git
ProxyCommand socat tcp:mega.co.proxy.example.com:8080:%h:%p
For more information about OpenSSH's config file options, checkout the ssh_config(5) man page.
Wrapping Up
I hope these notes were of some use or helped you in some way. I recently accepted my first open source role working for a larger company (>10,000 people) and wanted to write these notes down in the hopes it would help others working in open source projects behind walled gardens.
As always if you see something that wasn't correct or I need to update or clarify, please feel free to send me a note any kind of way that is convenient to you.
~Bryan