One of the first things you will notice when you start developing open
source software for a larger corporation is the company firewall or
proxy blocking all outgoing traffic not routed through their network
appliance for inspection and review first. This sadly includes all of
git fetch traffic.
Before we can do anything upstream we first need to configure Git to pull in updates through these corporate firewalls. This is where the multipurpose relay tool, socat, shines.
Proxy http(s) Remotes
Git uses the Smart HTTP protocol to download updates from a remote
that starts with
https://* URI. These type of connections have
become, by far, the most popular way to use Git today and is the default
protocol used by many popular forges like GitHub or GitLab.
Internally Git relies on the libcurl library to handle these HTTP connections which means Git comes with the ability to proxy these requests built-in.
It also means that Git will respect your system's
https_proxy environment variables. Simply add something like
this to your
~/.bashrc or execute these lines in your shell
before you need to pull updates from your remote project and Git will
proxy the requests through your corporate firewall appliance
$ grep 'https\?_proxy' ~/.bashrc export http_proxy=http://mega.co.proxy.example.com:8080/ export https_proxy=http://mega.co.proxy.example.com:8080/
Alternatively, if you want to keep all your Git configurations in one
place, you can add this to our
~/.gitconfig to configure Git's
[http] proxy = http://mega.co.proxy.example.com:8080
If you only need to proxy the repositories hosted outside of the
company network, you can use a pattern like this to filter the domains
that use the
proxy configuration settings.
For example any repository hosted on kernel.org servers, use:
[http] [http "https://git.kernel.org"] proxy = http://mega.co.proxy.example.com:8080
Proxy Git Remotes
Another common way to fetch updates from a remote tree is by using their
git://* URI which uses the Git Server Protocol. Often used to
share projects that do not require any user authentication as it is the
fastest of the three transfer protocols available.
To proxy these types of requests we will need Git to run a simple shell
script, like the example below, any time we fetch a remote repository.
This example script will proxy all repositories outside of the
$ cat ~/bin/gitproxy #!/usr/bin/sh if [ $1 == *example.com ]; then # the repository is outside the company network, proxy it socat tcp:mega.co.proxy.example.com:8080:$1:$2 else # no need to proxy internal network traffic socat tcp:$1:$2 fi
Then, to have Git run this script, we can set the
setting in our
~/.gitconfig like this:
[core] gitProxy = ~/bin/gitproxy
Because this is just a simple shell script, we can make this as complicated or as simple as we want. For example, you can add more logic if you use a laptop that may need work outside of the office.
Proxy SSH Remotes
Finally, if the remote starts with
ssh://* or uses a typical ssh
firstname.lastname@example.org:* we are connecting via the SSH
protocol. These connections are typically used when you maintain the
remote repository or otherwise need to authenticate yourself before you
git push changes to the server.
In these situations, Git relies on SSH to handle the authentication and
connection to the remote, which means we will need to edit our
~/.ssh/config file to proxy these types of connections.
Inside OpenSSH's configuration file, we can use the
option for all remote excluding any URI inside our company's network.
Host * !*.mega.co.example.com User git ProxyCommand socat tcp:mega.co.proxy.example.com:8080:%h:%p
Or we can manually add the remotes we want proxied. For example any remote at git.kernel.org:
Host git.kernel.org User git ProxyCommand socat tcp:mega.co.proxy.example.com:8080:%h:%p
For more information about OpenSSH's config file options, checkout the ssh_config(5) man page.
I hope these notes were of some use or helped you in some way. I recently accepted my first open source role working for a larger company (>10,000 people) and wanted to write these notes down in the hopes it would help others working in open source projects behind walled gardens.
As always if you see something that wasn't correct or I need to update or clarify, please feel free to send me a note any kind of way that is convenient to you.