Building an OpenBGPD bgplg Looking Glass from Scratch

Anybody who’s spent time troubleshooting BGP advertisements and/or Internet reachability issues has probably made use of one or many looking glass servers. These are servers, usually managed and maintained by ISPs, which provide a set of useful troubleshooting tools (ping, traceroute) and outputs (show ip bgp) that allow you to view the Internet from the perspective of their network. Large, geographically diverse carriers often allow you to select a router you would like to troubleshoot from, which is useful if the issue you are troubleshooting seems to be localized to a specific area.

As a network guy, I’ve spent A LOT of time on various looking glasses, and I’ve even set a few up — albeit many years ago. Internet routing tables can be chaotic at times and, in my opinion, there can never be too many looking glasses. So when the time came to set up a new looking glass at my day job, I figured it would be a good opportunity to take some notes to make it easier for the next guy or gal to get who is willing to show the world what the Internet looks like from their perspective. After a bit of research, I decided that I would install bgplg, a clean, robust OpenBGPD add-on.

Get the OS

I installed OpenBSD 5.2, which can be freely downloaded here

Install the OS

OpenBSD is a very lightweight OS, so I probably went overkill when I configured my VMWare guest with 1GB RAM, 2 CPUs and 15GB HDD space.

The installer is also very lightweight — just select (I)nstall when it asks and start walking through the prompts. I followed the defaults until it asked if I’d like to install X Windows. There’s no need for it on this server so, unless extenuating circumstances warrant it, I’d select "no". A few more questions and then it will ask to install sets. I used the CD as my source and, in keeping with the ideal of a minimal install, removed packages I felt were unnecessary: type "-x*", which will remove all X-related packages and then "-game*" to remove the game set. Simply press Enter when done selecting sets and the install will begin.

When installation is complete, eject your install media and type "reboot".

(Optional) Add an IPv6 Default Gateway

If you’ve configured IPv6, you will notice that there is no default IPv6 route installed. To fix this, edit your network config file (mine is /etc/hostname.em0) and add the following line to the end:

!route -n add -inet6 default <IPv6 default gateway>

When added & saved, you can activate your changes by running the netstart script:

# sh /etc/netstart

Configure bgplg & Apache

You’ll now want to enable bgplg, which is installed by default. The next steps are straight out of the man page:

# chmod 0555 /var/www/cgi-bin/bgplg
          # chmod 0555 /var/www/bin/bgpctl
          # mkdir /var/www/etc
          # cp /etc/resolv.conf /var/www/etc
          # chmod 4555 /var/www/bin/ping
          # chmod 4555 /var/www/bin/ping6
          # chmod 4555 /var/www/bin/traceroute
          # chmod 4555 /var/www/bin/traceroute6

Change the following values in /etc/rc.conf to start OpenBGPD & Apache at boot time:

bgpd_flags=""
          httpd_flags=""

Note: The default bgplg install runs in at /cgi-bin/bgplg in your Apache document root. Since my server only serves up the bgplg page, I tweaked the DirectoryIndex parameter in my Apache config (`/var/www/conf/httpd.conf1) to the following:

DirectoryIndex /cgi-bin/bgplg

After the change is made, you can reload Apache by running:

apachectl reload

If this setting is not changed, you can still access your looking glass at http://[Your IP]/cgi-bin/bgplg

OpenBSD runs Apache in a chroot. We tweaked a lot of permissions above to give the chroot environment access to basic network tools, but by default things like DNS lookups will still not work. To test DNS resolution and the ping tool from your chroot environment, you can run the following command:

# chroot -g www -u www /var/www /bin/ping google.com

which will likely result in the following error:

ping: socket: Permission denied

This is a result of the nosuid flag being set on the /var partition by default. You can verify this by looking at how /var is currently mounted:

# mount | grep var
          /dev/sd0e on /var type ffs (local, nodev, nosuid)

To fix this problem, remove the nosuid flag from the /var partition in <code>/etc/fstab</code> and reboot your machine. Your should now see something similar to:

# mount | grep var
          /dev/sd0e on /var type ffs (local, nodev)

Verify that your chrooted user is now working:

# chroot -g www -u www /var/www /bin/ping google.com
          PING google.com (74.125.129.100): 56 data bytes
          64 bytes from 74.125.129.100: icmp_seq=0 ttl=49 time=57.249 ms
          64 bytes from 74.125.129.100: icmp_seq=1 ttl=49 time=56.861 ms
          --- google.com ping statistics ---
          2 packets transmitted, 2 packets received, 0.0% packet loss
          round-trip min/avg/max/std-dev = 56.861/57.055/57.249/0.194 ms

Also, Apache should be up and running after the reboot. Even though we haven’t configured OpenBGPD yet, you should still be able to test ping & traceroute functionality.

Configure OpenBGPD

Finally, we’ll configure OpenBGPD. To do this, edit the `/etc/bgpd.conf`file and enter the relevant bgpd config information (see the man page for help). My config looks like:

#macros
          peer1="198.51.100.1"
          peer2="2001:DB8::1"
          
          # global configuration
          AS 65000
          router-id 198.51.100.2
          
          # restricted socket for bgplg(8)
          socket "/var/www/logs/bgpd.rsock" restricted
          
          # neighbors and peers
          group "peering AS65000" {
                  remote-as 65000
                  neighbor $peer1 {
                          descr   "IBGP-V4"
                          local-address 198.51.100.2
                  }
                  neighbor $peer2 {
                          descr "IBGP-V6"
                          local-address 2001:DB8::2
                  }
          }

Once you’re done configuring, you can restart OpenBGPD:

/etc/rc.d/bgpd restart

And that’s it. Next step: create/update your ASN’s PeeringDB record with your sexy new looking glass URL.

Ken Mix - December 13, 2012
About Knowledge Bombs
Random bits of knowledge that we don't want to forget and that might help you!
Cody Lerum
Ken Mix