I have used pfSense on an Alix 2d13 board for about two years now.  It works well with pretty much the default setup but the slow down of video content on YouTube due to CDN servers being slow prompted me to implement dnsmasq on pfSense to eliminate the lag, see if I could speed up some of my dns queries by caching dns results, and also block some unfriendly domains all at the same time.

dnsmasq service showing on pfsense routerDNS Caching Using dnsmasq

Pfsense allows you to add dnsmasq with a simple click under ‘Services’ -> ‘DNS forwarder’.  The key settings are “enable dns forwarder” and “query servers sequentially”.  This set pfSense to resolve DNS queries locally first and if the answer is not cached it should go to the next DNS server in the list and then cache that locally.  You can use your ISP’s DNS server as the secondary DNS server or use a public one like Google – 8.8.8.8 or 8.8.4.4.

You will see on this same page ‘Host Overrides’.  This feature allows you to insert mappings into you host file.  You can use this for local names on your network or external domains you want to block.  In the case that you have a name you would like to block, you can set the ip to 127.0.0.1 and whenever a resource from the domain is called it will fail (and you will not get the advertisement or other object you are trying to block).  Normally you would not want to hard code IP addresses to Domain names for purposes other than blocking or testing – however, on my network, I was bad and hardcoded our mail server domain to ip because I know pretty much every device we have will query this address.  If my server ip address were to ever change, this would break mail for everyone on the network.

Enabling DNS ForwarderSettings for dnsmasq in pfSense Web GUIOne other exception to the domain/ip hardcoding rule is to get around caching servers on your ISP.  I will talk about this more further down.

The ‘Host Overrides’ feature within the GUI of pfSense has a few downsides.  It does not like when you enter multi domains with different IPs and it doesn’t allow for wildcard domains.  An example of the wild card problem is something like doubleclick.net.  They have several variations like ads.doubleclick.net and ads1.doublick.net and can add many new subdomains tomorrow.  I want to be able to block them all.

The first thing I did was cruise to ‘Diagnositics’ – > ‘edit file’ and put in the filename of ‘/usr/local/etc/dnsmasq.conf’.  I copied most of my setting from another server only changing the local domain which is halo.  Here is my copy:

# Tells dnsmasq to never forward queries for plain names, without dots or domain parts, to upstream nameservers.
# If the name is not known from /etc/hosts or DHCP then a "not found" answer is returned.
domain-needed
# Bogus private reverse lookups.
# All reverse lookups for private IP ranges (ie 192.168.x.x, etc) which are not found in /etc/hosts or the DHCP leases file are answered
# with "no such domain" rather than being forwarded upstream.
bogus-priv
#
# LAN domain lookups
#
# Add local-only domains here, queries in these domains are answered
# from /etc/hosts or DHCP only.
local=/halo/
domain=halo
#
# Add the domain to simple names (without a period) in /etc/hosts in the same way as for DHCP-derived names.
# Note that this does not apply to domain names in cnames, PTR records, TXT records etc.
expand-hosts
#
# increase DNS cache size
cache-size=10000
# Set the maximum number of concurrent DNS queries. The default value is 150
dns-forward-max=300
resolv-file=/var/etc/resolv.conf
conf-dir=/usr/local/etc/dnsmasq.d

The last line is the directory where I am going to put all my hostnames and ip addresses.  You could include them all in this config file but it will be easier to update later if you have the configuration separate.

To make the directory, I need to ssh into the pfSense router.  By default ssh is turned off and should probably stay that way when you are not making edits.  To open ssh, go to ‘System’ -> ‘Advanced’ and look for the checkbox to enable secure shell.  Once that is done, you can fire up your favorite secure shell client and connect to the router with the same credentials you use for the web GUI.  You will be hit with a menu of options, press 8 to drop to shell.

If you are on an Alix the drive is set to read-only to preserve the compact flash card.  To make the needed changes (or if you want to install packages that will aid in scripting) you will need to type: “/etc/rc.conf_mount_rw” to remount the drive with read/write permissions.  When you are done, you run: “/etc/rc.conf_mount_ro” to set it back to read-only.

Once you can write to the disk, type “mkdir /usr/local/etc/dnsmasq.d” and you will have the directory you need to add the extra configuration files for dnsmasq.  At this point, you can either set the drive back to read-only and use the web GUI or you can use ‘vi’ to make your additional config files.  When you are done with shell, type exit.  One note – switching to read-write angered my web GUI and I had to restart it from the secure shell menu.

Now you are ready to start defining your addresses.  Open up the file editor again.  To get me started, I used an address list from yoyo.org.  It is already in the correct format for a dnsmasq configuration file.  There are plenty of these lists out there – you just have to do a google search and possibly change the format.  The format should always look like this:

address=/101com.com/127.0.0.1
 address=/101order.com/127.0.0.1
 address=/103bees.com/127.0.0.1
 address=/123found.com/127.0.0.1
 address=/123pagerank.com/127.0.0.1
 address=/180hits.de/127.0.0.1
 address=/180searchassistant.com/127.0.0.1
 address=/180solutions.com/127.0.0.1
 address=/1x1rank.com/127.0.0.1

Wildcards will look like:
address=/.somedomain.com/127.0.0.1

Of course in this same file you can also define good domains or local domains.  When you are done, make sure you save the file in the “/usr/local/etc/dnsmasq.d” directory.  You can name it anything – I named mine ad-domains.  Now someone super serious about this would automate the process with a shell script that updates the file automatically.  At this point I am too lazy and I want to see the overall impact first.

For the Comcast / Slow YouTube Issue – Here is what I did and it seems to be working well so far.  I took a look at http://redirector.c.youtube.com/report_mapping to see where I was getting content from.  Then I added the mappings for the appropriate hosts from the list on this forum.

After you make all of these changes, you should restart the dnsmasq service on the router to make sure the new configuration is happy and the hostnames are updated.  If you have any issue with it restarting from the web GUI, you can always drop back into shell and type “dnsmasq” on the command line.  You should get a message about what line in the config file is incorrect.  If you see nothing but the shell prompt, all is well and your dnsmasq is running on your pfSense router.

Final tests:

From shell, you can type ‘dig google.com’ and you should see the response time and the ip address.  The second time you type it, it should be cached and the result should be faster.  If DHCP is set up correctly with DNS forwarding, you can try the same test from another computer on your network.


Comments

  1. Max -

    Nice post, Thomas, thank you very much 🙂 There was one thing I didn’t understand. What did you do with the Ip-adresses you got from the redirector.youtube.com? I get a complete subnet there, but what do I do with that?

    Reply
    • Thomas -

      It has been a while, but you would take that hostname and figure out all the ips to block (assuming that the mapping doesn’t display your ip).

      So for x<=20, ping rX.HOSTNAME.c.youtube.com (ping r1.suddenlink-cll234.c.youtube.com, ping r2.suddenlink-cll234.c.youtube.com, etc, etc). Not all ISPs have the same number of servers. It seems like most people are successful rejecting all traffic from: 173.194.55.0/24, 206.111.0.0/16.

      Reply
  2. Max -

    Thank you again, Thomas (very much) 🙂 I must be too much of a noob (even ‘though I have pfSense full blown running, with Dual WAN, Load balancing, VPN, VLAN’s, Snort, Squid, etc, etc). I get this result: 81.244.xxx.xxx => belgacom-bru4 (81.244.128.0/17). The first one (81.244.xxx.xxx) is my own public IP. This is probably what you meant with ‘assuming that … your ip’. So this would mean I am not youtube-throttled by my ISP?

    Thank you again 😀

    Reply
  3. Basharat Ahmad -

    Great articles, but have some issue with pfsense. I setup everything as mentioned above but my pfsense have setup opt1 and I would like to use the above setting just for my lan network and I don’t want opt1 to use these settings because I put iptv on that opt1 port, any help will be appreciated.
    Thanks in advance.

    Reply
  4. Stanthewizzard -

    Hello

    With 2.2 … there is no dnsmasq conf file.

    Do you have any idea how to find it ?

    Thansk for help

    Reply
  5. Alex -

    Thanks Thomas!

    Still learning my way around pfSense, but like what I see so far.

    Regarding automated script, found nice script called PiHole
    http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0/#manualsetup

    trying to integrate into pfSense, and found a couple of edits that make it pull the list into pfSense … specifically in gravity.sh:

    replaced (Raspberry Pi Linux):
    IPv4dev=$(ip route get 8.8.8.8 | awk ‘{print $5}’)
    piholeIPCIDR=$(ip -o -f inet addr show dev $IPv4dev | awk ‘{print $4}’)

    with (for FreeBSD compatibility):
    IPv4dev=$(ifconfig re1 | fgrep ‘inet ‘ | awk ‘{print $2}’)
    piholeIPCIDR=$IPv4dev

    replaced (Raspberry Pi Linux):
    truncate -s 0 $piholeDir/$matter

    with (for FreeBSD compatibility):
    cat /dev/null > $piholeDir/$matter

    Haven’t found replacement for script equivalent of … sudo kill -HUP $(pidof dnsmasq)

    So it seems to generate a nice gravity.list, just not in the pfSense format (working on that).

    Also, need to make sure that gravity.sh has correct authority (chown 775 ./gravity.sh) … and you can also install other FreeBSD packages once you SSH into pfSense, for example:
    pkg install nano
    pkg install wget

    Reply
  6. Alex -

    New script to automatically create adblocking list, adapted from TOMATO WANUP script by haarp

    ###############################################

    #!/bin/sh
    ## name = blackhole.sh
    ## Adapted for pfSense from Tomato WAN Up script v3.3 by haarp

    TMPFILE=”/tmp/dnsmasq.work” ## dnsmasq temporary file
    GENFILE=”/usr/local/etc/dnsmasq.d/dnsmasq.custom” ## dnsmasq custom config

    SOURCES=””
    SOURCES=”$SOURCES http://winhelp2002.mvps.org/hosts.txt
    SOURCES=”$SOURCES http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
    SOURCES=”$SOURCES http://hosts-file.net/ad_servers.asp
    ##SOURCES=”$SOURCES http://hosts-file.net/.%5Cad_servers.txt
    ##SOURCES=”$SOURCES http://hostsfile.mine.nu/Hosts
    ##SOURCES=”$SOURCES http://sysctl.org/cameleon/hosts
    ##SOURCES=”$SOURCES http://adaway.org/hosts.txt
    ##SOURCES=”$SOURCES http://hosts-file.net/download/hosts.txt
    #SOURCES=”$SOURCES http://hosts-file.net/hphosts-partial.asp
    ##SOURCES=”$SOURCES http://www.malwaredomainlist.com/hostslist/hosts.txt

    ## Blacklist additional sites (add inside quotes, space-separated)
    BLACKLIST=”google-analytics.com”

    ## Whitelist sites from blocking (add inside quotes, space-separated)
    WHITELIST=””

    echo “Download starting”
    until ping -q -c1 google.com >/dev/null; do
    echo “Waiting for internet”
    sleep 5
    done

    echo -n “” > $TMPFILE
    for s in $SOURCES; do
    { (wget $s -O – || elog “Failed: $s”) | \
    tr -d “\r” | \
    sed -e ‘/^[[:alnum:]:]/!d’ | \
    awk ‘{print $2}’ | \
    sed -e ‘/^localhost$/d’ >> $TMPFILE
    } &
    done

    wait

    if [ -s $TMPFILE ]; then
    echo “Download finished”
    else
    echo “Failed: Download unsuccessful, aborting”
    rm $TMPFILE
    exit 1
    fi

    echo “Generating $TMPFILE”
    for b in $BLACKLIST; do
    echo “$b” >> $TMPFILE
    done

    for w in $WHITELIST; do
    sed -i -e “/$w/d” $TMPFILE
    done

    sort -u $TMPFILE -o $TMPFILE
    awk ‘{print “address=/”$0″/127.0.0.1/”}’ $TMPFILE > $GENFILE

    echo “Config generated, $(wc -l < $GENFILE) unique hosts to block"
    echo "Restarting dnsmasq"
    service dnsmasq restart

    echo "Deleting $TMPFILE to free memory"
    rm $TMPFILE

    ###############################################

    Reply
  7. Hunter -

    Hi Thomas,

    thanks for your sharing, could you tell me Which version was used in the article.

    thanks.

    Reply

Leave a Reply