Site Tools


advanced-adblock

This is an old revision of the document!


Adblock v2

As an evolution of the original adblock script, v2 comes with a rich feature-set and improved interaction capabilities.

Functionality

Adblock can accept different format of lists (always plain text) extracting the domains from files in different layout (v1 required the “0.0.0.0 domain.com” format). checkList(), checkRam(), safeDnsmasqRestart() are routines to assure lists can be handled by the router (in terms of capacity) and other secondary issues are spotted like syntax errors, etc. Adblock is essentially self-diagnosing self-caring the most common issues. The script can be operated via command line and the best startin point is adblock help which produces the following output. adblock status is another way to monitor the script behaviour without using the GUI.

Logging

The new GUI allows you to set up the maximum loglevel to be generated by the script. In an attempt to make the log messages more intuitive/meaningful emoji are occasionally used. The supported error levels are 0,3,4,5,6,7. While on 7 additional debugging files are produced by the script and saved in /tmp

Filter-file size limit

It's calculated automatically by the router and given a best practice acceptable limit to enable/download lists keeping an eye on the amount of data we're asking the router to handle. The limit can be adjusted manually if needed but it *might* become a source of instability if resources are tight. There are too many models or router to test this individually so a common sense 10% of Installed RAM has been picked up as the default value.

Custom path - It is now possible to specify an option filesystem path where the script can store the files. Defining the path changes the way the script works and enables some additional functionality. This essentially allows to store lists, their http headers and the actual compiled blockfile on external storage (e.g. USB) offloading demand on the RAM and allowing for information to survive a reboot. This is particularly functional when the script is re-run (manually or automatically) and there are no updated lists, in this case the execution is cut short to save time/resources. When custom path is defined adblock will first download all the headers of the enabled lists only, then it'll compare them with the stored one locally and a decision if a full run is needed or not. For completeness a second check is also performed on the content of the nvram variables so this should read, if the config wasn't changed and no updated lists are available, then don't update and carry on running.

Black lists

The well know table where lists are defined now also accepts a new format called list-of-lists, this essentially consists in feeding the script with a URL where a list of additional URL can be found instead of the traditional blocklist we already know. To be mentioned list-of-lists is an excellent way to save NVRAM space however is comes with some minor limitation e.g. the verification of the lists to decide if they need to be downloaded or less is not performed. So essentially this is either:

URL |–> list or URL |–> list-of-lists-file |–> URL |–> list |–> URL |–> List |–> URL |–> List |–> URL |–> List The script handles the EOL of files in an automatic way and converts it internally if needed.

Custom blacklisting

Here a range of syntax are accepted - standard domain (one per line) - a path to a file where the domains are defined (e.g. /mnt/sda1/blacklist.txt) - a domain prepended with a + e.g. +example.com will blacklist the domain but also prune any subdomain found in the configuration/blockfile - list-of-lists too ???

Whitelisting

- standard domain (one per line) - a path to a file where the domains are defined (e.g. /mnt/sda1/whitelist.txt) - a domain prepended with a % will make it “strict” whitelisting. This means that the domain itself is to be resolved normally where any defined subdomain is to be blocked instead. - list-of-lists too ???

Troubleshooting

The GUI integration with adblock provides already some crucial details to understand if the script is operating correctly or not.

Adblock v1

The Adblock menu contains settings to configure FreshTomato's built-in ad blocker. The page is divided into sections. These sections include: Adblock Settings, Blacklist URL, Blacklist Custom and Whitelist.

FreshTomato's ad blocker works through DNS cache poisoning. It downloads lists of URLs/domains to block. It then replaces whatever dnsmasq resolves as the advertiser's correct IP address with an address of 0.0.0.0. Since 0.0.0.0. is an invalid/unknown IP address, no connection is made to that URL and no page is drawn, if pixlserv is installed and integrated with Adblock. If pixlserv is not integrated, some pages/frames will draw with an error from the web browser, stating the page could not be found.1)

For Adblock to work properly, client devices must be configured to use FreshTomato's IP address (dnsmasq) as their DNS server. This can be done either by static IP configuration (on the device itself) or through FreshTomato's DHCP Service. For the latter method you would first enable DHCP under Basic/Network. To enable the actual DNS server, select “Use internal DNS” under Advanced/DHCP/DNS. These two steps are mandatory, as clients bypassing FreshTomato's DNS server will not have their ads blocked.

There are three optional settings that help ensure smooth Adblock operations:

  1. Enabling Intercept DNS port in the DHCP/DNS menu.
  2. Enabling Prevent Client auto DoH in the DHCP/DNS menu.
  3. Enabling the “DoH Server” list in Adblock itself. This was added to the defaults since release 2021.6 . If this entry is missing, it can be added manually:
    https://raw.githubusercontent.com/oneoffdallas/dohservers/master/list.txt
    This prevents DoH requests from being resolved.

Warning: FreshTomato's built-in Adblock function is separate from the ad blocking scripts seen on the Web pasted into FreshTomato's custom script window. If you also use one of those, and enable FreshTomato's built-in ad blocker, conflicts and problems may happen. Choose and enable one but not both at the same time.

Adblock Settings

Enable: Checking this box enables FreshTomato's built-in ad blocker.

Debug Mode: Checking this box enables debug mode in the log. This tells FreshTomato that you want all DNS queries that are routed to dnsmasq to be logged to the system log (syslog). This is useful in testing/troubleshooting Adblock. For more on testing, see the Testing/Troubleshooting Adblock section later on this page.

Blacklist URL

This section contains a table containing a list of the various blacklists FreshTomato can download and use for ad blocking.

On: Clicking on one of the blacklist rows will make a checkbox appear at the far left of the row. Checking that box will enable the download (and update) and usage of that particular DNS blacklist. When you are finished selecting which blacklists you wish to use, click Save for the changes to take effect.

Blacklist URL: Shows the location on the Internet where that particular blacklist can be found.

Description: Display a name (if the creator used one) for the particular blacklist.



Delete: Clicking this button on a checked Blacklist URL will permanently delete that Blacklist. Note that there is no option to reset these to the original Blacklist URL entries. If you delete a Blacklist URL that is important to you, you will need to find it, and enter it back into the Blacklist URL table.

Add: Clicking Add inserts a blank row in which you can type a new URL from which to download and use a new Blacklist.

An Autoupdate function will launch randomly every day between 2:00AM and 2:59 to download the most up-to-date Blacklists from the URLs in the list.

Custom Blacklist

The Custom Blacklist section contains a field into which you enter custom blocking entries. All entries must be separated by spaces for the function to work properly for each entry.

Custom Whitelist

Here you enter custom URLs that you would like to allow, by default. Entry rule are similar to the Custom Blacklist field. You must separate all entries with spaces.

Adblock Notes

Testing/Troubleshooting Adblock

Blacklisted or Custom entries can be tested to see if Adblocker is properly redirecting them by using the nslookup tool on Windows, Linux or OS X. Simply run nslookup and enter the domain/URL in question. If it resolves to 0.0.0.0, then Adblock is working properly, and the domain will be blocked. You can also check FreshTomato's Syslog (System log) to see whether Adblock activity is reflected there. For example, this entry:

Jan 9 19:57:29 rt-n66 daemon.info dnsmasq[4872]: config hbx.media.net is 0.0.0.0

shows that the dnsmasq daemon replaced the true IP address of hbx.media.net with 0.0.0.0. Adblocker worked in this case.

If the router crashes, you may have used too many large Blacklists, and the router exhausted available RAM. Try using smaller blacklists, or fewer of the large ones.

There are a few sites available out there that allow you to test the effectiveness of your adblock configuration, when doing so please make sure you don't have any adblock/script-block plugin enabled on your browser. Some examples of adblock testing sites:

Adblock doesn't work with DoH and other Encrypted DNS lookups

Increasingly, more and more devices are using DNSSEC, and encrypted DNS protocols, such as DoH [DNS over HTTP(S)]. Adblock will not work with these DNS lookup methods. This is both because they are encrypted, and because in many cases, they are directed to a third-party server, not to the FreshTomato router/device.

As mentioned earlier, enabling the “DoH Server” List may remedy this problem.

IOS for paid iCloud service only

On the “client autonomy” topic it is to be noted that IOS devices have some settings that can potentially interfere with the adblock operations. The option “Limit Address Tracking” can be disabled on a WLAN basis as per image here below:

or to have this disabled globally look for “Private Relay”

Considering you might connect to different WLANs with your mobile/handset/laptop at different sites/locations it does somehow make more sense to disable it on a WLAN basis regardless, leaving it off specifically where your connection is provided by FreshTomato with adblock running. Once again, these two options above are only available if you are a paid iCloud customer.

1)
If only elements are blocked, it's likely the page will load without error – uncluttered.
advanced-adblock.1673025491.txt.gz · Last modified: 2023/01/06 17:18 by rs232