FreshTomato Wiki

User Tools

Site Tools

Trace:
advanced-adblock

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
advanced-adblock [2023/08/05 17:20] – [Domain Blacklist URLs & Group-of-Lists] hogwildadvanced-adblock [2024/09/15 01:40] (current) – [Troubleshooting v2] -Condense hogwild
Line 1: Line 1:
-====== Adblock ======+====== Adblock (DNS filtering) ======
  
-This menu contains settings to configure FreshTomato's ad blocker. It is strongly advised that you read the **How Adblock Works** section to understand its functioning, even if you use v2.+This menu contains settings to configure FreshTomato's ad blocker-DNS filtering. This feature can block ads, some malware, crypto miners, privacy-invading scripts and more, depending on the lists you use. 
 + 
 +You are strongly advised to read the How Adblock Works section to understand its functioning, even if you use v2. 
 + 
 +In documentation, the function/menu will be spelled capitalized ("Adblock"). The actual script will be spelled "adblock", in lower case. \\
  
-In this documentation, the function/menu will be spelled capitalized ("Adblock"). The script that executes will be spelled in lower case ("adblock"). \\ 
  
 ===== v1 and v2 ===== ===== v1 and v2 =====
Line 16: Line 19:
 | ARM | [[advanced-adblock#adblock_v1|v1]] | [[advanced-adblock#adblock_v2|v2]] | | ARM | [[advanced-adblock#adblock_v1|v1]] | [[advanced-adblock#adblock_v2|v2]] |
  
- \\ Adblock v1 functionality is a reduced version of ad-blocking scripts taken from code outside FreshTomato. Enabling one of those scripts and FreshTomato Adblock at the same time may cause conflicts. Do not enable both.+ \\ Adblock v1 functionality is a reduced version of ad-blocking scripts taken from code outside FreshTomato. Do not enable one of those scripts and FreshTomato Adblock at the same time. Doing so may cause conflicts.
  
 Adblock v2 uses advanced methods to block ads. It should be the preferred choice whenever possible. Adblock v2 uses advanced methods to block ads. It should be the preferred choice whenever possible.
Line 23: Line 26:
 ===== How Adblock Works ===== ===== How Adblock Works =====
  
-FreshTomato's Adblock works through DNS cache poisoning, as follows:+Adblock works through DNS cache poisoning, as follows: 
 + 
 + \\ 
 + 
 +  - A client makes a DNS request to resolve a (sub)domain 
 +  - Dnsmasq searches both its caches for a match of the requested (sub)domain. \\ One cache is for positive lookups, and one cache is for failed lookups. (( 
 +You can adjust settings for both these caches in the //dnsmasq custom configuration// field in the\\ [[advanced-dhcpdns|DHCP/DNS/TFTP]] menu. 
 +)) \\ If an entry in either cache is found, the script proceeds to step 5. 
 +  - Dnsmasq searches its configuration files for a corresponding entry. \\ For adblock, that's the dnsmasq.adblock file that holds the (sub)domains to block. \\ If found, the entry is processed. In the case of adblock, the action is: \\ resolve the domain to "0.0.0.0" or "NXDOMAIN" (a non-existent domain). \\ Dnsmasq then proceeds to step 5. 
 +  - Dnsmasq queries an upstream DNS server for the (sub)domain, directly \\ or via stubby/dnscrypt. \\ The result is either a positive lookup or an "NXDOMAIN". 
 +  - The end result of the above steps is sent to the cache and returned \\ to the requesting client. 
 + 
 + \\ 
 + 
 +Given a list of sources, the original script simply blocks ads. However, there are other reasons for your network to avoid communicating with certain servers. This function was therefore appended with "DNS Filtering" to reflect its increased abilities.
  
-  * dnsmasq resolves the domain's correct IP address. 
-  * The adblock script then replaces that address with an address of 0.0.0.0 . 
-  * The 0.0.0.0 address is sent to the client requesting DNS resolution. 
-  * Since 0.0.0.0. is an NXDOMAIN (invalid/unknown address), no connection is made to that URL. 
  
-Given a list of sources, the original script simply blocks ads. However, there are other reasons for your network to avoid communicating with certain servers. This function was therefore renamed "DNS Filtering" to reflect its progression towards a wider function. 
 ===== Adblock Settings ===== ===== Adblock Settings =====
  
Line 41: Line 53:
 **Debug mode (v1)** **Debug mode (v1)**
  
-Checking this box enables debug mode for dnsmasq in the log. This tells FreshTomato that you want all DNS queries routed to dnsmasq to be logged to the system log (syslog). This is useful when testing/troubleshooting Adblock. For more on testing, see the //Notes and Troubleshooting// section of this page. This option is identical to "Debug Mode" in the [[advanced-dhcpdns|DHCP/DNS/TFTP]] menu. Thus, in v2, it was removed from the Adblock menu.+Checking this enables debug mode for dnsmasq in the log. This tells FreshTomato to log all DNS queries routed to dnsmasq to the system log. This is useful when testing/troubleshooting Adblock. Because this option is identical to "Debug Mode" in the [[advanced-dhcpdns|DHCP/DNS/TFTP]] menu, it was removed from the Adblock menu in v2. 
 + 
 +For more on testing, see the Notes and Troubleshooting section of this page.
  
  \\  \\
  
-{{::adblock-v2-settings.jpg?580x199}}+{{::advanced-adblock-dns_filtering_settings.png?464}}
  
  \\  \\
Line 51: Line 65:
 **Max Log Level (v2)**: **Max Log Level (v2)**:
  
-Thew new v2 interface lets you set the maximum log level output the script will generate.   \\  \\ +The newer v2 interface lets you set the maximum log level output the script will generate.   \\  \\ Options include:
- +
-Supported levels include:+
  
   * 0:  The script will log only basic messages like start and stop.   * 0:  The script will log only basic messages like start and stop.
-  * 3   (Error) is the default value. Once you are confident with your setup, this setting is sufficient.+  * 3   (Error). Once you're confident with your setup, this setting is sufficient. (Default).
   * 4   * 4
   * 5   * 5
   * 6   * 6
-  * 7  (Debug level) writes very detailed log information. This is helpful when troubleshooting common problems.+  * 7  (Debug level) writes very detailed log information. This is helpful for troubleshooting. 
 + 
 + \\
  
-The higher the setting, the more detailed will be the information recorded in the logs.+The higher the setting number, the more detailed the information the logs will record.
  
  \\  \\
Line 70: Line 84:
  \\  \\
  
-A good way to view the logs is to go to the [[status-log|Logs]] menu and enter 'adblock' in the "Find in syslog" field. This filters the log view to show only entries adblock entries. At level 7, adblock produces additional debugging files. These files are saved in /tmp/adblock.debug.$now, (where "$now" is the absolute number of seconds elapsed). References to adblock in the the trace file will then be displayed in the Web interface.\\  \\+ \\ 
 + 
 +A good way to view Adblock log entries is to go to the [[status-log|Logs]] menu and type "adblock" in the "Find in syslog" field. This filters the view to show only adblock entries. 
 + 
 +When set to level 7, Adblock writes additional debug files. These files are saved in /tmp/adblock.debug.$now, (where "$now" is the absolute number of seconds elapsed). References to adblock in the the trace file will then be displayed in the Web interface.\\  \\
  
  {{::20230125-163305.png?776x296|Viewing adblock log entries}}\\  \\  {{::20230125-163305.png?776x296|Viewing adblock log entries}}\\  \\
Line 76: Line 94:
 **Blockfile size limit (v2 only)**: **Blockfile size limit (v2 only)**:
  
-Adblock v1 may crash FreshTomato if it loads lists whose combined information exceeds your router's resource capacity. This can be difficult to assess and troubleshoot. Adblock v2 comes with a customizable "//limit//" field. If the limit field is left empty, FreshTomato will calculate its value automatically. It will also calculate the value automatically if the router was reset to defaults.+Adblock v1 may crash FreshTomato if it loads lists whose combined information exceeds your router's resource capacity. This can be difficult to troubleshoot. Adblock v2 comes with a customizable "//limit//" field. If this field is left empty, FreshTomato will calculate its value automatically. It will also automatically calculate the value if the router was reset to defaults.
  
  \\  \\
Line 84: Line 102:
  \\  \\
  
-This limit is calculated as 10% of physical RAM (when external storage is set). When no external storage is found, the limit is calculated as 6.5% of RAM. The limit can be also manually configured. However, if your device becomes unstable, it is advised that you revert to the automatically-calculated value.+When external storage is used, the limit is calculated as 10% of physical RAM. When no external storage is found, the limit is calculated as 6.5% of RAM. The limit can be also manually configured. However, if your device becomes unstable, please revert to the auto-calculated value.
  
-To be clear, this is a limit, not a target. There's nothing wrong with having a small blockfile compared to the limit. A larger blockfile will result in longer restart times for the dnsmasq service. This is important because there will be no DHCP/DNS/TFTP functionality until dnsmasq finishes restarting.+This is a limit, not a target. There's nothing wrong with having a smaller blockfile than the limit. A larger blockfile will result in longer restart times for the dnsmasq service. This is important because there will be no DHCP/DNS/TFTP functionality until dnsmasq finishes restarting.
  
  \\  \\
Line 92: Line 110:
 **Custom Path (v2):** **Custom Path (v2):**
  
-An important v2 feature is the ability to configure an (optional) path to permanent storage where adblock can store relevant files. You are strongly advised to specify a custom path. A custom path enables extra functionality in the script. It allows adblock to store whole lists, their http headers and the actual compiled blockfile on external storage. This offloads RAM demand (/tmp) to permanent storage, and allows for information to survive a reboot. +An important v2 feature is the option to configure a path to permanent storage where Adblock can store relevant files. Doing so is strongly advised. A custom path enables extra functionality of the script. It allows Adblock to store whole lists, their http headers, and the actual compiled blockfile on that storage. This offloads RAM demand (/tmp) to permanent storage, and allows for information to survive a reboot. This is helpful when troubleshooting.
- +
- \\ {{adblock-v2-custom_path.jpg?573x31}}+
  
  \\  \\
  
-This is very useful when the script is re-run (manually or automatically) and there are no updated lists or configuration changes. The check for configuration changes involves controlling any modification to nvram variables since the last script run.+{{::advanced-adblock-custom_path-v2.png?492}} \\  \\
  
-When a custom path is defined, adblock will:+This is helpful when the script is rerun (manually or automatically) and there are no updated lists or configuration changes. The check for configuration changes involves controlling any modification to NVRAM variables since the last script run. 
 + 
 +When a custom path is defined, Adblock will:
  
   - Download all http headers from only the enabled lists   - Download all http headers from only the enabled lists
   - Compare those headers with the ones stored locally   - Compare those headers with the ones stored locally
   - Decide whether to re-process the blocklists or not.   - Decide whether to re-process the blocklists or not.
 +
 + \\
  
 This runs as follows: This runs as follows:
  
-  * If the configuration wasn't changed and; +  * If the configuration did not change and; 
   * No updated lists are available   * No updated lists are available
   * Skip the re-processing and return to idle.   * Skip the re-processing and return to idle.
 +
 + \\
  
 This saves time and resources. This saves time and resources.
  
-Here, configuration means any externally-mapped files where //blacklist_custom// or whitelists are defined (For example: /mnt/USB/my_whitelist.txt). Using a Custom path also allows adblock to load faster on post-(re)boot.+Here, "configuration" means any externally-mapped files where //blacklist_custom// or whitelists are defined (For example: /mnt/USB/my_whitelist.txt). Using a Custom path also allows adblock to load faster on post-(re)boot.
  
  \\  \\
Line 121: Line 143:
 ==== Domain Blacklist URLs & Group-of-Lists ==== ==== Domain Blacklist URLs & Group-of-Lists ====
  
-This table contains a list of the blacklists FreshTomato can download and use to block ads.+This table contains a list of blacklists FreshTomato can download and use to block ads.
  
-**On:**  Checking this box will make a gold star appear. The star confirms Adblock will enable downloading of the blacklist in that row. After selecting which blacklists you wish to use, click **Save**.+**On:**  Checking this makes a gold star appear. The star confirms Adblock will enable downloading of the blacklist in that row. After selecting which blacklists you wish to use, click **Save**.
  
 **Blacklist URL:**  This shows the location on the Internet where that blacklist can be found. **Blacklist URL:**  This shows the location on the Internet where that blacklist can be found.
  
-**NOTE:** Since release 2023.4 by default, no blocklists were defined as part of the standard releases. This was done to to reduce NVRAM demand. However, you can add your preferred lists manually. A good summary of these lists is the official [[adblock_dns_filtering|Adblock (DNS filtering) lists]] page on this Wiki.+ \\ 
 + 
 +Since release 2023.4, default settings do not include any blocklists. This helps reduce NVRAM demand. You can add your preferred lists manually. A summary of these lists is the official [[adblock_dns_filtering|Adblock (DNS filtering) lists]] page on this Wiki.
  
  \\  \\
  
-{{::adblock-v2-domain_blocklist_urls_and_list-of-lists.jpg?802x363}}+{{::advanced-adblock-domain_blacklist_urls.png?991}}
  
  \\  \\
Line 146: Line 170:
   * Reset FreshTomato to default settings   * Reset FreshTomato to default settings
  
-**Add:**  Clicking this inserts a blank row. In this row, you can type a new URL from which to download and use an additional Blacklist URL.+ \\ 
 + 
 +**Add:**  Inserts a blank row. In this row, you can type a new URL from which to download and use an additional Blacklist URL.
  
 You can also add a comment in the Description field. You can also add a comment in the Description field.
Line 155: Line 181:
 ==== Compatible data formats ==== ==== Compatible data formats ====
  
-Target lists to be downloaded must be in plain text format. As above, v2 can extract domains from lists in a variety of formats. Essentially, adblock uses a regex against each line of the file. Lines from the source file are parsed, comments are removed and then the domains are extracted. This is true regardless of where (in which column) those domains are in the list.+Target lists to be downloaded must be in plain text. V2 can extract domains from lists in a variety of formats. Essentially, adblock uses a regex against each line in the file. 
 + 
 +  - Lines from the source file are parsed 
 +  - Comments are removed and then; 
 +  - The domains are extracted.  
 +  - This is true regardless of where (in which column) those domains are in the list. 
 + 
 + \\
  
 **Group-of-lists** **Group-of-lists**
  
-Currently, v2 also accepts a new list format called "Group-of-lists". This involves feeding the script a URL which contains links to a list of further URLs. The Group-of-lists is an excellent way to save NVRAM space.+Currently, v2 also accepts a new list format called "Group-of-lists". This involves inputting into the script a URL of links to a list of further URLs. The Group-of-lists is excellent at saving NVRAM space.
  
  \\  \\
Line 178: Line 211:
 </code> </code>
  
- \\  \\ The Group-of-lists format has limitations. One limitation is that it doesn't perform verification of the lists to assess whether they need to be downloaded or not.+ \\  \\ A limitation of the Group-of-lists format is that it doesn't verify the lists to assess whether they need to be downloaded or not.
  
-Adblock automatically handles EOL (End of Line) characters in files, when necessary. It converts them internally.+When needed, Adblock automatically handles EOL (End of Line) characters in files by converting them internally.
  
-Group-of-lists are visible in the log file. For example, they might appear as:  "[8][2]"  The text in this example would mean that adblock is processing the 8th defined list in the Domain blacklist URLs & Group-of-lists on the second line (URL).  \\   \\+Group-of-lists are visible in the log. For example, if you see: "[8][2]"  that would mean adblock is processing the 8th defined list in the Domain blacklist URLs & Group-of-lists on the second line (URL).  \\   \\
  
  
 ==== Domain blacklist custom ==== ==== Domain blacklist custom ====
  
-{{:pasted:20230425-145638.png?775x165}}+{{:pasted:20230425-145638.png?831}}
  
  \\  \\
Line 193: Line 226:
 === Domain Syntax === === Domain Syntax ===
  
-Several syntaxes are valid for this list:+The syntax for this list is as follows:
  
-  * Standard domains (one entry per line) +  - Standard domains (one entry per line). 
-  * A path to a local file where domains are defined. The file should contain one domain per line. \\ For example: "/mnt/usb/my_blacklist.txt". +  - A "#" or "!" preceding text marks the line as comment only-to be ignored by script. 
-  * A domain prefixed with a "+" sign will blacklist that domain and force a pruning of any subdomains found in the final blockfile.+  - A domain prefixed with "+" blacklists the domain and prunes subdomains found in the final blockfile.
  
-When adding custom entries:+ \\ \\ This table summarizes the Domain blacklist syntax:
  
-If you blacklist: "sub.domain.com", then+^ Adding custom entry ^ Will blacklist / Do the following ^ 
 +| "sub.domain.com" | "sub.domain.com" \\ "sub2.sub.domain.com" \\ and all subdomains of "sub.domain.com" | 
 +| "+sub.domain.com"  | "sub.domain.com" \\ \\ Will also prune blockfile of dupes/redundant entries.\\ This reduces RAM usage, CPU cycles and storage space  | 
 +| preceding "#" or "!"  | Marks the line as comment only- to be ignored by script.  | 
 +|   |   |
  
-"sub1.sub.domain.com"+ \\ You can also specify a path to a file of domains (one domain per line). For example: "/mnt/usb/blacklist.txt". Using external storage is both convenient and offers certain advantages for maintaining your blocklist. For example, moving a blacklist to another router would be as simple as removing a USB flash drive from one router and inserting it into the other (assuming both were set with the same storage location setting).
  
-"sub2.sub.domain.com"+ \\
  
-and all other subdomains will be blacklisted.+**Sort a-z ↓** **:**  Sorts the contents of this field alphabetically. This was introduced in release 2024.1.\\
  
-However, this may not be what you want.+ \\
  
-To prevent subdomains of an entry from being filtered, prepend a "%" sign to an entry. 
  
-For example, if you enter "%sub.domain.com", only "sub.domain.com" will be blacklisted. "Sub1.sub.domain.com", "sub2.sub.domain.com" and other subdomains will not be filtered.+==== Domain whitelist ====
  
-Prepending a "+" sign to a domain entry will blacklist that domain and all subdomains and force a pruning of any subdomains found in the final blockfile. Pruning removes duplicate/redundant entries from the blocklist file. Duplicate or redundant entries can increase processor cycles, and RAM usage. Thus, pruning significant amounts of redudant entries can reduce CPU, memory and storage needs. +{{:pasted:20230425-145820.png?834}}
- +
-Any line starting with a "#" or "!" character is considered a comment and will be ignored. +
- +
-**Sort a-z ↓** **:**  Clicking this button sorts the contents of this field alphabetically.  \\+
  
  \\  \\
  
 +=== Domain whitelist syntax ===
  
-==== Domain whitelist ====+The Domain whitelist can use:
  
-{{:pasted:20230425-145820.png?778x164}}+  * Standard domains (one entry per line). 
 +  * Prepending a "%" sign before an entry indicates a "strict" whitelisting. \\ Whitelisting always takes precedence over blacklisting.\\ For example, if a blacklist contained "ads.example.com", whitelisting "example.com"  \\ will undo the blacklisting of  "ads.example.com"  and all other subdomains. 
 +  * Prepending a "#" or "!" before an entry marks it as comment only-to be ignored by Adblock.
  
  \\  \\
  
-Domain whitelist can use:+This table summarizes the Domain whitelist syntax:
  
-  * Standard domains (one entry per line) +^ Adding custom entry ^ Will whitelist / Do the following ^ 
-  * A path to a file of domains. For example: "/mnt/usb/my_whitelist.txt". The file must contain one domain per line. +| "sub.domain.com" | "sub.domain.com" \\ "sub2.sub.domain.com" \\ and all other subdomains of "sub.domain.com" | 
-  * A domain prepended with a "%" sign will indicate a "strict" whitelisting. Whitelisting always takes precedence over blacklisting. For example, if a blacklist contained "ads.example.com" , then whitelisting "example.com" will undo the blacklisting of  "ads.example.com"  and all its subdomains. +| "%sub.domain.com"  | "sub.domain.com" \\ but not subdomains of "sub.domain.com".\\ \\ Will also prune blockfile of dupes/redundant entries.\\ This reduces RAM usage, CPU cycles and storage space.  | 
-  * Blacklisting all subdomains may not be your goal. Whitelisting "%example.com"  will whitelist only "example.com", and will not negate blacklisting of "ads.example.com" or any of its subdomains.+| "#" or "!"  | Marks the line as comment only-to be ignored by Adblock.  | 
 +|   |   |
  
-Any line starting with a “#” or “!” character is considered commented and ignored.+ \\
  
-**Sort a-z ↓** :  Clicking this button sorts the field content alphabetically.+You can also specify a path to a file of domains (one domain per line). For example: "/mnt/usb/whitelist.txt". Using external storage is both convenient and offers certain advantages for maintaining your blocklist. For example, moving a whitelist to another router would be as simple as removing a USB flash drive from one router and inserting it into the other (assuming both were set with the same storage location setting).
  
- \\   \\+\\ **Sort a-z ↓** :  Clicking this button sorts the field contents alphabetically. 
 + 
 + \\
  
 === Maintaining the Domain whitelist === === Maintaining the Domain whitelist ===
  
-A good way to maintain the whitelist is to share it on your LAN, and educate users about nslookup verification and whitelist additions.+One way to maintain the whitelist is to share it on your LAN, and teach users nslookup verification and whitelist additions.
  
 Such a process could be automated as follows: Such a process could be automated as follows:
Line 254: Line 292:
     - How to use nslookup on a URL to verify DNS lookups are being poisoned     - How to use nslookup on a URL to verify DNS lookups are being poisoned
     - Where to find the whitelist (via its samba share) and;      - Where to find the whitelist (via its samba share) and; 
-    - After the whitelist has been edited, to configure settings in [[admin-buttons|Buttons/LED]] as follows: \\ \\  {{:pasted:20230125-142049.png}}\\  \\  \\ When the correct router button is pressed, these settings will trigger the ''/usr/sbin/adblock update'' function . \\ The time to complete the update may vary. A quick-run might take a few seconds. A full-run might take a few minutes.+    - After the whitelist has been edited, to configure settings in [[admin-buttons|Buttons/LED]] as follows: \\ \\  {{:pasted:20230125-142049.png?418}}\\  \\  \\ With these settings, pressing the correct router button triggers the ''/usr/sbin/adblock update'' function. \\ The time to complete an update varies. A swift run may take a few seconds. A full run might take minutes.
  
  \\   \\  \\   \\
Line 261: Line 299:
 ==== Enforcing Client Compliance ==== ==== Enforcing Client Compliance ====
  
-For Adblock to work properly, client devices **must be configured** to use FreshTomato's IP address (dnsmasq) as their DNS server. This can be done by configuring a static IP address on the client device itself or through FreshTomato's DHCP Service.+For Adblock to work properly, client devices **must be configured** with FreshTomato's IP address (dnsmasq) as their DNS server. This is done by configuring a static IP address on the client device itself or through FreshTomato's DHCP Service.
  
-For the latter, first enable DHCP in the [[basic-network|Network]] menu. To enable the DNS server, select "Use internal DNS" under in the [[advanced-dhcpdns|DHCP/DNS/TFTP]] menu.+For the latter, enable DHCP. Then, enable the DNS server, and select "Use internal DNS" in the [[advanced-dhcpdns|DHCP/DNS/TFTP]] menu.
  
-These steps are mandatory. Clients that bypass FreshTomato's DNS server will not have their ads blocked.+These steps are mandatory. Clients that bypass FreshTomato's DNS server won't have their ads blocked.
  
-Three optional settings help ensure proper Adblock operation:+ \\ 
 + 
 +Three optional settings improve proper Adblock operation:
  
   - Enable //Intercept DNS port// in the [[advanced-dhcpdns|DHCP/DNS/TFTP]] menu.   - Enable //Intercept DNS port// in the [[advanced-dhcpdns|DHCP/DNS/TFTP]] menu.
-  - Enable //Prevent Client auto DoH// in the [[advanced-dhcpdns|DHCP/DNS/TFTP]] menu. +  - Enable //Prevent Client auto DoH// in [[advanced-dhcpdns|DHCP/DNS/TFTP]]. 
-  - Enable the "DoH Server" list in Adblock. This prevents DoH requests from being resolved.  \\ This is the default DoH Server list since release 2021.6. \\  It can be found at: [[https://raw.githubusercontent.com/oneoffdallas/dohservers/master/list.txt]] \\ \\+  - Enable the "DoH Server" list in Adblock to prevent DoH requests from being resolved. \\ \\ The default DoH Server list since release 2021.6 can be found at: \\ [[https://raw.githubusercontent.com/oneoffdallas/dohservers/master/list.txt]] \\ \\
  
-Adblock v1 functionality is a reduced version of ad-blocking scripts taken from code outside FreshTomato. Enabling one of those outside scripts and FreshTomato Adblock at the same time may cause conflicts. Do not enable both. Adblock v2 uses advanced methods to block ads. It should be the preferred choice whenever possible.+V1 functionality is a reduced version of scripts from code outside FreshTomato. Do not enable both an outside script and FreshTomato Adblock at the same time. Doing so may cause conflicts.
  
-Adblock/DNS Filtering affects name resolution only. If an application communicates directly via IP address, Adblock cannot prevent that.+Adblock v2 uses advanced methods to block ads. It should be the preferred choice whenever possible. 
 + 
 +Adblock/DNS Filtering **affects name resolution only**. Adblock cannot block an application that communicates directly via IP address.
  
-Thorough management of domain blocking can be tedious work. For example, with email spam, you'll probably have to deal with false positives.+Thorough management of domain blocking can be tedious. For example, with email spam, you'll probably have to deal with false positives.
  
-Regardless of version, the script is always named "adblock", and can be found at:  "/usr/sbin/adblock",+Regardless of version, the script is always named "adblock", and can be found at:  "/usr/sbin/adblock".
  
  \\  \\
Line 286: Line 328:
 ===== Adblock v2 ===== ===== Adblock v2 =====
  
-Adblock v2 improves on the original (v1) script. V2 adds a richer feature set, with increased user interaction via a toolbar and command line options.+Adblock v2 improves on the v1 script. It adds a richer feature set, increased user interaction (via toolbar) and command line options.
  
  \\ \\ \\  {{:pasted:20230210-144106.png?829}}\\  \\  \\ \\ \\  {{:pasted:20230210-144106.png?829}}\\  \\
Line 293: Line 335:
 ==== v2 Improvements ==== ==== v2 Improvements ====
  
-  * Adblock v2 performs similar basic DNS filtering functionality to v1, but in a more precise and controlled way. +  * Performs similar DNS filtering to v1, but in a more precise and controlled way. 
-  * Allows you to run certain script functions without having to run all of them, to save time. +  * Lets you to run certain script functions but not others to save time. 
-  * Can accept different list formats (in plain text), and extract domains from files with different layouts. \\  This includes Easylist format. There may be false positives using Easylist format. For details, see: [[https://easylist.to/|https://easylist.to/]] +  * Accepts different list formats (in plain text), and extract domains from files \\ with different layouts. 
-  * Prevents resource starvation. Also, before running, v2 assesses its configuration for blocklist capacity. It's basically self-diagnosing and healing.  +    * This includes Easylist format. However, this format may create false positives.\\ For details, see: [[https://easylist.to/|https://easylist.to/]] 
-  * Supports Quick-run. This mode allows you to add a domain to the lists without needing to reprocess (download) entire lists again. +  * Can prevent resource issues. Before running, it assesses its blocklist capacity. \\ It can do basic self-diagnosis and healing of some resource problems it detects. 
-  * Supports external storage. Now, lists and headers are stored in optional permanent storage to help decide which lists to download. +  * Supports Swift-run mode. You can add a domain to lists without having to\\ reprocess (download) whole lists again. 
-  * Prevents false positives via a new, hard-coded 30-minute interval between runs. This is enabled after an update is completed. +  * Supports external storage. Lists and headers can be stored in permanent storage \\ to help decide which lists to download. 
-  * Includes more troubleshooting options. Debug logging and script tracing can be enabled to look deep into the \\ lowest operational level of the script. +  * Prevents false positives via a hard-coded 30-minute interval between runs. \\ This enables after an update. 
-  * Can be operated from the web interface or at a command line.+  * Includes more troubleshooting options.  
 +    * Debug logging and script tracing can look \\ deep into the lowest operational level of the script. 
 +  * Can be operated via the web interface or via the command line.
  
  \\  \\
  
-To get help with adblock at the command line, type ''adblock help'' . This will produce the following output:\\ \\ {{:pasted:20230210-141113.png?464x467|adblock help output}}\\+ \\ 
 + 
 +To get help with adblock at a command line, type ''adblock help'' :\\ \\ {{:pasted:20230210-141113.png?464|adblock help output}}\\
  
  \\  \\
Line 313: Line 359:
  \\  \\
  
-\\ {{:pasted:20230125-163607.png}}\\   \\  \\ \\  The additional debugging files can also be accessed at the command line by typing the command:  ''adblock status'' .+\\ {{:pasted:20230125-163607.png}}\\   \\  \\ \\ Further debugging files can also be accessed at the command line by typing:  ''adblock status'' .
  
- \\  {{:pasted:20230207-193120.png}}\\  \\+ \\ {{:pasted:20230207-193120.png}}\\  \\ 
 + 
 + \\ \\  \\ 
 + 
 +To open the last trace file automatically, type ''adblock trace'' .
  
- \\ \\ To open the last trace file automatically, simply type ''adblock trace'' at a command shell. You will be prompted to select the preferred viewer in which to view the trace files.+You'll be prompted to select the viewer in which to view trace files.
  
-\\ {{:pasted:20230123-133409.png?818x56}}\\  \\  \\+\\ {{:pasted:20230123-133409.png?909}}\\  \\  \\
  
  \\  \\
Line 332: Line 382:
  \\  \\
  
-| {{ :pasted:20230123-173851.png?392 }} | **Load:**   (start) will try to run the script. If the script's last run was a full run and then it stopped, Load will exit the script and recover functionality (normal operation?) without a full processing of lists. This will save processing time. Load also occurs at boot time. | +| {{ :pasted:20230123-173851.png?392 }} | **Load:**   (start) will try to run the script. If the last run was a full one, and then it stopped, Load will exit the script and recover functionality (normal operation?) without a full processing of lists. This saves processing time. Load also occurs at boot time. | 
-| ::: | **Unload:**   (stop) stops the script, regardless of what it's doing. It will also force-kill any child process if this is to be run while the script is already loading. If a full version of the script is found and external storage enabled, Unload will move the blockfile from external storage without destroying it. This allows extremely quick load times. Unload preserves the adblock.dnsmasq file. | +| ::: | **Unload:**   (stop) stops the script, regardless of what it's doing. It will also force-kill any child process if this is to be run while the script is already loading. If a full version of the script is found and external storage enabled, Unload will move the blockfile from external storage without destroying it. This allows very quick load times. Unload preserves the adblock.dnsmasq file. | 
-| ::: | **Update:**   forces an update. There is a 30-minute interval between updates. Thus, if Update is run too often, adblock may not update, but instead, log the reason (active hold-time) it didn't update. When relevant, both GUI and command line will report the interval time. To force an update, first click Unload, and then Update. This bypasses the interval period.  |+| ::: | **Update:**   forces an update. There is a 30-minute interval between updates. Thus, if Update is run too often, adblock may not update, but instead, log why (active hold-time) it didn't update. When relevant, both GUI and command line will report the interval time. To force an update, click Unload, and then Update. This bypasses the interval period.  |
 | ::: | **Reset-limit: **  ''nvram unset adblock_limit'' variable and recalculate the default value for you.  | | ::: | **Reset-limit: **  ''nvram unset adblock_limit'' variable and recalculate the default value for you.  |
-| ::: | **Clear all files:**   This is helpful when certain configuration problems occur. For example, if files or nvram become corrupted, or if adblock_limit or dnsmasq.adblock become too large. The former can happen if adblock_limit was calculated before taking into account new or larger running processes. \\ \\ In such cases, if changing settings doesn't correct the problem, the "Clear all files" button will: \\ - Kill all adblock-related processes which are still running or stuck \\ - Delete all existing adblock-related data files from pemanent storage \\ - Delete all adblock-related data files from /etc and /tmp \\ - Restart dnsmasq without adblock, (as a last resort). \\ \\ Running Unload, Reset Limit, Clear all files is a more radical approach to resetting things.  \\ The biggest difference between clear and unload is that clear creates an environment as if adblock had never run, whereas Unload preserves adblock.dnsmasq. This makes quick-start possible when only unload has been run.  | +| ::: | **Clear all files:**   This is helpful when certain configuration problems occur, such as when files or NVRAM become corrupted, or adblock_limit or dnsmasq.adblock become too large. The former can happen if adblock_limit was calculated before considering new or larger running processes.\\ \\ In such cases, if changing settings doesn't fix the problem, the "Clear all files" button will: \\ - Kill all adblock-related processes that are still running or stuck. \\ - Delete all existing adblock-related data files from pemanent storage. \\ - Delete all adblock-related data files from /etc and /tmp. \\ - Restart dnsmasq without adblock, (a last resort). \\ \\ Running Unload, Reset Limit, Clear all files is a more radical approach to resetting things.  \\ The biggest difference between clear and unload is that clear creates an environment as if adblock had never run, whereas Unload preserves adblock.dnsmasq. This makes quick-start possible when only unload has been run.  | 
-| ::: | **Snapshot:**   is used for troubleshooting. It captures the list of relevant files and saves it to: /tmp/adblock.snapshot.${now.in.seconds}. You can take multiple screenshots before, after and even while the script is running. A popup confirms the snapshot was taken and the file location.  | +| ::: | **Snapshot:**   is for troubleshooting. It captures the list of relevant files and saves it to: /tmp/adblock.snapshot.${now.in.seconds}. You can take multiple screenshots before, after and when the script is running. A popup confirms a snapshot was taken and the file location.  | 
-| ::: | **Enable Only:**   In v1, to enable/disable adblock, you had to click Save. This isn't always convenient, especially if a large number of lists is loaded. This button lets you enable adblock (''nvram set adblock_enable=1'' + ''nvram commit'') on the stand.  | +| ::: | **Enable Only:**   In v1, to enable/disable adblock, you had to Save. This isn't always convenient, especially if a lot of lists are loaded. This lets you enable adblock (''nvram set adblock_enable=1'' + ''nvram commit'') on the stand.  | 
-| ::: | **Disable Only:** This lets you disable adblock (''nvram set adblock_enable=0'' + ''nvram commit'') without having to click Save. \\ |+| ::: | **Disable Only:** lets you disable adblock (''nvram set adblock_enable=0'' + ''nvram commit'') without clicking Save. \\ |
  
  \\  \\
Line 345: Line 395:
  \\  \\
  
-Refresh: There is now a refresh command at the bottom of the table. Refreshes are limited to a minimum frequency of 5 seconds. The more frequently you run refreshes, the higher the system load will climb.+Refresh: A new command at the bottom of the table. Refreshes are limited to a minimum frequency of 5 seconds. Refreshing more frequently will cause the system load to climb.
  
  
-===== Quick-run (v2) and Full-run Operations =====+===== Swift-run (v2) and Full-run (v1) Operations =====
  
  \\  \\
Line 356: Line 406:
  \\  \\
  
-Modifications to the adblock script configuration can be categorized as follows:+Modifications to the adblock script configuration are categorized as follows:
  
   - List (enable/disable)   - List (enable/disable)
Line 370: Line 420:
  \\  \\
  
-Only two operations from the above list can be run in Quick-run mode, as they don't require full processing:+The only two operations from the above list that can be run in Swift-run mode are:
  
-  * No. (4) Addition of simple custom blacklist domain/s +  * No. 4 - Addition of simple custom blacklist domains 
-  * No. (5) Addition of simple whitelisted domain/s+  * No. 5 - Addition of simple whitelisted domains
  
- \\+ \\ This is because they don't require full processing.
  
-Operations (4) and (5) are the most common actions performed when maintaining adblock script configuration. Quick-run operations are performed live on a running blockfile. Therefore, the adblock script must be loaded in order to run functions (4) or (5). A quick-run can be completed in literally seconds. A full-run may take minutes to complete. Any operations other than (4) or (5) above require a full-run.+These operations are the most common performed when maintaining adblock script configuration. Swift-run operations are performed live on a running blockfile. Therefore, the adblock script must be loaded in order to run functions (4) or (5). A swift run can be completed in seconds. A full run may take minutes. All operations other than (4) or (5) above require a full run.
  
-Please note that "quick" is a temporary action and will allow your change to become immediately functional. In contrast, a successive full-run (manual or scheduled) would process added domains in the standard way.+Please note that "swift" is a temporary action to allow your change to become immediately functional. In contrast, a successive full run (manual or scheduled) would process added domains in the standard way.
  
  
 ===== Command line operations (v2) ===== ===== Command line operations (v2) =====
  
- \\  Another feature of v2 is the ability to interact with adblock via command line.  \\   \\   \\  {{:pasted:20230125-184348.png}}+ \\  Another new v2 feature is the ability to interact with adblock at the command line.  \\   \\  As seen above in this "adblock help" screenshot, v2 has many more functions available: 
 + 
 +{{:pasted:20230125-184348.png?550}}
  
 \\ \\
Line 390: Line 442:
  \\  \\
  
-As seen above in the "adblock help" screenshot, v2 has many more functions available to use:+The table below summarizes all command switches (v2 only):
  
- \\ Help+^ Command  switch  ^ Type the following  ^ Performs the following task/Comments. ^ 
 +| Help  | "adblock help"  | Displays a summary guide of valid commands.  | 
 +| Status  | "adblock status"  | Gives constant updates on adblock's operational status.  | 
 +| Start  | "adblock start"  | The same as "Load" in the Web interface.  | 
 +| Stop  | "adblock stop"  | The same as "Unload" in the Web interface  | 
 +| Update  | "adblock update"  | The same as "Update" in the Web interface. \\ | 
 +| Upgrade  | "adblock upgrade"  | Checks online for new script versions. If one is available, \\ it prompts you to install the upgrade. This downloads the latest script \\ version into /tmp (RAM) then mounts it over the built-in version.Since the script is stored in RAM, it will be lost after a reboot. \\ Therefore, an extra parameter is supported: "adblock upgrade silent". \\ If an upgrade is available, this command triggers it without prompting you. \\ This switch may be used in the init script, to ensure \\ the latest version runs, even after a reboot.  | 
 +| Test  | "adblock test $domain"  | Let you do a quick test to see if a domain name is being blocked. \\ Helpful when troubleshooting.  | 
 +| Reset  | "adblock reset"  | The same as "Reset" in the web interface.  | 
 +| Clear  | "adblock clear"  | The same as "Clear all files" in the web interface. Resets the blockfile maximum limit. | 
 +| Trace/Status  | "adblock trace"  | Only available at the command line. When the script is run with Loglevel set at 7, \\ adblock runs in "trace mode". \\ Every command executed by the script is saved \\ into a trace (log/debug) file. You can find the trace file \\ using "adblock status". Running "adblock trace" will open the latest \\ appropriate trace file for you even more easily. | 
 +| Snapshot | "adblock snapshot" | The same as "Snapshot" in the web interface. | 
 +| Enable  | "adblock enable"  | The same as "Enable" in the web interface.  | 
 +| Disable  | "adblock disable"  | The same as "Disable" in the web interface.  |
  
- ''adblock help      ''   = Displays a summary guide of the valid commands. + \\   \\
- +
- \\ Status +
- +
- ''adblock status     '' = Gives constant updates on adblock's operational status. +
- +
- \\ Start +
- +
- ''adblock start      '' = This is the same as "Load" in the Web interface. +
- +
- \\ Stop +
- +
- ''adblock stop      ''   = This is the same ad "Unload" in the Web interface. +
- +
- \\ Update +
- +
- ''adblock update     '' = This is the same as "Update" in the Web interface. +
- +
- \\ Upgrade +
- +
- ''adblock upgrade    '' = This parameter checks online for new script versions. If one is available, it asks if you want to install the upgrade. This is a workaround until the next release. This will download the latest version of the script into /tmp (RAM) then mount it over the built-in version. +
- +
-Since it is stored in RAM, it will be lost after a reboot. Therefore, an extra parameter is supported:  ''adblock upgrade silent'' . If an upgrade is available, this command triggers it, without prompting you. This switch may be used in the init script, to ensure the latest version runs, even after a reboot. +
- +
- \\ Test +
- +
- ''adblock test $domain''= This switch lets you quickly troubleshoot a domain name. +
- +
- \\ Reset +
- +
- ''adblock reset      '' = This is same as "Reset" in the web interface. +
- +
- \\ Clear/Clean +
- +
- ''adblock clear      '' = This is the same as "Clear all files" in the web interface. It resets the blockfile maximum limit.''adblock clean'' +
- +
- \\ Trace/Status +
- +
- ''adblock trace      '' = This is a command line only function. When the script is run with Loglevel = 7, adblock is also running in "trace mode". Every command executed by the script is saved into a trace (log/debug) file. You can find the trace file using ''adblock status'' . Running ''adblock trace'' will open the latest relevant trace file for you with even less effort. +
- +
- \\ Snapshot +
- +
- ''adblock snapshot   '' = This is the same as "Snapshot" in the web interface. +
- +
- \\ Enable +
- +
- ''adblock enable    ''   = This is the same as "Enable" in the web interface. +
- +
- \\ Disable +
- +
- ''adblock disable   ''   = This is the same as "Disable" on the web interface. +
- +
- \\ +
- +
- \\+
  
  
 ==== Adblock test command (v2) ==== ==== Adblock test command (v2) ====
  
- \\+ \\ For quick and easy testing of broken resolution or other DNS issues, use the "test" switch.
  
-{{:pasted:20230210-135935.png}}\\  \\+Typing:  "adblock test example.com" will quickly test whether name resolution is functioning.
  
-To simplify and speed up testing of broken resolution or other DNS issues, use the "test" switch.  \\ Typing:  ''adblock test example.com'' will quickly verify whether name resolution is functioning.+{{:pasted:20230210-135935.png}}\\  \\  \\
  
 Specifically: Specifically:
  
-  * dnsmasq answer - Displays the actual name resolution as seen by the router and LAN users. \\ The word "Blocked" is displayed if access to the domain was blocked. +  * Dnsmasq answer - Displays the actual name resolution as seen \\ by the router and LAN users. \\ The word "Blocked" is displayed if access to the domain was blocked. 
-  * Cloudflare answer - This query bypasses the usual dnsmasq process and sends a query to address 1.1.1.1 . \\ This can help you to verify if a domain exists. +  * Cloudflare answer - This query bypasses the usual dnsmasq process \\ and sends a query to address 1.1.1.1 . \\ This can help verify if a domain exists. 
-  * Blockfile ref - This checks whether there are any references to the tested domain in the blockfile. +  * Blockfile ref - Checks if the blockfile contains references to the tested domain.
- +
- \\ With these three pieces of information, you can easily assess whether a domain is blocked, non-existent, or intact. +
- +
  
 + \\ With these 3 pieces of information, you can easily assess whether a domain is blocked, non-existent, or intact.
  
  
 ===== Automatic list updates ===== ===== Automatic list updates =====
  
-As part of normal adblock operation, a crontable entry is added to perform a daily list update. Update times are now randomized to prevent DDoS of the list providers' servers. A new update time is set each time you run ''adblock stop'' or reboot. The update's time window varies between 02:00 AM and 05:59 AM local time, depending on the script version you're running (original v1, modified v1, or v2).+Normally, adblock adds a crontable entry to perform a daily list update. Update times are now randomized to prevent DDoS of the list providers' servers. Each time you run ''adblock stop'' or reboot, a new update time is set. The update's time window varies between 02:00 AM and 05:59 AM local time, depending on the script version run (original v1, modified v1, or v2).
  
  \\  \\
  
-To verify the current scheduled update time:+To verify the current scheduled update time, type:
  
 ''root@router:/# cru l | grep adblockJob''\\ '' 28 4 * * * /usr/sbin/adblock update #adblockJob#'' ''root@router:/# cru l | grep adblockJob''\\ '' 28 4 * * * /usr/sbin/adblock update #adblockJob#''
Line 487: Line 495:
 This picture illustrates the syntax of crontable expressions: This picture illustrates the syntax of crontable expressions:
  
- \\ + \\ {{::crontab-syntax.gif?400x162}}
- +
-{{::crontab-syntax.gif?400x162}}+
  
  \\  \\
Line 500: Line 506:
 ===== Future development goals ===== ===== Future development goals =====
  
-  * Add an option to let FreshTomato users share their Whitelists. \\ This would allow the creation of a shared database of "good" domains, according to users. \\ This shared whitelist would be downloaded just as you would download any blacklist. +  * Add an option to let FreshTomato users share their Whitelists. \\ This would allow creation of a shared database of "good" domains, according to users. \\ This shared whitelist would be downloaded the same way you would download any blacklist. 
-  * Add a test function to the web interface (currently only command line)+  * Add a test function to the web interface (currently only command line).
  
 \\ \\
Line 560: Line 566:
  \\  {{:pasted:20230207-193120.png}}\\  \\  \\  {{:pasted:20230207-193120.png}}\\  \\
  
- \\ \\ To open the last trace file automatically, simply type ''adblock trace'' at a command shell. You will be prompted to select the preferred viewer in which to view the trace files.+ \\ \\ To open the last trace file automatically, type ''adblock trace'' at a command shell. You'll be prompted to select the viewer in which to view trace files.
  
-\\ {{:pasted:20230123-133409.png?818x56}}\\  \\  \\+\\ {{:pasted:20230123-133409.png?909}}\\  \\  \\
  
  \\  \\
Line 570: Line 576:
 **Apple iOS devices (paid iCloud service only)** **Apple iOS devices (paid iCloud service only)**
  
-Apple IOS devices have settings that may interfere with adblock operations. The "Limit Address Tracking" option can be disabled on a per WLAN basis, as seen in the image below:+Apple IOS devices have settings that may interfere with Adblock operations. The "Limit Address Tracking" option can be disabled on a per WLAN basis, as seen below: 
 + 
 + \\ {{:pasted:20230124-162651.png?576}}
  
  \\  \\
- 
-{{:pasted:20230124-162651.png?573}} 
  
 \\  {{:pasted:20230124-162536.png?558}}\\ \\  {{:pasted:20230124-162536.png?558}}\\
  
  \\  \\
- 
-Alternatively, you can disable this globally via the "Private Relay" setting, as seen here: 
  
  \\  \\
 +
 +Alternatively, you can disable it globally via the "Private Relay" setting, as seen below:
  
 {{:pasted:20220611-144641.png}}\\ \\ {{:pasted:20220611-144641.png}}\\ \\
  
-You might connect your mobile/laptop to different wireless networks at different locations. For this reason, it makes sense to disable Private Relay on a per WLAN basis. It is advised that you disable it wherever FreshTomato provides your Internet connection and Adblock is running.+You might connect your mobile/laptop to different WiFi networks at different locations. For this reason, it makes sense to disable Private Relay on a per WLAN basis. It is wise to disable it wherever FreshTomato provides your Internet connection and Adblock is enabled.
  
-Remember, these options are only available for paid iCloud customers.+Remember, these options are only available to paid iCloud customers.
  
  
advanced-adblock.1691252446.txt.gz · Last modified: 2023/08/05 17:20 by hogwild

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki