Squid Proxy is Slow
Sometimes browsing through Squid proxy integrated with Web Safety may seem to be slow. This article will try to show some possible steps to remedy this situation.
Check Basic Network Connectivity
To ensure the problem of slowliness of proxy is not in your network being slow by itself, check the basic downloading speed using
curl programs running on the proxy box. The following article may also be helpful https://serverfault.com/questions/318909/how-passively-monitor-for-tcp-packet-loss-linux/969781.
Usually the ping/tracert/wget tests might reveal possible network problems. The following shows typical outpot of the fast proxy box.
root@node12:~# ping fast.com PING fast.com (184.108.40.206) 56(84) bytes of data. 64 bytes from a23-216-240-121.deploy.static.akamaitechnologies.com (220.127.116.11): icmp_seq=1 ttl=58 time=12.2 ms 64 bytes from a23-216-240-121.deploy.static.akamaitechnologies.com (18.104.22.168): icmp_seq=2 ttl=58 time=11.8 ms 64 bytes from a23-216-240-121.deploy.static.akamaitechnologies.com (22.214.171.124): icmp_seq=3 ttl=58 time=9.77 ms --- fast.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 9.771/11.280/12.249/1.084 ms
root@node12:~# wget -d fast.com URL transformed to HTTPS due to an HSTS policy --2020-08-31 15:42:40-- https://fast.com/ Resolving fast.com (fast.com)... 126.96.36.199, 2a02:26f0:d7:3b1::24fe, 2a02:26f0:d7:38e::24fe Connecting to fast.com (fast.com)|188.8.131.52|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 25650 (25K) [text/html] Saving to: ‘index.html.3’ index.html.3 100%[=====================================================================================================================>] 25.05K --.-KB/s in 0.007s 2020-08-31 15:42:40 (3.46 MB/s) - ‘index.html.3’ saved [25650/25650]
DNS Lookup Speed
Check the speed of DNS server that Squid uses. There are two places in Web Safety UI that can help with this. First the DNS lookup speed can be seen in the UI / Dashboard / Squid tab as indicated on the following screenshot. The values shall be close to zero.
Then check the internal DNS statistics of Squid, using UI / Squid / General / Runtime Info / DNS (or manually using
squidclient mgr:idns command). The DNS queue shall be empty proving DNS responses are quick. Also check the number of DNS errors are minimal as indicated on the following screenshot.
If DNS servers indeed are slow, try to deploy DNS caching server, as described in the article Browsing slow? Deploy DNS caching server.
Check CPU and RAM Usage
Web filtering is quite CPU intensive process so it requires powerful CPUs. It might be you are hitting the limits of the available hardware. This can be checked by running
htop command on the proxy box and analysing the CPU usage of
wsicapd processes. The more users are using the proxy, the more CPU resources will be consumed. If CPU is close to 100% you might need to consider upgrading the hardware or adding more virtual CPUs to the virtual appliance. Typical output is shown at the following screenshot.
If you have enabled Squid’s cache, check that proxy box has enough free RAM available. If system goes into swap generally it means serious freezing of all activities, including web filtering.
Deploy Cluster of Web Safety
Another possible way to distribute the load is to deploy a cluster of Web Safety appliances as explained in the article Adding redundancy to Active Directory integrated Squid.
Check Active Directory Integration
If proxy is integrated with Microsoft Active Directory it means Web Safety needs to perform user/group matching for every request that comes from the browser. The domain controllers configured on UI / Squid / Auth / Active Directory Integration need to be correct, click the Test Connection button to see that everything runs quickly. Generally the response for the connection test shall be instant.
Check Squid Logs
Be sure to also check the contents of the Squid’s cache log in UI / Squid / Logs / Cache Log, it might show some problems with configuration (routing, traffic redirection, incorrect Safe Browsing API keys, eCAP ClamAV antivirus, etc).
Check Safe Browsing and Antivirus Logs
If you have enabled Anti-Virus or Safe Browsing scanning on the proxy box, please see is there anything in the UI / Antivirus / Safe Browser / Logs.