URL Filtering USG

URL Filtering

Uniform Resource Locator (URL) filtering regulates online behavior by controlling which URLs users can access.

URL filtering regulates online behavior by controlling which URLs users can access
- Example: For certain category of URL packets, the FW can change the DSCP value (DSCP priority) in the packets, so that other network devices can distinguish between packets based on the changed DSCP priorities and take a specified action on each type.

URL filtering allows you to manage users' online behavior on an individual basis, by user or user group, by schedule, and through the security zone
- Example: When enterprise users implement HTTP or HTTPS requests, the FW can allow, alert, or block users' requests by the URL filtering function.

URL Format

The standard format of a URL is protocol://hostname[:port]/path[?query].
URL matching modes are prefix matching, suffix matching, keyword matching, and exact matching.

URL Filtering USG-2767799-1

The priorities of URL matching modes are as follows:

Exact matching > suffix matching > prefix matching > keyword matching

URL Filtering USG-2767799-2

You can configure URL and domain name rules in whitelist, blacklist, user-defined categories, and predefined categories.

URL Filtering Mode

After an access request matches a URL or domain name rule a FW processes based on the specified URL filtering mode

The FW provides URL filtering based on the blacklist, whitelist, URL categories, URL reputation, and malicious URL to manage users' online behavior.

URL category query

URL categories are classified into user-defined URL categories and predefined URL categories. A URL category can contain several URLs, and a URL can belong to multiple categories.

After obtaining a URL from a received URL request, the FW matches the URL preferentially with user-defined URL categories.

Predefined URL category query modes are as follows:
- Local query: The predefined URL category database is loaded to the cache after the first device startup. If no matching URL category exists the FW attempts to searchfrom the remote query server.
- Remote query: The FW sends the URL to the remote query server for further search. If a matching URL category exists the FW processes the URL request based on the response and its category to the local cache for subsequent URL query.

The control actions defined for URL filtering include allow, alert, and block in ascending order of their severity.

Allow: allows users to access the requested URL.
Alert: allows users to access the requested URL and generates a log.
Block: blocks users to access the requested URL and generates a log.

If a URL belongs to multiple categories, the FW takes an action based on the action mode

Strict: The FW takes the strictest action among all matched categories.
Lenient: The FW takes the loosest action among all matched categories.

URL Filtering USG-2767799-3

URL Remote Query Process

How the FW implements URL remote query.

Generally, the security center, dispatch server, and query server work together to implement URL remote query. Their functions are as follows:

Security center: The domain name of the security center is sec.huawei.com. The security center authenticates the FW. If the authentication succeeds, the security center provides the FW with the address and port of the dispatch server address in the country or region where the FW resides
Dispatch server: provides the FW with the addresses and ports of query servers in the region where the FW resides. Dispatch servers are deployed by region. Therefore, you need to correctly configure country/region information on the FW. Otherwise, the addresses and port numbers of dispatch servers cannot be obtained.
Query server: processes query requests and returns query results to the FW.

For Security Center it is needed internet to communicated to SEC.HUAWEI.COM but if these FW cannot connect to internet and need a Remote Query can purchase Huawei SecoCenter and deploy it on the local network. The SecoCenter has the dispatch and query servers integrated.

The FW supports two remote query modes, namely, the remote and local modes.

In remote mode, the FW communicates with the security center. The dispatch server forwards query requests to the query server in the country/area configured on the FW.
In local mode, the FW communicates with the SecoCenter but not the security center.

URL Filtering USG-2767799-4

The FW sends an authentication request to Huawei security center and requests the address of the dispatch server.
If the authentication succeeds, the security center provides the FW with the address and port of the dispatch server address in the country or region where the FW resides.
The FW sends a request for the address and port of the query server to the dispatch server.
After confirming the device information of the FW, the dispatch server provides the FW with the address and port of the query server. Generally, the FW receives the addresses and port numbers of multiple query servers.
The FW sends a speed test message to all query servers, selects the query server that responds most quickly, and requests URL category information from the query server.
The query server sends the desired URL category information to the FW, and the FW continues URL filtering based on the category information.

Notes

Restrictions and Precautions

URL filtering applies only to HTTP or HTTPS URL requests. To filter HTTPS URL requests, you also need to configure SSL-encrypted traffic detection or encrypted traffic filtering of URL filtering.

SSL-encrypted traffic detection decrypt HTTPS traffic. The FW implements filtering for decrypted traffic. The function needs to encrypt a large volume of traffic, which compromises the forwarding performance of the device.

Encrypted traffic filtering of URL filtering does not decrypt HTTPS. Instead, it obtains the domain name (HOST) of the website by parsing packets. During the TLS negotiation, the FW obtains the domain name (HOST) of the website that a user wants to access based on the Server Name field in the ClientHello packet from a client and the Common Name and Subject Alternative Name fields in the Certificate packet from the server. The FW verifies the values of the three fields. The information contained in the three fields may be tampered with by malicious users. Therefore, some traffic evades URL filtering due to a field verification failure.