wiki.ipfire.org

The community-maintained documentation platform of IPFire

User Tools

Site Tools


configuration:services:ids

Intrusion Detection System

What is it?

An Intrusion Detection System (IDS) is a program or a framework supposed to detect, analyse and block network attacks. It does not replace a packet filter (which is enabled in IPFire by default, see Firewall Documentation) but can eliminate some limitations of it.

There are basically two types of IDSs: Host-based Intrusion Detection Systems (HIDS), which are running on a single computer, and Network-based Intrusion Detection Systems (NIDS). A NIDS is able to protect a complete network and is normally running on a firewall, gateway or dedicated server. IPFire features a NIDS.

How does it work?

A packet filter can be compared to a doorman: It has only limited information about a network packet, such as; source, destination and type. However, an Intrusion Detection System is more like a security guard, who is able to inspect the visitor for arms or spy tools. Further, the security guard has access to lists of wanted and other documents.

So an IDS is able to detect harmful network packets because of their content, because the source address appears in a blacklist or because the network traffic looks like a known attack. A packet filter (normally) is unable to operate at this level. Again, IDS and packet filter cannot replace each other, they are both important to provide a good level of network security.

The IDS used by IPFire is called Snort. Snort uses rules, which are combined in rule databases and can be downloaded from certain web sites. These rules contain patterns or blacklisted IPs.

What rules are available?

There are four rule sources available:

  1. Emergingthreats.net Community Rules – They are free and community-maintained rules (further information) and cover scanning activities, attack patterns agains various protocols, blacklists and more. No registration is required to use those rules.
  2. Snort/VRT GPLv2 Community Rules – These are free and GPL licenced snort rules. Usually, the quality is good. Accoding to the Snort blog, no registration is required.
  3. Sourcefire VRT rules for registered users – These rules are usually more than 30 days old and can be used for free. Registration is required. Usually, the quality of these rules is a bit better than these of the Emergingthreats.net Community Rules.
  4. Sourcefire VRT rules with subscription – Same as above, but they are chargeable and more current. These might be useful in productive environment, where you need reliable and up-to-date IDS rules.

Disadvantages of an IDS

There are some negative aspects when running an IDS. Most are not critical but some could prevent you from setting up the IDS on IPFire.

  1. An active IDS requires a considerable amount of system resources (CPU load and memory). In general, 2 Gigabytes of RAM and a fast CPU (few faster cores are better than many slow ones) should be there if you use an IDS.
  2. Until Core Update 100, the IDS was not running on ARM devices because of a bug. For this reason and to ensure your system does not have known security vulnerabilities, please ensure you always run the latest version of IPFire available.
  3. It is strongly recommended to install and use the Guardian Add-On if you are using an IDS. Guardian automatically blocks IP addresses which trigger IDS alerts. Without it your IDS only reports problems but does not actually protect your network in any way.
  4. Currently IPFire does not feature automatic rule updates, so you'll need to check for and install rule updates manually every week. (An advanced user may create a cron job)
  5. An IDS is not a magic bullet. Please use this guide to make your firewall system stronger against attacks: IPFire Security hardening

Choose the networks your IDS will protect

First, open the WebUI and go to “Services|Intrusion Detection”; the page is not a very big deal at this moment since the IDS is not running.

Second, choose the network interfaces you want the IDS to be active on. Usually, you might at least enable it for the RED interface since it should protect you against attacks from the internet. But also internal networks, such as GREEN and BLUE, can be monitored.

The more networks you check, the more system resources will be later needed for the IDS. Further, the rules you will activate in the next step will affect all activated networks.

Hit “Save” after you made your choice.

Download the rule database

Choose the rule database you want to use from the dropdown box and enter the registration code in the input field, if necessary. Again, hit “save” for the changes to take effect.

Then, download the rule database by clicking at “download new ruleset”. This procedure may take a while; the actual speed depends from your internet connection speed and the clock speed of your CPU.

Choose rules

This part is the most difficult: You need to choose which rules should be active.

This decision depends on the needs of your network (like operating systems in use, active services, protocols in use). Please refer to the homepage of your rule source to get further information about the purpose of some rule categories;

Some rules are based on blacklists (such as the Emerging Threats CIArmy list) and indicate that a certain IP has a bad reputation for some reason. This does not necessarily mean that it attacked your firewall, in case it appears in the log files, unless it triggered some other rules. Nevertheless, it is usually safe to use IDS rules based on blocklists since they are very conservative most of the time, making blocking a legitimate IP address very unlikely.

We cannot give you any advice here.

Select the rules you want to be active by clicking at the checkbox. After that, hit the “update” button at the end of the web page. The IDS will restart now to apply the changes.

It is highly recommended that you also install and activate the Guardian Add-On.

What rule provider is best and what rules are best?

An excellent answer from TimF

First some definitions for the sake of discussion:

  • Ruleset - a complete set of rules coming in a single download. Contains a number of rulefiles. These are what you select in the pull-down on the WUI.
  • Rulefile - a set of rules that address a similar problem area. Contains a number of rules. These are what you select with the check-boxes in the WUI when the page comes up initially.
  • Rule - a set of instructions for detecting a condition.

Note that rules are not necessarily independent - for example, a rule to detect malware in a PDF file will rely on another rule that detects that the traffic is actually a PDF file - if the latter rule isn't enabled, the former rule won't work. The basic detection rules are generally in emerging-policy, emerging-info, and file-identify (I think) - but there are other dependencies.

The rulesets are:

  • Talos VRT Subscription. This is probably the best ruleset - apparently, they expend quite a bit of effort in order to ensure that the ruleset is of high quality. Downsides - it's generally only updated twice a week and you have to pay for it. Note Talos VRT was formerly known as Sourcefire.
  • Talos VRT Registered. This a one month old version of the subscription ruleset. The quality is the same, you don't have to pay, but it is a month old.
  • Emerging threats open. This is a free ruleset. It's smaller than the Talos VRT ruleset, and is therefore less comprehensive. It's updated every weekday. These rules are in rulefiles starting with 'emerging-' on the WUI.
  • Community rules. While this is available separately, it's also included in all the other rulesets, although it's a month out of date in the Talos VRT Registered ruleset and of uncertain age in the Emerging Threats ruleset. Note that this ruleset is curated by Talos VRT. If you download this separately, these rules are in the rulefile 'community.rules' on the WUI.

Note that you can have multiple rulesets installed, which is why the available rulefiles doesn't seem to change when you download a new set. This isn't a problem, but if you really want to get rid of an old ruleset do the following:

  1. Log on to the system via ssh.
  2. Delete the appropriate rule files from /etc/snort/rules:
    1. Community rules - community.rules
    2. Emerging threats - rulefiles starting with emerging-
    3. Talos VRT - any other rulefile.
  3. Edit /etc/snort/snort.conf and delete the include lines at the end of the file corresponding to the rule files you've deleted - don't just comment the lines out.

So, what ruleset should you use? For the sake of completeness (since other people might view this post later), I'll cover more scenarios than just yours. Note that you may well be constrained by how fast your IPFire computer is and how much memory it's got.

  • Large business/organisation. You should either have your own cyber security team or a contract with a specialist company. Ask their advice - if they can't give it or can't explain it fire them and get someone better. The answer won't be 'load this ruleset on a computer running IPFire' (or if it is, fire them) - it'll be more complicated and, of course, more expensive. This is also the advice for any organisation where the consequences of compromise are serious (for example, hospitals, utilities etc), no matter what size.
  • Medium sized business/organisation. Consider if you need a cyber security team, but the minimum would be the Talos VRT Subscription with a large number of rules enabled. If you're using my automatic update script you would be aiming to use the Security-over-Connectivity policy.
  • Small business/organisation. Consider the consequences of a compromise. If they're serious, either to you or to someone else (don't forget your responsibilities under the GDPR), you should be using the Talos VRT Subscription, otherwise you may get by with either Talos VRT Registered or Emerging Threats.
  • Home use. The Emerging Threats ruleset is probably sufficient, but you could use the Talos VRT Registered set. A Policy of Balanced-between-Security-and-Connectivity is probably sufficient. If you volunteer for a charity or similar and as a consequence keep either personal or financial information on your home network, you should consider the Talos VRT Subscription, but you should be eligible for the personal use license, which is much cheaper.

As well as this, if your computer has limited processing power or memory, it may tip the balance towards the Emerging Threats ruleset.

This is all in my opinion, and I am not a cyber security professional.

So what rules do I use? None of the above scenarios. Note that (probably due to security briefings long ago) I tend to be careful a little more careful than many people on these matters.

I run two IPFire systems: one at home, and one for a small charity. In both cases, I use the Talos VRT Registered, Emerging Threats open and community rules. I primarily use the Talos VRT set, with the community ruleset to bring the community rules up to date and then I add the emerging-current-events full file to address newer threats than the month old Talos VRT registered rules. In consequence, I have about 10000 rules enabled. I also run an IP blocklist (more on that later).

I have no problems with the Talos VRT ruleset. I think that the reason for the reports of problems is that the Emerging Threats rulesets, by default, has IP blocklist rules enabled. These generate an alert when a packet is received from one of the IP Addresses; you can get from several hundred to several thousand of these per day, however this is misleading because these packets would be blocked by the firewall anyway. If you disable these rulefiles you'll get very few rules alerting - most days I don't get any. This is a good thing - if you're getting several hundred alerts dues to IP addressing being blocked, it's unlikely you'll spot the small number of messages that tell you you've actually been compromised. The Talos VRT rulesets don't have equivalent rules enabled and so you don't get the large number of spurious messages.

Do you need IP Blocklists? If you're not exposing any services to the internet then probably not. The firewall's default rules will block any unsolicited traffic unless it happens to hit a port that's open due to an outgoing connecting, and in that case the application using the port will almost certainly reject the traffic.

If you're providing a service visible from the internet then a blocklist is probably a good idea. In this case rather than using the IDS rules I would use one of the scripts that are intended to install a blocklist in the firewall - this solves the problem of not being able to see important IDS alerts. There's at least three scripts about, but I can only find two at the moment:

The reason for not using the Emerging Threats rulefiles for doing this blocking is twofold: Firstly, it's inefficient - blocking using the firewall uses less memory and less processor power, and secondly, as mentioned above, you tend to get so many alerts triggered by these rules that you can't see the alerts you actually need to worry about.

What rulefiles should you enable? That's going to depend on the use you make of the system, the amount of memory you've got and the speed of your computer in reference to the speed of the internet connection.

In general, the Emerging Threats rulefile for a threat are will have fewer rules than the equivalent Talos VRT rulefile, so if you're short of memory or processing power go for the Emerging Threats version. I also don't see any point in including both the Talos VRT and Emerging Threats rules for the same topic.

So, go through the list of rulefiles and decide which ones are relevant to your situation. Bear in mind that you may have to include some rulefiles you don't expect, like the emerging-policy, emerging-info, and file-identify ones that are relied on by rules in other rulefiles. Also, I remember reading an article an number of years ago that WINE had advanced to the point where it was capable of running some Windows malware, so even if your computers are all Linux, it may be appropriate to enable some Windows rules. The same applies to Open/Libre Office and Microsoft Office - each can open the other's files.

In this case, the list of rulefiles looks reasonable, but you need to enable emerging-policy and emerging-info, and double check that you're not using both the Emerging Threats and Talos VRT versions of a rule.

Once you've set up some rules, let them run for a while and then check memory usage (Status → Memory from the WUI). If it looks like you're running short of memory you'll have to use fewer rules. Also check the processor usage (Status → System). Again, if you're running short of processor power you need to use fewer rules.

You also need to check for IDS alerts (Logs → IDS Logs), preferably daily. You may well get the occasional report of malware from someone on the internet probing your system, but if you get lots of reports of malware then you'll have to track down the device that's responsible and clean it (which is another topic).

Finally, you need to keep your rules up to date. There are new threats appearing everyday, so this is vital. You can do this from the WUI by selecting the ruleset and then downloading it, or you can use one of the scripts:

If you use the last script it will remember the enable/disable state of individual rules and you can also select a policy which determines whether new or changed rules are enabled or disabled.. The drawback is that this script uses a lot of memory when it runs. Using the other script or a manual update will reset the enable/disable state of individual rules to a default (which corresponds to the policy balanced-between security-and-connectivity. This may or may not be a problem. If you find that something you are doing on your network triggers rules and that leads to guardian blocking the device, you then either have to use the second script or alternatively do manual updates and resign yourself to having to change the rule state each time.

I hope this helps answer your questions. Unfortunately, there's no simple answer, and you may well have to make adjustments to get everything working correctly. I had to do quite a bit of work initially, but now my systems are in a state where I just need to check the logs regularly, otherwise they run quite happily without my intervention.

Further readings

configuration/services/ids.txt · Last modified: 2018/10/02 09:05 by Rafael Reinoso