Note this is a BETA version of the FAQ and the contents within are subject to major revisions and may contain large errors.

Vulnerability Detection Systems (VDS) FAQ

This FAQ is maintained by David Meltzer.  Please send any additions, corrections, or comments to me at .  If you are interested in adding or changing major sections of the FAQ, please contact me first to coordinate.

Current Version: v0.4 (Last updated 1/30/03)

Table of Contents

[0] Where do I get the latest revision of this FAQ?

The most current version of this FAQ is always available at: https://intrusec.com/vds.

Updates to the FAQ will be announced on the VDS mailing list.

[0.1] How do I join the VDS mailing list?

I will be setting up a loosely moderated mailing list regarding VDS shortly..

 I will update this FAQ once it is available, but if you would like to "pre-subscribe" to the list so you are notified as soon as it goes up, send an e-mail with "SUBSCRIBE" in the subject or text of message to .

More information about the VDS List including will be available at https://intrusec.com/vds.

[0.2] Who is the intended audience for this FAQ?

First, this FAQ assumes you have some basic knowledge about network security and are familiar with such things as vulnerabilities, scanners, and IDS.  If you don't, there are a zillion other web pages and FAQs that are most useful to read before this one, so bookmark this page and come back later.

This FAQ should be useful to the following kinds of people:

My goal with this FAQ is to help everyone learn about vulnerability detection systems, help administrators and consultants learn how to take advantage of their existing security tools and with a little bit of configuration and integration build their own vulnerability detection systems, and help software developers build vulnerability detection functionality into their existing and upcoming security products.

[1] What is a Vulnerability Detection System (VDS)?

A vulnerability detection system (VDS) is a continuously monitoring, always-on system that can detect and alert administrators to the presence of vulnerabilities as they appear.  Think of it conceptually akin to an IDS except instead of constantly monitoring for hackers attempting to break-in, you are constantly monitoring for vulnerabilities hackers COULD use to break-in, before they actually do.

A vulnerability detection system is a type of monitoring system that you can build yourself out of existing security products you are probably already using by doing a little bit of tweaking and integration work.  Of course, vendors are sure to come along to make that easier for you, but the main advantage of pre-built solutions will be to save you time and money in building, deploying, and managing these systems, and be wary of those who claim otherwise (See 1.7).

As with most security systems, VDS can be either host or network-based, but this FAQ in particular is mostly concentrated on network-based VDS as it seems to be the more interesting and growing area. 

Here is a chart that describes the 4-quadrants of security, Vulnerabilities vs. Intrusions on the X-Axis and Auditing vs. Monitoring on the Y-Axis:

         Vulnerabilities        Intrusions
                           |
                           |
Audit         Security     |       Log
              Scanners     |     Analysis
                           |
         ------------------+---------------------
                           |
                           |
Monitor        *VDS*       |       IDS
                           |
                           |

 

[1.1] Is VDS the same thing as a vulnerability scanner or security assessment tool?

No, but a vulnerability scanner is the closest relative to a VDS and they share a common goal.  Vulnerability scanners take a snapshot of a system and report the vulnerabilities that appear at that point in time. In contrast, a vulnerability detection system is continuously monitoring a network for the appearance of new vulnerabilities so that if one appears, administrators are instantly alerted to the presence of it.  A good way to think of it is that vulnerability scanners audit periodically and VDS monitor continuously.

 

[1.2] Is VDS the same thing as IDS?

IDS and VDS are very similar in that they both provide continuous monitoring of systems and send real-time alerts (or other actions) when something is detected.  The difference is in what they are monitoring for.  Intrusion detection systems look for hackers attempting to exploit vulnerabilities in a network. In contrast, a vulnerability detection system looks for those vulnerabilities that a hacker would want to exploit. By detecting vulnerabilities instead of intrusions, administrators can find out about a vulnerability and take action to close it or block its exploitation BEFORE a hacker finds it. 

Sometimes, and especially when it comes to host-based IDS, there can be some overlap in functionality because the same techniques are being used to detect intrusions as potential vulnerabilities.  For example, a host-based IDS might monitor critical configuration files for changes that indicate a hacker has broken into it.  A host-based VDS might monitor that same critical configuration file for changes that indicate a new vulnerability has opened up due to a configuration change.  There are also situations where there is debate over whether something is an intrusion or a vulnerability or both.  For example, a backdoor on a system is both a sign of an intrusion and the existence of a vulnerability.

Even though IDS and VDS are very similar on a conceptual level, from the standpoint of how they work under the hood they are completely different.

 

[1.3] Is VDS the same thing as correlation engines or security infrastructure management software?

Security management systems take in alerts from a variety of engines, including firewalls, vulnerability scanners, and intrusion detection systems.  A VDS is another type of engine that reports data in an alert format and is naturally suited for delivering that data into a larger management framework.  The information will look exactly like an IDS, except it is reporting vulnerabilities instead of intrusions. 
 

[1.4] Does VDS replace existing security tools?

Conceptually, no.  Clearly VDS does not replace IDS, as they are detecting entirely different things, and since VDS is not the magical tool that secures your entire network with no effort (see 1.7), you still need to worry about hackers and intrusion attempts.

There are two reasons you still need vulnerability scanners even with a VDS:
    1. NEW vulnerabilities - When a new vulnerability is discovered, you still need to scan for that vulnerability when a check for it is added to your favorite vulnerability scanner. 
    2. Auditing - Monitoring for vulnerabilities augments periodic audits, it does not replace them.  The most pressing reason to do this is due to the possibility of 'false negatives' in a VDS, where a new vulnerability appears but is not detected.  False negatives are always a possibility in any VDS that operate in a more complex fashion than running a vulnerability scanner over and over.

Another way to think of why you should still use a vulnerability scanner is to look at modern antivirus software.  While most AV software will detect a virus when it comes into your system (via disc, e-mail, download, etc.), vendors recommend and most people still run those weekly scans of your hard drive for the same two reasons listed above: detect previously unknown virii and in case something slipped by the monitoring system. 

All that being said, since VDS is in the same ballpark as both vulnerability scanners and IDS, it is not and will not be uncommon for vendors to combine several of these tools into a single integrated product or appliance and sell it as a 'complete' solution. 

[1.5] Do VDS work in real-time?

Real-time is not in the actual definition of VDS, just that it is continuously monitoring and always-on.  Most VDS are not real-time, because there will be some gap time between the vulnerability actually appearing and the VDS detecting it.  Whether a VDS works in real-time or not depends on exactly how it works.  Typically, anything that relies on passive means of detection will operate in real-time, and anything that utilizes active 'polling' techniques will be "near real-time" (do not infer from this that passive VDS are better than active VDS, there are trade-offs to both).

On the other hand, from a practical perspective, real-time is always going to be relative (1ms? 1 second? 1 minute? 1 hour?).   If you are accustomed to detecting vulnerabilities within a day or week of their appearance with your current security methods, and with a VDS you are able to detect them in a matter of a few seconds or minutes after they appear, that may be pretty real-time to you.

There is some upper limit on how long you are practically willing to wait before a vulnerability is detected that you will still consider something a vulnerability detection system and not a auditing system.  I would be suspicious of calling something a VDS if in a typical situation and configuration the "lag time" between a vulnerability appearing and being detected exceeded a few hours.  From a marketing perspective, however, the more popular VDS becomes, the more you will probably see commercial companies declaring they are a VDS in name or functionality regardless of what it actually does.

 

[1.6] How does a VDS protect my network?

It doesn't.  Like many other security tools, a VDS gathers and reports information administrators need to protect their networks.  Like an IDS, there are methods you can use to turn detection into protection if you are willing to take a more pro-active approach to security on your network and turn your vulnerability detection system into a vulnerability protection system  These techniques include:

 

[1.7] Is VDS a new idea?

No.  First you have to give credit to all the creators of vulnerability scanners and IDS from whom VDS borrows conceptually.  There has been a continuous movement over the last several years towards making vulnerability scanners run faster, scheduled, and providing only changed results.  You could say any vulnerability scanner that can be run in a while loop and output differences in results is at its core a vulnerability detection system, and that covers almost every commercial and open-source vulnerability scanner made. 

Of course, modern VDS are markedly different from that simplistic model, but improvements in technology are different from new ideas...

Also, there is apt to be some argument over whether VDS is an appropriate or useful term to describe this type of software.  This is apt to be a never-ending and useless argument for a couple of years until its been used so many times its too late to change it.

I figured it was time for a FAQ because I've seen an increasing number of people talking about these kind of systems, usually without specifically calling them a VDS or providing perspective on the general concepts and tools already out there, and making them seem a lot more magical than they really are, and expect over the next few months and years this is going to become a very hot topic within the security industry.

[Ed Note: I'm soliciting specific references I can add to this section for people that first suggested VDS-like ideas in public forums, if you have any such references please e-mail me.]

 

[1.8] Is VDS the silver bullet that will magically secure my entire network?

YES!  Just kidding.  No. 

Some specific reasons why VDS does not secure your network:

As almost every security expert will tell you, there are no silver bullets in security and the best approach is a multi-layered one that utilizes a variety of tools and techniques.  I would categorize VDS as another valuable tool that many security-minded administrators will find useful in helping them manage the security of their networks.

 

[2] How do Vulnerability Detection Systems Work?

Vulnerability Detection System describes a class of security tools and actually building one can be done using a variety of techniques.  Each one of these techniques can be used on their own to develop a vulnerability detection system, or they can be combined to form a more comprehensive VDS:

 

[2.1] What are Continuous Vulnerability Assessments?

This is the most basic method to build a VDS.  You can think of this as running a vulnerability scanner in a loop or on a schedule.  In actual implementation, continuous vulnerability assessments require more stringent requirements that should be considered  These particular qualities include:

[2.2] What is Passive Vulnerability Detection?

Passive vulnerability detection is using a passive monitoring system (in the network case, either a sniffer or an in-line packet processing device like a firewall or IPS) to detect the existence of vulnerabilities.  In contrast to active vulnerability assessment, which requires probing the actual systems to determine if they are vulnerable, a passive vulnerability detection system detects vulnerabilities looks at regular network traffic coming to and from systems to indicate signs of a vulnerability.

An example of passive vulnerability detection signatures would be looking at an SMTP banner anytime a connection is made to a server to see if that service is running a vulnerable version of that server.   The biggest advantage to passive vulnerability detection is that it takes place without any impact on the services being monitored.  However, since not all vulnerabilities can be detected without directly interacting with a server, and since vulnerabilities will be only be detected if regular user connections are being made to the server, it can only offer partial vulnerability coverage and is most useful to augment active vulnerability detection techniques.

A good paper discussing this area in more detail is "Passive Vulnerability Detection: Techniques to Passively Find Network Security Vulnerabilities" by Ron Gula, available at  http://downloads.securityfocus.com/library/pvd.html. 

Many, if not most, existing network IDS contain some number of passive vulnerability detection signatures, though few, if any, quantify or clearly differentiate which signatures are vulnerability detection vs. those that are intrusion/exploitation signatures.  If any IDS vendors would like to offer specific lists of signatures they have that full into this category, or can point me to a reference where they have done so, I'll update this section accordingly.
 

[2.3] What is Active Change Detection?

Active change detection is periodically testing something (a system, device, application, file, etc.) for differences.  This is a very old idea - tripwire, first created in 1992, is probably the most popular change detection tool at the host level.  Change detection is an important component of a vulnerability detection system because changes are the source of most vulnerabilities on a network (think of how much easier it would be to secure your network if nothing ever changed on it).

Here are some examples of changes on a network that are likely to introduce new vulnerabilities:

Network-level active change detection is conceptually a bit newer idea, but you could categorize anything as simple as a port scanner that can report diffs as such a tool.  An example of simple network-level active change detection would be detecting if any new systems have appeared on their network.  A simple way to do this would be to periodically ping their IP address range and note if a new machine is now responding to pings (of course this method has its drawbacks and there are numerous other ways to skin that cat). 

The biggest advantage of using network-based active change detection methods is that you can operate those tools like a vulnerability scanner - deploy it at one (or at most a few) points around your network and you can monitor your entire network with it.

Active change detection by itself typically only provides alerts of a violation in policy or that a POTENTIAL vulnerability exists.  For example, you might build an active change detection system that connects to and checks the banner of your SMTP server every half hour so that if the e-mail admin down the hall gets any funny ideas about upgrading versions or switching to a different server without asking your permission (you know the security guys are always in charge, right?) you will get an alert.  So, the banner changes, you get alerted.  Is that in and of itself a vulnerability?  Probably not. Do you want to know about it?  Probably.  Many popular services allow some type of versioning although the complexity of retrieving that version can vary. 

Active Change Detection (this also applies to the other forms of change detection listed below) has a lot of advantages over continuous vulnerability assessments, but to highlight a few:

On the other hand, it also has some disadvantages:

Active change detection by itself is an excellent way to stay abreast of what is going on on your network and locate vulnerabilities that may crop up even though there is no particular signature or detection method for it in your vulnerability scanner.  The trade-off is if you are not careful about how you use it, you may be overwhelmed with the 'noise' of changes happening, and if you lose the real vulnerabilities as the noise goes by, you are no better off than not having used it at all.

 

[2.4] What is Passive Change Detection?

Whereas active change detection relies on periodically polling, passive change detection relies on constant monitoring.  At a host-level, passive change detection can often be accomplished using event-driven system and API calls that let the change detection tool sit idly by until it is woken up by something happening (an entry written to an event log, a directory changing, etc.).  On the network-level, passive change detection typically means listening to network traffic (either with a sniffer (a typical network IDS) or some in-line network device such as a firewall or intrusion prevention system).

For example, in the active change detection section, I described ping scanning your network to locate new systems.  With passive change detection, instead of pinging, you could simply listen to a network segment for new IP addresses in your address range responding (yes, you could also do it at link-layer or a dozen other ways). 

Passive change detection has a lot of advantages, most notably that you can often detect changes in real-time (See 1.5).  The other advantage is that, like a network IDS, passive change detection can operate stealthly without utilizing any network resources or revealing its existence to others.

I've heard a variety of other creative ideas for using passive change detection to replicate active change detection functionality.  For example, in the previous section the administrator checked the banner of their SMTP server every hour.  Instead of actually connecting to the SMTP server, one could simply write an IDS signature that looked for deviations from the baseline signature, and whenever anyone connected to the server it could be checked by the IDS (obviously there are many considerations to implement this idea efficiently and in an easily manageable manner including automating the signature updates, preventing replications of alerts, and throttling the frequency of the signatures being run, to name a few).

 

[2.5] What is Change Management?

Active and passive change detection are great ways for administrators to add-on security to a network that isn't totally under their control (most networks fall under that).  However, if you do have some control over your network, you can skip the middle-man and integrate your vulnerability detection system directly with your change management system.  That is, if you already have procedures in place somewhere in your organization that restrict how, where, and when someone can plug a new device into your network, you should use those procedures to feed information into your VDS.

For example, if you have a ticketing system in place, you may have a specific type of ticket generated anytime a new server is to be assigned a static IP address.  With a little integration work, you can have your ticketing system send an alert to your security management system with that new IP address when it is assigned.  If you can know for sure a new system is about to come up on an address, you know exactly where potential new vulnerabilities are going to appear, and you can watch closely for what happens. 

Conversely, you can use ticketing system integration to detect the absence of events.  For example, if you have a static address range that new systems should not be using unless IT has specifically assigned an address, you could integrate that with your ticketing system so that if a new host is detected in that range but a ticket assigning that address has NOT been filed, you want to be notified.

As far as actually implementing this, you may be able to use tools your security management system provides to add this logic (If host H receives an Alert type B without Alert type A, assign risklevel = HIGH, otherwise assign risklevel = LOW), but even if you were using nothing more sophisticated than nmap, with grep and a few lines of script you could implement this same change detection logic.

 

[2.6] How do you integrate change detection and vulnerability assessment?

A good way to compensate for the 'noisy results' problem of change detection is to integrate your change detection system with a vulnerability scanner.  By configuring and launching a vulnerability scanner against specific devices, applications, or services after a change has been detected, you can maintain all the advantages of using change detection (minimal invasiveness, speed, vulnerability coverage) and minimize the costs.  If you are primarily interested in detecting known vulnerabilities, you may completely filter out the change detection results and simply use change detection as a front-end tool for managing your vulnerability scanner. 

A more likely scenario is use both together.  You may have one critical service on your network that you consider any change to be a high-priority, but all other services you will tolerate changes (you just want a low-priority notification when they occur), but you would like a non-invasive vulnerability scan launched against those services anytime they are changed (and if real vulnerabilities are found, those are high-priority results). 

Some vulnerability scanners are easier to integrate in this manner than others, but in general the considerations for integrating change detection with vulnerability scanners are the same as with building a continuous vulnerability assessment VDS (See 2.1). 

 

[2.7] How does SNMP relate to change detection?

SNMP is one of the best ways to detect changes across a variety of hardware and software.  If you have a network management system, you probably already are monitoring a myriad of systems using SNMP, and building a VDS out of your existing network management is a great step to take.

SNMP can be either active change detection (using SNMP GETs) or passive change detection (using SNMP Traps). 

For example, say you are using DHCP on your network and want to know about when new hosts appear on your network.  We described previously how you could actively detect new hosts (ping scan) or passively detect new hosts (sniff for IP addresses or ARP packets).  Using SNMP, you could instead configure your DHCP server to send an SNMP trap to your VDS anytime a new lease is issued for an IP address.  It would be a relatively easy integration task to receive that SNMP trap, perform a scan against the new host, and deliver the results of that scan to your security management system. 

 

[2.8] How do I prevent my IDS from generating thousands of alerts during vulnerability assessments?

Anyone who has scanned a network while running an IDS knows the annoyance of getting 500 events caught by the IDS.  If you are using a VDS that does continuous vulnerability assessments (See 2.1), you will now be faced with the pain of getting 100 events a minute, every minute, caught by your IDS.  The biggest danger you face when a scan is ongoing is that a hacker will time attack attempts so that his malicious attacks are lost in the legitimate scans.  Careful coordination is required to properly filter out all the VDS actions from those of the hackers. 

The most accurate technique to use if you have an IDS, VDS, and security management system that can correlate events, is to have your VDS or vulnerability scanner generate an alert for each vulnerability it ATTEMPTS (not finds) against a host.  This should be an 'informational' alert to your management system (these could be sent individually or in batch).  You then can create a rule in your management system that will downgrade the risk of any attack that is seen which also a corresponding informational alert from the VDS system coming from the same host.  Optimally, this would be configured so that the VDS only delivers alerts for vulnerabilities that are both currently being monitored for by the IDS and would trigger an alert from the IDS during its normal operation of the scanner.  Management systems in this case need to be particularly watchful to avoid correlating multiple events, as if two attacks are seen instead of one, one of the attacks could very well be that of a malicious attacker.

A less intensive technique to consider would be to simply correlate the source and destination address of a scan taking place with attacks, such that any attack coming from that address are considered part of a scan.  However, correlating at the host level instead of the vulnerability level could allow a sophisticated attacker to spoof attacks from that system and avoid detection for a limited number of checks and situations (UDP-based and other connectionless attacks particularly).  While less than ideal, in truth most human security analysts manually perform this type of operation when discarding attacks that come from internal vulnerability scanners.

 

[3] Who is making VDS software?

Here is a list of vendors and projects that make vulnerability detection systems or have tools that help people build their own vulnerability detection systems.  I have not included in this list any vulnerability scanners, intrusion detection systems, or security management systems, because there are other comprehensive lists of those already.

If you are a vendor (or know of one) and would like to be added to the list or modify your company's blurb, e-mail me.

[Ed Note: I am withholding adding entries to this section until I have a semi-comprehensive list and can include everyone.]

 

[4] Acknowledgements 

This FAQ is written and maintained by David Meltzer.

The following people have also contributed to this FAQ:

Copyright © 2003, David J. Meltzer.  All rights reserved.

This document may be redistributed only in its entirety.  It may not be used in whole or part for profit or in any commercial document without the express written permission of the copyright holder.  Mirrors of this document may be made only if the author is notified and the mirrored copy is kept up to date.