diff --git a/README b/README index f2793ed..8855644 100644 --- a/README +++ b/README @@ -35,33 +35,32 @@ community resources. The signatures that LMD uses are MD5 file hashes and HEX pattern matches, they are also easily exported to any number of detection tools such as ClamAV. -The driving force behind LMD is that there is currently limited availability -of open source/restriction free tools for Linux systems that focus on malware -detection and more important that get it right. Many of the AV products that -perform malware detection on Linux have a very poor track record of detecting -threats, especially those targeted at shared hosted environments. - -The threat landscape in shared hosted environments is unique from that of the -standard AV products detection suite in that they are detecting primarily OS +The driving force behind LMD is that, currently, there is limited availability +of open source/restriction free tools for Linux systems for malware detection +and, crucially, that do it right. Many of the AV products that perform malware +detection on Linux have a very poor track record of detecting threats, +especially those targeted at shared hosted environments. + +The threat landscape in shared hosted environments is distinct from what standard +AV produtcs are built for. Those detection suites are primarily intended for OS level trojans, rootkits and traditional file-infecting viruses but missing the ever increasing variety of malware on the user account level which serves as an attack platform. -Using the CYMRU malware hash registry, which provides malware detection data -for 30 major AV packages, we can demonstrate this short coming in current -threat detection. The following is an analysis of 8,882 MD5 hashes that ship -in LMD 1.5 and the percentage of major AV products that currently detect -the hashes. +We can demonstrate this shortcoming using the CYMRU malware hash registry, which +provides malware detection data for 30 major AV packages. LMD 1.5 ships with 8,882 MD5 +hashes. Following, we have a breakdown of how many of these hashes are detected by the +to-30 AV products -KNOWN MALWARE: 1951 +KNOWN MALWARE: 1951 % AV DETECT (AVG): 58 % AV DETECT (LOW): 10 % AV DETECT (HIGH): 100 UNKNOWN MALWARE: 6931 -What this information means, is that of the 8,883 hashes, 78% or 6,931 malware threats -are NOT detected by top-30 AV products. The 1,951 detected malware threats that are known -have an average detection rate of 58% among top-30 AV products with a low and high +This means that, out of the 8,882 hashes, 78% or 6,931 malware threats +are NOT detected by top-30 AV products. For 1,951 malware threats that are detected at all, +there is an average detection rate of 58% for all the top-30 AV products with a low and high detection rate of 10% and 100% respectively. This clearly demonstrates the significant lapse in user space malware detection that top-30 AV products currently provide. It is for this reason LMD was created, to fill a void, specifically for shared hosted environments. @@ -102,55 +101,47 @@ this reason LMD was created, to fill a void, specifically for shared hosted envi .: 3 [ THREAT SOURCE DATA ] -The defining difference with LMD is that it doesn't just detect malware based -on signatures/hashes that someone else generated but rather it is an -encompassing project that actively tracks in the wild threats and generates -signatures based on those real world threats that are currently circulating. +The defining characteristic of LMD is that it doesn't just detects malware +based on signatures/hashes that someone else generated. It is an all-encompassing +project that also **actively** tracks the real-world threats in circulation to +generate signatures. There are four main sources for malware data that is used to generate LMD signatures: -- Network Edge IPS: Through networks managed as part of my day-to-day job, -primarily web hosting related, our web servers receive a large amount of daily -abuse events, all of which is logged by our network edge IPS. The IPS events -are processed to extract malware url's, decode POST payload and base64/gzip -encoded abuse data and ultimately that malware is retrieved, reviewed, classified -and then signatures generated as appropriate. The vast majority of LMD signatures -have been derived from IPS extracted data. - -The network I manage hosts over 35,000 web sites and as -such receives a large amount of daily abuse, all of which is logged by our -network edge IPS. The IPS events are processed to extract malware url's, -decode POST payload and base64/gzip encoded abuse data and ultimately that -malware is retrieved, reviewed, classified and then signatures generated as -appropriate. The vast majority of LMD signatures have been derived from IPS + +1. **Network Edge IPS**: The network I manage hosts over 35,000 web sites and, as +such, receives a large amount of daily abuse, all of which is logged by our +network edge IPS. These events are processed to extract malware URLs, +decode POST payload and base64/gzip encoded abuse data. Ultimately, that +malware is retrieved, reviewed, classified and then are generated if necessary. +The vast majority of LMD signatures have been derived from IPS extracted data. - - Community Data: Data is aggregated from multiple community malware websites -such as clean-mx and malwaredomainlist then processed to retrieve new -malware, review, classify and then generate signatures. - - ClamAV: The HEX & MD5 detection signatures from ClamAV are monitored for + +2. **Community Data**: Data is aggregated from multiple community malware websites +such as clean-mx and malwaredomainlist, which then processed to identify new malware +and generate signatures. + +3. **ClamAV**: The HEX & MD5 detection signatures from ClamAV are monitored for relevant updates that apply to the target user group of LMD and added to the -project as appropriate. To date there has been roughly 400 signatures ported +project accordingly. To date, there has been roughly 400 signatures ported from ClamAV while the LMD project has contributed back to ClamAV by submitting over 1,100 signatures and continues to do so on an ongoing basis. - - User Submission: LMD has a checkout feature that allows users to submit + +4. **User Submission**: LMD has a checkout feature that allows users to submit suspected malware for review, this has grown into a very popular feature and generates on average about 30-50 submissions per week. .: 4 [ RELEASE UPDATES ] Updates to the release version of LMD are not automatically installed but can -be installed using the --update-ver option. There is good reasons that this is -not done automatically and I really dont feel like listing them so just think -about it a bit. +be installed using the --update-ver option. The latest changes in the release version can always be viewed at: http://www.rfxn.com/appdocs/CHANGELOG.maldetect .: 4.1 [ SIGNATURE UPDATES ] -The LMD signatures are updated typically once per day or more frequently -depending on incoming threat data from the LMD checkout feature, IPS malware -extraction and other sources. The updating of signatures in LMD installations -is performed daily through the default cron.daily script with the --update +The LMD signatures are updated typically once per day. The signatures in LMD +installations can be updated daily using the default cron.daily script with the --update option, which can be run manually at any time. An RSS & XML data source is available for tracking malware threat updates: @@ -186,17 +177,15 @@ php.pktflood.oey php.shell.rc99 php.shell.shellcomm .: 6 [ THREAT SHARING ] -I am a firm believer in not reinventing the wheel, for my own sanity or that -of others. As such all unique threat data is submitted to CYMRU & ClamAV so -that the open source and anti-malware community at large can grow from this -project. +All unique threat data is submitted to CYMRU & ClamAV, so that the open source and +anti-malware community at large can grow from this project. .: 7 [ CONFIGURATION ] The configuration of LMD is handled through /usr/local/maldetect/conf.maldet and all options are well commented for ease of configuration. -By default LMD has the auto-quarantine of files disabled, this will mean that +By default, LMD has the auto-quarantine of files disabled, this will mean that YOU WILL NEED TO ACT on any threats detected or pass the SCANID to the '-q' option to batch quarantine the results. To change this please set quarantine_hits=1 in conf.maldet.