All ETDs from UAB

Advisory Committee Chair

Robert M Hyatt

Advisory Committee Members

Steven J Bethard

Alan P Sprague

Document Type


Date of Award


Degree Name by School

Master of Science (MS) College of Arts and Sciences


Antivirus software vendors have been forced into an industry of reaction because an abundance of new malware is discovered daily. The antivirus industry, handicapped by a low ratio of malware analysts to malware and antiquated techniques, floods the market with products with regrettably low detection rates, often only marginally outperforming competitors. This thesis presents a new function-edge-based set detection and identification method which significantly outperforms traditional antivirus techniques. The technique used reduces a malware binary to its fundamental assembly code and performs functional analysis using a newly created function hashing technique. Function hash edges are created by combining calling and called functions into pairs. Each malware sample can be described as the set of function hash edges it generates. Multiple malware samples are grouped into unique parent-child function trees and each tree produced using this technique then represents the respective malware family to which these samples belong. These trees not only help correctly identify which family of malware a sample belongs to, but the tree itself can be used as a detection mechanism for any future versions of these malware samples. This new detection method provides many consumer level benefits because it can learn based on functional signature sets. The technique can be used to protect from future generations of a previously detected form of malware, therefore giving it an advantage over every other antivirus product on the market. In addition to these and many other consumer level benefits, this technique can also be used as an investigative or analytical tool in malware research and criminal investigations. It is also useful when questioning author attribution and code sharing. Additional implications of this new technique include uses as a tool for source code theft investigation and unauthorized use of compiled libraries.



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.