Methods for Finding Security Issues in Software

1.1 Introduction

The research question for the chosen topic will be “How can we improve the detection of security issues in software”.  This is important because software security has been and is a growing issue in the current century, especially with the introduction of new and exciting technologies that need tight and robust security systems.  The ability to find these issues is important as finding them will help to improve the security of every piece of new technology that is made, vastly improving the quality of the product.


1.2 Security Vulnerability Detection Techniques

There are a lot of different techniques that can be used to find security vulnerabilities.  This paper will outline just three; fuzzing, taint analysis and model checking.


1.2.1 Fuzzing

Fuzzing involves providing your program with randomised inputs to find test cases that may cause a crash.  It can give a quick overview on the overall robustness and help fix critical bugs.  It will not find all the bugs in a program as it may not trigger the paths required to find them.  There is a way to make fuzzing more efficient.  Using a Smart Fuzzer will vastly improve the tests.  Smart fuzzing is when you program the fuzzer to have knowledge of the input format, therefore the fuzzer can construct tests that relate to that part of the input.  The higher the intelligence, the better the fuzzer[1].
There are 5 different types of fuzzers; Mutation, Replay, Man-in-the-Middle, Generation and Evolutionary.  They all do roughly the same thing.
Mutation is known as a dumb fuzzer as it randomly mutates valid inputs to produce malformed inputs.  The easiest way of doing this is by replacing a random section of data.  Dumb fuzzers are lacking in understanding of the structure of the data[2].
Replay is when you take a saved sample input and simply replay them after mutation.  This will work well for file format fuzzing where a number of files can be saved to provide to the target program.  However, it can be more difficult if the protocol is simply non-replayable[1].
Man-in-the-Middle (MITM) is when a you place yourself in the middle of a client and server, intercepting and possibly modifying messages passed between them. You will be acting like a proxy. You can either fuzz the server or the client.  This fuzzer could have no knowledge about the protocol and simply randomly alter the some messages, or it could modify target messages.
Generation is known as a smart fuzzer as it generates input from scratch rather than mutating an existing input.  Smart fuzzers have an understanding of the file format and therefore can do more than dumb fuzzers.  Generation can get deeper into a protocol easier than other non-Generation based fuzzers[2].
Evolutionary fuzzing is advanced.  It allows the user to take feedback from each test case and learn the format input[1].
A fuzzer needs to perform a number of tasks : Generate test cases, Record the test cases, Interface with the target program, Detect crashes and split tasks into different modules[1].
There are different things to consider when writing test cases such as what the output should be for the protocol and the extremities of the code eg. Max and Min integer values, long blank strings, nulls etc…[1]
Reproducing the test case is very simple. Just record the exact input used when a crash is detected and then re-run the inputs.
Interfacing is straightforward. For network protocols, it may be as simple as sending the test case over the network.  It may be harder in some cases as the test case may not be easy to send over a network and will require some creative thinking to get it over the network.
Crash detection is crucial in any testing technique but especially crucial in fuzzing because if you can determine when a program has crashed then the fuzzing won't work.
Testing the effectiveness of fuzzers is very important and can be done using a few different techniques[3].
Retrospective Testing is the most straightforward way of evaluating a fuzzers effectiveness.  In this form of testing, a particular time period, x,  is chosen.  The fuzzers from x time ago would then get archived.  This x time period would then be analysed for any vulnerabilities discovered in any implementation.  The old fuzzers would then be used to test against the flawed implementations and a record would be kept of the outcomes.  The reason the old fuzzers are used is that fuzzers may have been updated recently to find different vulnerabilities[3].
Code Coverage is important when testing any software as it lets you know how much of the code the tests actually cover.  There are many weaknesses to using this with fuzzers as it doesn't actually measure the effectiveness of finding the bugs, just how much of the code wasn't tested[3].
Simulated Vulnerability Discovery involves selecting an implementation and getting an experienced vulnerability analyst to add a variety of vulnerabilities to this implementation.  A different analyst then proceeds to fuzz the flawed implementation to try and evaluate how many bugs are re-discovered.  This is a good form of testing as it tests for bugs that are actually present and not just simulated[3].
Overall, fuzzing is a good way of testing for software vulnerabilities as they can be programmed to go deep into a protocol or just test for low level bugs. They can record the tests and replicate the inputs.  They are easy to implement and can be left running for a long time with no inputs.






1.2.2 Taint Analysis

Taint analysis can be seen as a form of Information Flow.  It works by





1.2.3 Model Checking