[klee-dev] Want to replay ".ktest" files from a single text file

[Sang Phan]

Hi Sanghu,

Your problem is very popular in fuzzing: running the same program with
millions of (mutated) inputs.
Perhaps you may want to look at how a fuzzer solves that problem, and
do something similar:
https://lcamtuf.blogspot.com/2015/06/new-in-afl-persistent-mode.html

Best,
Sang

On Sun, Jun 17, 2018 at 8:02 PM, Andrew Santosa <asantosa1999 at gmail.com> wrote:
> Hi Sanghu,
>
> If I were you I would try to parallelize the replay runs using a Makefile.
> For example, you can implement a Makefile that looks like this:
>
> -----------------------------------------------------------------------------
> KTEST_FILES=$(wildcard klee-last/*.ktest)
>
> OUTPUT_FILES=$(KTEST_FILES:.ktest=.output)
>
> all: ${OUTPUT_FILES}
>
> .SUFFIXES: .ktest .output
>
> .ktest.output:
>     KTEST_FILE=$< ./a.out > $@
> -----------------------------------------------------------------------------
>
> (Note that the last line should start with a tab character).
> You can then execute:
>
> make -j `nproc`
>
> to execute the test cases in parallel.
> Note that make will stop as soon as there is a replay
> that returns non-zero value.
>
> Best,
> Andrew
>
>
>
>
>
> On Sunday, 17 June 2018, 11:22:51 PM SGT, Sangharatna Godboley <sanghu1790 at gmail.com> wrote:
>
>
>
>
>
> Hi Andrew,
>
> Thanks for your response. I am using libkleeRuntest library. I am following the instructions provided in the following link to replay the test cases:
>
> https://klee.github.io/tutorials/testing-function/
>
> I have tried implementing the loop using a programming language that iterates over all of the ktest files. However, the issue is that the speed is still slow on 100K test replays. I am trying to find a way to natively run the 100k klee test cases on the LLVM IR. To increase the speed of test replays I am thinking to avoid opening and closing of test input files 100k times.
>
> Is there any way to resolve this issue?
>
> Thanks,
> Sanghu
>
>
> On Sat, Jun 16, 2018 at 10:30 AM, Andrew Santosa <asantosa1999 at gmail.com> wrote:
>> Hi Sanghu,
>>
>> You don't seem to be using klee-replay here. Otherwise, instead of invoking a.out
>> you would have invoked klee-replay, with a.out as its argument.
>>
>> Moreover, you can't just expect klee-replay to work on a single file containing all the
>> ktest files by simple appending. I don't think this file will be in the right format for a
>> ktest file. Instead, you should implement a loop using your programming language
>> of choice that iterates over all of the ktest files and runs klee-replay on each.
>>
>> Best,
>> Andrew
>>
>> On Thursday, 14 June 2018, 5:06:05 PM SGT, Sangharatna Godboley <sanghu1790 at gmail.com> wrote:
>>
>>
>>
>>
>>
>> Hi,
>> I have a problem. For my experiment, KLEE generates larger test data for eg. > 100,000 "test000001.ktest" files. I am replaying these test input files using klee-replay tool with this command
>>
>> "KTEST_FILE=klee-out-0/test000 001.ktest ./a.out".
>>
>> Here is the problems, I need to open and close ".ktest" files for "a.out" 100,000 times, which is time consuming. I want to optimize the time. I want to avoid opening and closing of ".ktest" files for these many times. Is there any way to solve this problem? I am thinking to redirect all these test input files into one "sample.txt" file using the following command:
>>
>> "ktest-tool klee-out-0/test******.ktest > sample.txt"
>>
>> But, I am now stuck to process this sample.txt with ./a.out by using klee-replay tool.
>> Can anyone help me for this problem?
>>
>> Thanks
>> Sanghu
>>
>> ______________________________ _________________
>> klee-dev mailing list
>> klee-dev at imperial.ac.uk
>> https://mailman.ic.ac.uk/ mailman/listinfo/klee-dev
>>
>
>
>
> --
>
>
>
>
> _______________________________________________
> klee-dev mailing list
> klee-dev at imperial.ac.uk
> https://mailman.ic.ac.uk/mailman/listinfo/klee-dev