[klee-dev] Want to replay ".ktest" files from a single text file

[Andrew Santosa]

Hi Sanghu,

If I were you I would try to parallelize the replay runs using a Makefile.
For example, you can implement a Makefile that looks like this:

-----------------------------------------------------------------------------
KTEST_FILES=$(wildcard klee-last/*.ktest)

OUTPUT_FILES=$(KTEST_FILES:.ktest=.output)

all: ${OUTPUT_FILES}

.SUFFIXES: .ktest .output

.ktest.output:
    KTEST_FILE=$< ./a.out > $@
-----------------------------------------------------------------------------

(Note that the last line should start with a tab character).
You can then execute:

make -j `nproc`

to execute the test cases in parallel.
Note that make will stop as soon as there is a replay
that returns non-zero value.

Best,
Andrew





On Sunday, 17 June 2018, 11:22:51 PM SGT, Sangharatna Godboley <sanghu1790 at gmail.com> wrote: 





Hi Andrew,

Thanks for your response. I am using libkleeRuntest library. I am following the instructions provided in the following link to replay the test cases:

https://klee.github.io/tutorials/testing-function/

I have tried implementing the loop using a programming language that iterates over all of the ktest files. However, the issue is that the speed is still slow on 100K test replays. I am trying to find a way to natively run the 100k klee test cases on the LLVM IR. To increase the speed of test replays I am thinking to avoid opening and closing of test input files 100k times. 

Is there any way to resolve this issue?

Thanks,
Sanghu


On Sat, Jun 16, 2018 at 10:30 AM, Andrew Santosa <asantosa1999 at gmail.com> wrote:
> Hi Sanghu,
> 
> You don't seem to be using klee-replay here. Otherwise, instead of invoking a.out
> you would have invoked klee-replay, with a.out as its argument.
> 
> Moreover, you can't just expect klee-replay to work on a single file containing all the
> ktest files by simple appending. I don't think this file will be in the right format for a
> ktest file. Instead, you should implement a loop using your programming language
> of choice that iterates over all of the ktest files and runs klee-replay on each.
> 
> Best,
> Andrew
> 
> On Thursday, 14 June 2018, 5:06:05 PM SGT, Sangharatna Godboley <sanghu1790 at gmail.com> wrote: 
> 
> 
> 
> 
> 
> Hi,
> I have a problem. For my experiment, KLEE generates larger test data for eg. > 100,000 "test000001.ktest" files. I am replaying these test input files using klee-replay tool with this command
> 
> "KTEST_FILE=klee-out-0/test000 001.ktest ./a.out".
> 
> Here is the problems, I need to open and close ".ktest" files for "a.out" 100,000 times, which is time consuming. I want to optimize the time. I want to avoid opening and closing of ".ktest" files for these many times. Is there any way to solve this problem? I am thinking to redirect all these test input files into one "sample.txt" file using the following command:
> 
> "ktest-tool klee-out-0/test******.ktest > sample.txt"
> 
> But, I am now stuck to process this sample.txt with ./a.out by using klee-replay tool.
> Can anyone help me for this problem?
> 
> Thanks
> Sanghu
> 
> ______________________________ _________________
> klee-dev mailing list
> klee-dev at imperial.ac.uk
> https://mailman.ic.ac.uk/ mailman/listinfo/klee-dev
> 



--