[klee-dev] Question about ELOC counting in the KLEE paper

[Cristian Cadar]

Hi Edmund,

If I recall correctly, the global optimizations mentioned in the paper
were those done by LLVM -- since KLEE operates at the LLVM level, it
makes sense to consider those, and not the ones done by gcc.  The
version of LLVM used in the experiments was somewhere around LLVM 2.2
and 2.3, which I'm not sure how aggressive it is in terms of dead code
elimination.  Of course, since dead code elimination is a conservative
optimization, those numbers are an overestimate of the actual sizes.  My
co-authors might have more details; unfortunately, I don't have the
script that we used to generate that figure anymore.

Best,
Cristian

On 03/09/2014 22:05, Edmund Wong wrote:
> Hi KLEE authors,
> 
> We are doing research based on KLEE. We followed the tutorials to use
> gcov for measuring coverage. But we don't know how to measure the raw
> size of test subjects (including library code) in the way mentioned in
> the KLEE paper (Section 5.1):
> 
>     However, we do include library code when measuring the raw size of
>     the application: KLEE must successfully handle this library code
>     (and gets no credit for doing so) in order to exercise the code in
>     the tool itself. We measure size in terms of executable lines of
>     code (ELOC) by counting the total number of executable lines in the
>     final executable after global optimization, which eliminates
>     uncalled functions and other dead code. This measure is usually a
>     factor of three smaller than a simple line count (using wc -l).
> 
> 
> We have tried to do the following:
> 
> We first do a fresh build of Coreutils with gcov profiling. Then we
> invoke the target program, e.g., 'ls', for a couple of times. Gcov
> profiling information is logged into a .gcda file for each source code
> compiled into 'ls'. We then invoke gcov on all those .gcda files (in
> lib/ and src/) and sum up the ELOC of their corresponding source files.
> However, this result doesn't agree with Fig. 4 in the KLEE paper
> (Section 5.2):
> 
>     Figure 4 breaks down the tools by executable lines of code (ELOC),
>     including library code the tool calls. While relatively small, the
>     tools are not toys — the smallest five have between 2K and 3K ELOC,
>     over half (52) have between 3K and 4K, and ten have over 6K.
> 
> 
> E.g., for 'uptime', we get its ELOC as ~1200, which is far less than 2K.
> 
> Is anything wrong with the above steps? Could you tell us the
> method/tool used to count ELOC "in the final executable after global
> optimization"? Thanks a lot!
> 
> Edmund Wong
> 
> 
> _______________________________________________
> klee-dev mailing list
> klee-dev at imperial.ac.uk
> https://mailman.ic.ac.uk/mailman/listinfo/klee-dev
>