[klee-dev] using klee without llvm-gcc?
Hongxu Chen
leftcopy.chx at gmail.com
Sun Mar 24 08:17:57 GMT 2013
Thanks so much for all your help, David and Daniel.
Now I figure out where I was wrong during the build of uclibc. I added a
redundant option when I use make,
make --with-llvm=<path> CFLAGS+=-O0 CFALGS+=-std=std99
which, however, lead to the error.
Now I can build uclibc smoothly with llvm-gcc. However I still CANNOT
build it
with clang since there is a header called `limits.h' that
`#include_next<limits.h>', but clang failed to find the next header file
called
limits.h. So maybe I didn't set $C_INCLUDE_PATH right.
Thanks,
Hongxu Chen
On 03/23/2013 03:55 AM, David Lightstone wrote:
> I decided to play a bit with the Ubuntu 12.04 64bit environment
>
> There is a lot of logistics which need to be established before you have
> a valid development environment.
>
> At this point I am not certain whether all the following were necessary,
> but I installed them
>
> sudo apt-get install g++
>
> sudo apt-get install gcc
>
> sudo apt-get install libgmp-dev
>
> sudo apt-get install libmpfr-dev
>
> sudo apt-get install build-essential
>
> sudo apt-get install ia32-libs
>
> sudo apt-get install g++-multilib
>
> there may have been some other things that I did not record as having
> been installed
>
> In building LLVM 2.9 (llvm-gcc, llvm and clang)
>
> to find crt1.o I had to add to all build scripts.
>
> LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
>
> export LIBRARY_PATH
>
> Precisely why I do not yet know, probably ld does not include it.
>
> Building stp required a change to
>
> Scripts/makefile.common
>
> #dbl delete -m32 to compile on 64 bit machines
>
> #CFLAGS_M32 = -m32
>
> CFLAGS_M32 =
>
> uclibc was straight forwared
>
> klee was straight forward, not certain of the revision number that I used
>
> regression tests did not all pass
>
> internal stp (no unexpected failures)
>
> external stp (no unexpected failures)
>
> uclibc with internal stp (2 regression failures)
>
> uclibc with external stp (2 regression failures)
>
> Monday I will see how it builds against the latest KLEE
>
> Dave Lightstone
>
> *From:*David Lightstone [mailto:david.lightstone at prodigy.net]
> *Sent:* Tuesday, March 19, 2013 5:12 PM
> *To:* 'Hongxu Chen'
> *Cc:* Daniel Liew (daniel.liew at imperial.ac.uk)
> *Subject:* RE: [klee-dev] using klee without llvm-gcc?
>
> Reply within body, in red
>
> Dave Lightstone
>
> *From:*Hongxu Chen [mailto:leftcopy.chx at gmail.com]
> *Sent:* Tuesday, March 19, 2013 2:16 PM
> *To:* david.lightstone at prodigy.net <mailto:david.lightstone at prodigy.net>
> *Cc:* 'Daniel Liew'
> *Subject:* Re: [klee-dev] using klee without llvm-gcc?
>
> On 03/19/2013 09:45 PM, David Lightstone wrote:
>
> (1)When you say successfully, what is the evaluation criteria?
> Having (a) the tool build, and (b) successfully execute the tool’s
> testing suite are 2 very much different things
> The only meaningful criteria is end to end check against the KLEE
> unittests
>
> Indeed I didn't check the unit tests; thanks for your advice and I'll
> try. Next time whenever I build some code I would check the tests if
> possible.
>
> (2)With some, but not all, software tools, building in the wrong
> directory, can cause corruption of the distribution
> With uClibc I don’t think this is a problem. It may be a problem for
> KLEE, it certainly is a problem for LLVM or GCC. Once corruption occurs,
> make clean is of little help
>
> Well, I don't believe the failure of uclibc is the fault of KLEE,LLVM or
> anything else. I admit make clean is sometimes not enough, but I suppose
> that most of the case it works(especially with gnu autotools). I haven't
> came across the cases you mention since I didn't build much source code
> before.
>
> (3)I normally use scripts to build tools. That way I can regression test
> the build process or reproduce the build process on another machine (it
> may prove to be a debug effort on that new machine, but the effort is
> systematic and uncorrupted by a human being having forgotten to do
> something). Baseline the build on another clean virtual machine, then
> port the build scripts to whatever you wish to use as a development host
> (I would back up the virtual machine and use it as a clean development
> environment that can be restored when an experiment fails)
> One of the things that I normally do in the scripts is lock down the
> paths and if possible explicitly identify the various tools that will be
> used. Externally defining CC and CXX. I don’t particularly like hacking
> files. It is usually not necessarily. If you must do so. I have found
> that creating a parallel “patch” directory tree to be useful. Save a
> copy of the modified file in the appropriate place in the “patch”
> directory tree can be quite useful. This is especially so when you need
> to restart. The alternative is creating a local branch in a local
> configuration management system.
>
> The "patch" is indeed a good idea, thanks. I didn't realize I could do
> that way:-) Also I would try to avoid modify configuration files.
>
> The intent is destroy old directory, untar distribution to a replacement
> directory, copy patch directory into replacement directory
>
> (you probably can also use the patch directory to create patch files for
> purposes of change distribution, but have never needed to do this, so I
> am not entirely certain)
>
> By using a local CM, you check in to the branch when you are happy with
> a change, and when something goes wrong you just check out from the tip
> of the branch to get a known version
>
> (4)Copying configuration files from one machine to another is not
> necessarily a safe practice.
> Explanation is via an example - the installation of LLVM is based upon
> an automake / autoconfig host characterization process (performed when
> configure is invoked). The 2 hosts may have different characterizations.
> The KLEE build process is dependent upon the state of the host when the
> automake/autoconfig host characterization occurred. If perchance you
> build LLVM (without say dejagnu installed) then build KLEE and attempt
> to run the unittests. The unittests will not be performed because
> dejagnu is not present. If you subsequently install dejagnu and attempt
> to run the KLEE unittest, they will not be performed. The LLVM
> configuration information upon which KLEE depends does not indicate the
> presence of dejagnu. LLVM will also need to be reconfigured
>
> It is my mistake that I used an configuration file that was auto
> generated by autotools. I did so just to check whether the failure
> really results from llvm build process.
>
> I do not know what you mean here.
>
> I always run the tool’s configure script to create the host dependent
> makefile and other configuration files
>
> Which Linux distribution are you using? what version?
>
> I am using Ubuntu 12.04 LTS, 64bit, and gcc is 4.6.3(Ubuntu/Linaro
> 4.6.3-1ubuntu5).
>
> Again, thanks for your illuminating words.
>
> Hongxu Chen
>
> Dave Lightstone
>
> *From:*Hongxu Chen [mailto:leftcopy.chx at gmail.com]
> *Sent:* Tuesday, March 19, 2013 8:46 AM
> *To:* Daniel Liew; david.lightstone at prodigy.net
> <mailto:david.lightstone at prodigy.net>
> *Cc:* klee-dev
> *Subject:* Re: [klee-dev] using klee without llvm-gcc?
>
> Hi, Daniel and David, it's very kind of you to share so many details,
> thanks very much.
>
> Firstly I would say that I made a mistake, STP was still built with
> gcc/g++; sorry about that.
>
> The source package of uclibc hasn't been corrupted since it was newly
> downloaded, also I always
> `make clean' before a new build.
>
> I actually built uclibc successfully in one of my virtual machines
> several days ago, and
> today I took the $(LLVMROOTDIR)/Makefile.config, but still the same
> error message.
> Also when uclibc is configured with `--with-gcc' option, the output in
> the old virtual machine
> is different from the one I am currently on(although the gcc version is
> the same).
> So I suppose that there may be some errors with my system environments,
> but I haven't
> figured out it.
>
> At present, I just use the pre-compiled uclibc; although klee always
> complains that data
> layout is different, it at least works.
>
> Thanks for your help!
>
> Regards,
> Hongxu Chen
>
> On Tue, Mar 19, 2013 at 7:48 PM, Daniel Liew <daniel.liew at imperial.ac.uk
> <mailto:daniel.liew at imperial.ac.uk>> wrote:
>
> On 18 March 2013 15:43, Hongxu Chen <leftcopy.chx at gmail.com
> <mailto:leftcopy.chx at gmail.com>> wrote:
> > Thanks for your reply, Dan.
> > I followed your advice using clang2.9 to build bca files and build
> llvm,stp
> > successfully; but UCLIBC fails.
>
> Just to say there isn't a requirement to build STP with a LLVM
> compiler but the fact that it did compile is good news :)
>
>
> klee has partially been built and libs like
> > libkleeRuntimeIntrinsic.bca,libkleeRuntimePOSIX.bca and
> libklee-libc.bca has
> > already came out. For uclibc, the error message is a bit confusing:
> >
> > In file included from libcrypt/crypt.c:11:
> > libcrypt/libcrypt.h:11:78: error: expected function body after
> > function declarator
> > extern char *__md5_crypt(const unsigned char *pw, const unsigned
> > char *salt) attribute_hidden;
> >
> > ^
> > libcrypt/libcrypt.h:12:78: error: expected function body after
> > function declarator
> > extern char *__des_crypt(const unsigned char *pw, const unsigned
> > char *salt) attribute_hidden;
> >
> > ^
> > libcrypt/crypt.c:18:10: warning: implicit declaration of function
> > '__md5_crypt' is invalid in C99 [-Wimplicit-function-declaration]
> > return __md5_crypt((unsigned char*)key, (unsigned
> > char*)salt);
> > ^
> > libcrypt/crypt.c:18:10: warning: incompatible integer to pointer
> > conversion returning 'int' from a function with result type 'char *'
> > return __md5_crypt((unsigned char*)key, (unsigned
> > char*)salt);
> >
> > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > libcrypt/crypt.c:20:10: warning: implicit declaration of function
> > '__des_crypt' is invalid in C99 [-Wimplicit-function-declaration]
> > return __des_crypt((unsigned char*)key, (unsigned
> > char*)salt);
> > ^
> > libcrypt/crypt.c:20:10: warning: incompatible integer to pointer
> > conversion returning 'int' from a function with result type 'char *'
> > return __des_crypt((unsigned char*)key, (unsigned
> > char*)salt);
> >
> > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > 4 warnings and 2 errors generated.
> > make: *** [libcrypt/crypt.os] Error 1
> >
> > Here in Rules.mak.llvm, `LLVMGCC' has been changed to
> $(LLVMTOOLDIR)/clang.
> > Similar error occurred when I set LLVMGCC to a pre-compiled
llvm-gcc(from
> > llvm2.9 download
> >
page,http://llvm.org/releases/2.9/llvm-gcc4.2-2.9-x86_64-linux.tar.bz2).
> >
> > make: *** [libcrypt/crypt.os] Error 1
> > In file included from libcrypt/crypt.c:11:
> > libcrypt/libcrypt.h: In function '__md5_crypt':
> > libcrypt/libcrypt.h:11: error: expected declaration specifiers
> > before 'attribute_hidden'
> > libcrypt/libcrypt.h:12: error: expected '=', ',', ';', 'asm' or
> > '__attribute__' before 'attribute_hidden'
> > libcrypt/crypt.c:14: error: expected '=', ',', ';', 'asm' or
> > '__attribute__' before '{' token
> > libcrypt/crypt.c:21: error: expected '{' at end of input
> > make: *** [libcrypt/crypt.os] Error 1
> >
> > I also tried the `--with-gcc' option when configuring(AFAIK, CC is
> set to gcc),
> > and the error message is exactly the same as llvm-gcc case.
> >
> > 1. So has anyone else ever built
> > klee-uclibc-0.02-x64/klee-uclibc-0.02-i386 with
> > native gcc and met similar problems?
> > 2. The `Rules.mak.llvm' actually reads
> > `$(LLVMROOTDIR)//Makefile.config`, so maybe
> > I again made some mistakes when building llvm?
>
> Something very odd is happening there you shouldn't be getting parse
> errors. I have never seen that problem. I can only really suggest
> three things things
> - It's very unlikely but check your source code for uclibc hasn't been
> corrupted
> - run `make VERBOSE=1` and check the command being used to build
> crypt.os is correct.
> - I'm not really 100% sure why $(LLVMROOTDIR)/Makefile.config is being
> included but I managed to build without it being included in the
> makefile. You could try removing it then doing `make clean && make`
>
> For your reference this is the command that is executed on my system
> (I've shortened the llvm-gcc path) to build crypt.os
>
> llvm-gcc --emit-llvm -c ../libcrypt/crypt.c -o ../libcrypt/crypt.os
> -include ../include/libc-symbols.h -Wall -Wstrict-prototypes
> -fno-strict-aliasing -fno-stack-protector -fno-builtin -nostdinc
> -I../include -I. -DSTATIC -std=gnu99 -O0 -g3 -I/usr/include/ -isystem
>
/data/dev/KLEE/llvm-gcc/bin/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.2.1/include
> -DNDEBUG -fPIC -DNOT_IN_libc -DIS_IN_libcrypt
>
> Out of curiosity I did have a quick try at compiling
> klee-uclibc-0.02-x64 using clang (version 3.1 because that's what is
> on my system and llvm tools 3.1). I really do not advise it as I had
> to blindly change some of the code to make it compile and I'm not sure
> what the consequences are.
>
> I hacked the Rules.mak file with..
>
> LLVMGCC := clang
> CC = $(LLVMGCC) -emit-llvm
> AR = llvm-ar-3.0
> LD = llvm-ld-3.0
> NM = llvm-nm-3.0
>
> The configure script included with klee-ucblic-0.02 is just a hack so
> I didn't bother running it. I had to hack libc/misc/sysvipc/ipc.h
> header so that the defines were set (removed #ifdef __NR_ipc and
> associated #endif) and then I could run `make`.
>
>
> Regards,
> Dan Liew.
>
>
> >
> > Thanks,
> > Hongxu Chen
> >
> >
> > On Sun, Mar 17, 2013 at 10:18 PM, Daniel Liew
> > <daniel.liew at imperial.ac.uk <mailto:daniel.liew at imperial.ac.uk>>
wrote:
> >>
>
> >> On 17 March 2013 03:20, Hongxu Chen <leftcopy.chx at gmail.com
> <mailto:leftcopy.chx at gmail.com>> wrote:
> >> > Dear all,
> >> >
> >> > It seems that
> >> >
>
llvm-gcc-4.2-2.9(http://llvm.org/releases/2.9/llvm-gcc-4.2-2.9.source.tgz)
> >> > suggested by the getting started page of klee has many problems
and is
> >> > treated as obsolete by the llvm community. For me, an internal
error
> >> > occurred when I built it with an debug version of llvm-2.9. So I
> have some
> >> > questions about klee:
> >>
> >> It is indeed the case that llvm-gcc is obsolete. I found this when I
> >> first started working on KLEE but I ignored the issue as I needed to
> >> work with something that was stable.
> >>
> >> > Is it possible to build llvm without llvm-gcc when I have to use
> klee and
> >> > uclibc? This page(http://klee.llvm.org/GetStarted.html) says that
> llvm-gcc
> >> > has to be added into $PATH so that llvm would choose llvm-gcc
> other than gcc
> >> > or clang to build llvm. The key difference is that llvm-gcc
generates
> >> > libkleeRuntimeIntrinsic.bca(and the related posix,uclibc
runtime .bca
> >> > library; this thread tells the details ). I took a glance at the
> configure
> >> > file of llvm but didn't figure out how it is done. So I am
> wondering whether
> >> > clang-compiled llvm also has libkleeRuntimeIntrinsic.bca and the
> like. I
> >> > came across another project called klee-fp, which uses clang and
> llvm-3.0;
> >> > additionally they also use uclibc, But there is a compilation
> error when I
> >> > built klee-fp(actually the error is in the compilation of stp,
> which is
> >> > inside klee-fp source code folder).
> >> > If those .bca files can only be generated by llvm-gcc, then is
> there any
> >> > trick when I use llvm-gcc
> >> >
> binary(http://llvm.org/releases/2.9/llvm-gcc4.2-2.9-x86_64-linux.tar.bz2)
> >> > (instead of building llvm-gcc from souce) to compile llvm? There
> are still
> >> > some compilation errors for me.
> >>
> >> I tried compiling llvm-gcc from source about a year ago I
couldn't get
> >> anywhere with it so I just used the llvm-gcc binaries and KLEE will
> >> build fine. There is nothing really stopping you as far as I know
from
> >> using clang instead of llvm-gcc to build libkleeRuntimeIntrinsic.bca
> >> and other KLEE bitcode libraries. You should use a version of clang
> >> though that uses the same version of LLVM that you intend to use with
> >> KLEE. KLEE relies on LLVM's build system so for things to work in
KLEE
> >> so a LLVM compiler (either llvm-gcc or clang) needs to be detected
> >> (i.e. in PATH) when you configure llvm for building.
> >>
> >> This will probably cause you a little bit of hassle if you build llvm
> >> and clang from scratch as you'll probably need to build llvm and
clang
> >> using gcc and then build llvm again using your newly built clang
> >> compiler.
> >>
> >> If you use your distribution's version of clang and build llvm from
> >> source so that it matches your clang version you will make life
easier
> >> for yourself.
> >>
> >> Having said that, I don't believe KLEE supports LLVM 3.1 properly
(the
> >> IR has changed since llvm 2.9) so expect some issues to appear if you
> >> use clang. I'm also not sure if KLEE's testframe work will behave if
> >> you use clang instead.
> >>
> >> KLEE being stuck using llvm-gcc and llvm 2.9 is definitely an issue
> >> which we need to address at some point. It's something that
personally
> >> I'd like to address as well.
> >>
> >> Regards,
> >> Dan Liew.
>
More information about the klee-dev
mailing list