Alp Mestanogullari's Blog

Getting GHC HEAD and LLVM working together

Posted in Uncategorized by alpmestan on 2010/03/11

Since dons’ two posts, I have seen many people on the haskell-cafe mailing-list and the haskell-related IRC channels asking how they could get GHC HEAD and the LLVM-related stuff, build them and all. There are a lot of this information on the ghc trac but I have been told it would be nice to have a comprehensive guide with all the necessary steps and information.

Getting, patching and building LLVM

It is as simple as executing the following commands:

$ svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
$ cd llvm
$ patch -p0 -i ~/llvm-ghc.patch
$ ./configure –enable-optimized # probably also want to set –prefix
$ make
$ make install

where the given llvm-ghc.patch file can be downloaded here : http://www.cse.unsw.edu.au/~davidt/downloads/llvm-ghc.patch.

Getting GHC HEAD and Patching it to add the LLVM backend

First, get the latest archive matching that pattern : ghc-HEAD-AAAA-MM-JJ-ghc-corelibs-testsuite.tar.{gz or bz2} at http://darcs.haskell.org/, the latest currently being ghc-HEAD-2009-10-23-ghc-corelibs-testsuite.tar.gz. It ships GHC with all its dependencies, also including a testsuite.

Put the archive file in (on my computer) /home/alp/haskell/. Uncompress it with for example:

$ tar xzf ghc-HEAD-2009-10-23-ghc-corelibs-testsuite.tar.gz

You should get a /home/alp/haskell/ghc/ directory. Now we have to fetch the latest patches applied to GHC since the archive you downloaded got published on darcs.haskell.org. To do this, simply do:

$ chmod +x ./darcs-all
$ ./darcs-all pull -a

(the former may not be necessary though)
It can take some time, there may be a lot of patches to fetch and apply, don’t worry.

If you are not interested in having the LLVM backend, you can stop reading that section and go straight to the next one.

Once done with that, you have to download the patch to add the support to the LLVM backend to GHC. Just download the following file : http://www.cse.unséw.edu.au/~davidt/downloads/ghc-llvmbackend-full.gz (it actually just is a darcs patch, not an archive or anything else — you may want to rename it with a .patch extension), put it in ghc’s directory (/home/alp/haskell/ghc/ here) and apply the patch :

$ darcs apply ghc-llvmbackend-full.patch

(or .gz)

You just have one more patch to apply (isn’t life about applying patches?), written by Don Stewart, to be able to pass LLVM options to GHC without conflicting with other options. You can get it at http://www.galois.com/~dons/add-new-llvm-code-generator-to-ghc_.dpatch. Just move it to your ghc directory and darcs apply it, like you just did for the previous patch. (note: this step might not be necessary if the previous patch or the ghc repository contain it)

Ok, we are almost done! Now create a file named “build.mk” in (on my computer) /home/alp/haskell/ghc/mk/, and put the following content in that file:

GhcWithLlvmCodeGen = YES
GhcEnableTablesNextToCode = NO

The first, obviously, enables LLVM code generation in GHC. The second disables a feature currently being in conflict between the LLVM and the C code generators, if I remember correctly. Now, let’s build GHC, hooray!

Building and using the patched GHC non-intrusively

Once all the patches are fetched and applied, you just have to do:

$ sh boot
$ ./configure
$ make

Now, have a cup of coffee, read newspapers, try to solve P vs NP, whatever. It takes some time.

Once done with ‘make’, there should be many binaries in (on my computer) /home/alp/haskell/ghc/inplace/bin/. The ghc binary is called ghc-stage2

To check that the LLVM backend is enabled, as explained on Don’s post, just do the following (in the your ghc/inplace/bin/ directory):

$ ghc-stage2 –info

and verify that it gives you (“Have llvm code generator”,”YES”).

To make use of them properly without installing your brand new GHC HEAD system-wide, here is one possibility; add the following to your ~/.bashrc (or whatever you use, it’s just about defining variables):

GHC613 = /home/alp/haskell/ghc/inplace/bin
GHC613BIN=$GHC613/ghc-stage2
GHC613PKG=$GHC613/ghc-pkg

And now, when you will want to install libraries via cabal for your new GHC, just execute commands close to the following (to install say the vector package):

$ cabal install vector –with-compiler=$GHC613BIN –with-hc-pkg=$GHC613PKG

And it will build and install the vector package using GHC HEAD and its library directories (in ~/.ghc/).
Moreover, if you want to build packages with the llvm backend, just ask cabal gently:

$ cabal install vector –with-compiler=$GHC613BIN –with-hc-pkg=$GHC613PKG –ghc-options=-fllvm

You can even chain options this way, passing for example -optlo-03. For more informations on that, check Don’s posts:

But be careful, -fvia-C and -fllvm aren’t much friends, you may sometimes have to either edit the .cabal file of the package you want to install and removing -fvia-C from there, or via the –flags option of cabal install. For more informations, cabal help install.

Here are few links that you may find to be of interest:

I hope this post will be of help and that it will lead some of you to contribute to GHC !

About these ads
Tagged with: , ,

18 Responses

Subscribe to comments with RSS.

  1. Jake McArthur said, on 2010/03/11 at 2:01 am

    You forgot to include the `sh boot` step.

  2. alpmestan said, on 2010/03/11 at 2:15 am

    Oh yeah. Fixed. Thanks Jake !

  3. Jake McArthur said, on 2010/03/11 at 5:13 am

    I just created a script in my ~/bin directory called cabal-llvm. Now I can just use cabal-llvm instead of cabal when I want to use GHC 6.13 with the LLVM backend!

    #!/bin/bash

    GHC613=~/src/ghc/inplace/bin
    GHC613BIN=$GHC613/ghc-stage2
    GHC613PKG=$GHC613/ghc-pkg

    echo “cabal $@ –with-compiler=$GHC613BIN –with-hc-pkg=$GHC613PKG –ghc-options=-fllvm”
    cabal $@ –with-compiler=$GHC613BIN –with-hc-pkg=$GHC613PKG –ghc-options=-fllvm

  4. alpmestan said, on 2010/03/11 at 12:35 pm

    Nice script indeed. The only missing thing would be to handle as automatically as possible the case of libraries containing an explicit -fvia-C flag, for example, like it’s been the case for uvector for me.

    By the way, I had a *very* significant performance improvement with the LLVM backend, on the sample given with HNN. I took around 270-280 ms for the native and C code generator (6.12.1), and only around 90 ms with the LLVM backend (with GHC HEAD) ! The first version of HNN is uvector based.

    EDIT : few hours after I published this post, I have seen the llvm patch (making LLVM understand GHC’s calling conventions for better performances) has been pushed upstream ! [see here]

  5. walrus said, on 2010/03/12 at 2:02 am

    Just a problem: what happened to the ghc-llvmbackend-full.patch link? Appears to be down in the moment.

    • walrus said, on 2010/03/12 at 2:16 am

      Forget about that. The problem was with Google Chrome, it tried to open like a gz file and barked. :P

  6. alpmestan said, on 2010/03/12 at 2:16 am

    The link works. Some browsers (chrome e.g) seem to be confused by the actual format of the file. The file is a plain darcs patch but the .gz extension may confuse the browser. Just download it (via right click > save as or simply wget) and call darcs apply on it while in ghc’s main directory.

  7. saynte said, on 2010/03/12 at 2:14 pm

    I thought this looked interesting and tried out the steps. I updated GHC-head to the most recent revision, but then it doesn’t compile (I think something Simon Marlow is working on). I reverted a few of his most recent patches patches and now GHC compiles.

    However, in this shape the binaries produced by the LLVM backend just seg-fault. Is there a particular `good’ revision of GHC that these patches can be applied to and have usable generated binaries?

  8. alpmestan said, on 2010/03/12 at 2:21 pm

    Well, my version has been built based on patches until Feb 28th (ghc version 6.13.20100228). If you handle to revert until that date’s patches it should be fine.

  9. saynte said, on 2010/03/13 at 5:49 am

    Ah, I realized my problem: llvm now has the correct changes in the trunk, so I skipped the patching step for llvm. What I had to do to make it work was patch one of the GHC generated files to use the new calling convention that was just added to llvm.

  10. alpmestan said, on 2010/03/13 at 7:37 am

    Can you detail your solution a bit ? People in the community are having the same issues apparently. Thanks !

  11. saynte said, on 2010/03/13 at 10:18 am

    I think the following change does it:

    hunk ./compiler/llvmGen/Llvm/Types.hs 527
    – show CC_Fastcc = “fastcc”
    + show CC_Fastcc = show (CC_Ncc 10)

    I believe David had patched the “fastcc” calling convention directly in LLVM. However, now his patched fastcc is a separate calling convention now in LLVM-svn, `cc 10”. The above diff simply replaces how he prints his CC_Fastcc with the new calling convention (cc 10).

  12. KrzysztofSkrzetnicki said, on 2010/03/22 at 7:45 pm

    I was following your instructions. However, darcs-all seems to hang:

    ./darcs-all get
    warning: adding –partial, to override use –complete
    warning: . already present; omitting
    == running darcs get –partial http://darcs.haskell.org/ghc-tarballs ghc-tarballs
    Converting old-fashioned repository to hashed format…
    *******************************************************************************
    Fetching a hashed repository would be faster. Perhaps you could persuade
    the maintainer to run darcs optimize –upgrade with darcs 2.4.0 or higher?
    *******************************************************************************
    No checkpoint.
    Writing inventory 1 done, 0 queued.

    After this, no more output is generated – darcs-all seems to hang on instruction:
    > darcs get –partial http://darcs.haskell.org/ghc-tarballs/ ghc-tarballs

    Any ideas?

  13. alpmestan said, on 2010/03/22 at 7:52 pm

    What if you do
    $ ./darcs-all pull -a
    as said in my post ?

  14. KrzysztofSkrzetnicki said, on 2010/03/23 at 2:55 pm

    Ok, my bad: I downloaded different tarball previously and I thought that the one mentioned in the post is the same. After downloading GHC, I unpacked it, patched (with both patches to GHC) and edited Types.hs:

    instance Show LlvmCallConvention where
    show CC_Ccc = “ccc”
    — show CC_Fastcc = “cc 10″
    — show CC_Fastcc = “fastcc”
    show CC_Fastcc = show (CC_Ncc 10)
    show CC_Coldcc = “coldcc”
    show (CC_Ncc i) = “cc ” ++ (show i)
    show CC_X86_Stdcc = “x86_stdcallcc”

    However, even simplest program build with -fllvm segfaults:

    █▓▒░tener@laptener░▒▓██▓▒░ wto mar 23 03:46:11
    ~/dokumenty/programowanie-wlasne/haskell/sorting/ ghc –make hello -fforce-recomp && ./hello
    [1 of 1] Compiling Main ( hello.hs, hello.o )
    Linking hello …
    hi

    █▓▒░tener@laptener░▒▓██▓▒░ wto mar 23 03:45:58
    ~/dokumenty/programowanie-wlasne/haskell/sorting/ $GHC613BIN –make hello -fforce-recomp && ./hello
    [1 of 1] Compiling Main ( hello.hs, hello.o )
    Linking hello …
    hi

    Output of “–info”:

    $GHC613BIN –info
    [("Project name","The Glorious Glasgow Haskell Compilation System")
    ,("Project version","6.13.20100321")
    ,("Booter version","6.12.1")
    ,("Stage","2")
    ,("Have interpreter","YES")
    ,("Object splitting","YES")
    ,("Have native code generator","YES")
    ,("Have llvm code generator","YES")
    ,("Support SMP","YES")
    ,("Unregisterised","NO")
    ,("Tables next to code","NO")
    ,("Win32 DLLs","")
    ,("RTS ways","l debug thr thr_debug thr_l thr_p dyn debug_dyn thr_dyn thr_debug_dyn")
    ,("Leading underscore","NO")
    ,("Debug on","False")
    ,("LibDir","/home/tener/dokumenty/programowanie-wlasne/haskell/llvm-ghc-backend/ghc/inplace/lib")
    ]

    █▓▒░tener@laptener░▒▓██▓▒░ wto mar 23 03:46:07
    ~/dokumenty/programowanie-wlasne/haskell/sorting/ $GHC613BIN –make hello -fllvm -fforce-recomp && ./hello
    [1 of 1] Compiling Main ( hello.hs, hello.o )
    Linking hello …
    zsh: segmentation fault ./hello

    I tried gdb but didn’t learned much:

    $ gdb hello
    GNU gdb (GDB) 7.0.1
    Copyright (C) 2009 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law. Type “show copying”
    and “show warranty” for details.
    This GDB was configured as “i686-pc-linux-gnu”.
    For bug reporting instructions, please see:

    Reading symbols from /home/tener/dokumenty/programowanie-wlasne/haskell/sorting/hello…done.
    (gdb) r
    Starting program: /home/tener/dokumenty/programowanie-wlasne/haskell/sorting/hello
    [Thread debugging using libthread_db enabled]

    Program received signal SIGSEGV, Segmentation fault.
    0x0804a89b in __stginit_ZCMain ()
    (gdb)

    Any clues what’s wrong?

  15. alpmestan said, on 2010/03/23 at 2:59 pm

    Did you patch llvm too ? Or checked out a version of LLVM already containing the patch ?

  16. KrzysztofSkrzetnicki said, on 2010/03/23 at 3:22 pm

    I used most recent version of llvm from svn, so I didn’t patch it manually. Manual inspection of sources indicate it includes GHC patch and support for “cc 10″ call convention.

  17. David Terei said, on 2010/03/24 at 3:16 am

    I’ve released a new version of the back-end and update the build instructions on the wiki. The new patch works now with the newer versions of LLVM that include the changes needed by GHC.

    http://hackage.haskell.org/trac/ghc/wiki/Commentary/Compiler/Backends/LLVM/Installing


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 251 other followers

%d bloggers like this: