• CV
• education
• softography
• research
• experience
• hack blog

• personal info
• contact

Table of
contents

9.1.2010

Using manually tweaked PTX assembly in your CUDA 2 program

So you want to optimize or rewrite the PTX code CUDA 2.x compiler produced for you? Well, you should; not only is PTX virtual assembly fairly easy to write, but CUDA compiler technology is far from mature and there are surely manual optimizations to be made. Or you can just compile empty stub functions and fill in the actual PTX code yourself if you're feeling heroic.

In this entry I'll show you how to export PTX code from your CUDA program, and how to compile it back once you've edited it to your liking. Even though this is rather straight forward, I had to mess around for hours before I figured out how to do it. Or maybe I'm just a poor googler.

First, let's say your CUDA code is called mycode.cu and your program binary will be mycode. Create a devcode structure like this:

Now you have a structure that the CUDA runtime library can use when it runs the final program binary, except that instead of the actual device code object there is a symbolic link pointing to mycode.cubin in your current directory. Note that you don't have to do this again every time you compile a new version of your code. Next you can compile your existing C for CUDA code (or a framework) into an editable ptx file like this:

You are free to edit mycode.ptx now and work your magic in. When you're done, you should compile it as the device binary file mycode.cubin (validating the previously created link):

Now you can continue on building your program as usual. During linking, just include the object file mycode.o.ptx (and link against the CUDA runtime lib).

Comments

  15.2.2012

Hello Ville,
Can you elaborate the steps after generating the cubin file?. How to link the object file?
Thanks in advance

  28.3.2012

Hello Jay,
Sorry for not replying earlier; I don't yet have a notification system in the blog software so I didn't see this until today :-)
The file you should link in your binary is the mycode.o.ptx file, which is a normal object (.o) file.  The device binaries (the cubin file) are loaded at runtime by this object.
So in the simplest case, you might compile and link your program like this:  g++ -lcudart -o program main.cpp mycode.o.ptx

Please note, however, that CUDA 2 is obsolete.  Later CUDAs support a lot cleaner ways to include customized PTX in your kernels.  Also, this example is for the runtime API.  The driver API allows you to explicitly upload a modified .ptx file at runtime.

Best regards

  23.6.2016

It seems after modifying  the PTX file, it is not used further. Only the .cu file is used for generating cubin. So even if we link with mycode.o.ptx  which was generated before modifying PTX, how it can make a difference.

  23.6.2016

Hi Ginu!
It's been a while since I've used this approach, so my recollections are a bit vague.  But re-reading the post, the idea is that the program binary containing the mycode.o.ptx object will search for the "mycode.cubin" file at runtime from your filesystem, and the actual kernel code is not built into the program binary.  After you have modified the PTX and run the last command (mvcc ... --cubin mycode.cu) your "mycode.cubin" will have been replaced by the edited PTX which is what the program will load when you run it.
It might be that CUDA has changed in this regard and this approach doesn't work with current CUDAs--I'm not sure, I haven't tried. :-)
Hope this helps,