Not enough memory in GPU?!

As what was planed at the very beginning of the summer, I should have learned quite a lot about CUDA C programing. However-there is always a however-I am moving on at a super slow speed. Anyway, there is some update today.

I was following one of the examples in the CUDA C text book. Actually, I didn’t even write my own program but used the example code provided. The nvcc​ did a pretty good job in compiling the program. BUUUT, the GPU part of the program crashed and gave an error message “out of memory”. I was kinda surprised, because honestly I have not seen any error about the memory since I started learning CUDA. The 2GB memory in my GeForce GTX 750Ti gives me the feeling that I can never fully take advantage of them. Well, now, I guess this little example program just destroyed my illusion. The error specifically pointed out that the problem came out from cudaMalloc(). By the way, the example program is a 2D Matrix Addition program.

Screenshot from 2017-08-17 12:43:04

Alright, time to debug. Deep sigh. The CPU part was alright. That means the data initialization was good. Suddenly noticed that the program used the bitwise left shift operator to assign the matrix dimension in both x and y: nx=1<<14; ny=1<<14, which generates a matrix in dimension of 16384*16384. With that each element inside is a float, the size of the matrix is 16384*16384*4Bytes=1024MB. BOOM!! So this is only 1GB, how can my 2GB 750Ti not work??? Only 5 seconds later, I realized how stupid I was: the GPU kernel needs three pointers for the calculation, Matrix A, Matrix B, and resultant Matrix C. That means I need to allocate memory three times for three matrix on the GPU. And in total, that is 1024MB+1024MB+1024MB=3GB. The error was right, out of memory.

Well, the solution is easy enough. Simply changing the size of the matrix, the program works just fine.

Now, the genius Maxwell wants to leverage the full power of the GPU, i.e using all of the memory for the computation. The nvidia-smi told me that the X window is using almost half of the GPU memory. Ehhh, that’s not cool. Need to shut down that. A linux without GUI, i.e. command line, is dope. I hope I can do a screen shot, but I don’t know how to do it when GUI is off… sudo service lightdm stop killed all video processes. It’s interesting to see that only 7MB is used in the GPU. After some calculation and test, the GPU runs up to about 13200*13200 matrix addition. Yet, this number is not the number that coming out from using all of the 2GB memory. I guess that even just with the command line on the screen, the GPU is using quite a bit of memory.

Well, probably next time I should just use the CPU on-chip video card for the display so that I can free the GPU for computation only. Even though GeForce GTX 750Ti is actually a gaming video card, not a computational video card…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s