cjkkkk / cuda_gemm

A simple high performance CUDA GEMM implementation.

introduction

A simple high performance CUDA GEMM, Block Sparse GEMM and Non-uniform Quantized GEMM implementation.

C = alpha * A * B + beta * C

algorithm

located in src/cuda/

• MatrixMulCUDA
  • one element of C is assigned one thread
  • global memory coalesce of B
• MatrixMulCUDA1
  • texture load
• MatrixMulCUDA2
  • one 4 * 4 grid of C is assigned one thread
• MatrixMulCUDA3
  • vectorized A B load
• MatrixMulCUDA4
  • vectorized C store
• MatrixMulCUDA5
  • block sparse version
• MatrixMulCUDA6
  • vectorized A B load coalesce
• MatrixMulCUDA7
  • warp shuffle to enable C store coalesce
• MatrixMulCUDAQuantize8bit
  • 8 bit non-uniform quantized matmul

experiments

located in benchmark/

• benchmark_dense
  • Compare My Gemm with Cublas
• benchmark_sparse
  • Compare My block sparse Gemm with Cusparse
• benchmark_quantization_8bit
  • Compare My Gemm with Cublas
• benchmark_quantization
  • Compare My Gemm with My quantized non-uniform 8 bit Gemm

TODO

• (MatrixMulCUDA7) write back to C matrix, warp shuffle to enable global memory coalesce
• (MatrixMulCUDA8) double buffering

run

mkdir builds
make benchmark_[experiment name]
bash scripts/benchmark_[experiment name].sh

Note

• sparsity约为1%的时候, cusparse的性能可以超越cublas
• 合理分配寄存器 尽可能让参数在编译器确定节省计算资源和寄存器数目

cuda_gemm's Issues

Wrong boundary conditions

I believe you are using the boundary of C for matrices A and B

CUDA_gemm/src/cuda/dense.cu

Line 107 in 14b5173

As[i + A_TILE_ROW ][A_TILE_COL] = row < M && col < N ? A[OFFSET(

CUDA_gemm/src/cuda/dense.cu

Line 125 in 14b5173

Bs[i + B_TILE_ROW][B_TILE_COL] = row < M && col < N ? B[OFFSET(

Wrong results when M/N/K can not be divided by BM/BN/BK

Hi, I found that if I try $M = N = K = 100$, the result is wrong:

$ ./builds/benchmark_dense 100 100 100
Grid Dim: (2 2) Block Dim: (16 16)
My gemm Performance= 38.52 GFlop/s, Time= 0.052 msec, Size= 2000000 Ops,
CuBlas Performance= 179.52 GFlop/s, Time= 0.011 msec, Size= 2000000 Ops,
Error! Matrix[00000]=10154469497706623404430887145349054464.00000000, ref=1464674637695684277803204596465664.00000000 error term is > 1.000000E-06
Result= FAIL
ratio= 0.214553

But for larger $M = N = K = 1000$, the result is again correct

$./builds/benchmark_dense 1000 1000 1000
Grid Dim: (16 11) Block Dim: (16 16)
My gemm Performance= 10464.73 GFlop/s, Time= 0.191 msec, Size= 2000000000 Ops,
CuBlas Performance= 15660.82 GFlop/s, Time= 0.128 msec, Size= 2000000000 Ops,
Result= PASS
ratio= 0.668211

wrong results with random inputs

Hello,

thanks for your great work! When I set the input matrices to random value, like

`for( int i = 0; i < M * K; i++ ) {
h_A[i] = static_cast (rand()) / static_cast (RAND_MAX);
}

for( int i = 0; i < K * N; i++ ) {
  h_B[i] = static_cast <float> (rand()) / static_cast <float> (RAND_MAX);

}`

I got unpassed verification by using benchmark_dense.cu. Do you have any idea about this?

Best regards