microsoft / archprobe

A profiler to disclose and quantify hardware features on GPUs.

ArchProbe

ArchProbe is a profiling tool to demythify and quantify mobile GPU architectures with great details. The mechanism of ArchProbe is introduced in our MobiCom'22 paper "Romou: Rapidly Generate High-Performance Tensor Kernels for Mobile GPUs". We appreciate you cite the paper for using this tool.

@inproceedings{liang2022romou,  
  author = {Liang, Rendong and Cao, Ting and Wen, Jicheng and Wang, Manni and Wang, Yang and Zou, Jianhua and Liu, Yunxin},  
  title = {Romou: Rapidly Generate High-Performance Tensor Kernels for Mobile GPUs},  
  booktitle = {The 28th Annual International Conference On Mobile Computing And Networking (MobiCom 2022)},  
  year = {2022},  
  month = {February},  
  publisher = {ACM},  
  doi = {10.1145/3495243.3517020},  
}

Examples of Architecture Overview prodiled by ArchProbe for Adreno 640 and Mali G76
Architecture details collected with ArchProbe, presented in our technical paper.

How to Use

In a clone of ArchProbe code repository, the following commands build ArchProbe for most mobile devices with a 64-bit ARMv8 architecture.

git submodule update --init --recursive
mkdir build-android-aarch64 && cd build-android-aarch64
cmake -DCMAKE_TOOLCHAIN_FILE="$ANDROID_NDK/build/cmake/android.toolchain.cmake" -DANDROID_ABI="arm64-v8a" -DANDROID_PLATFORM=android-28 -G "Ninja" ..
cmake --build . -t ArchProbe

To run ArchProbe in command line via adb shell, you need to copy the executables to /data/local/tmp.

If you are using Windows, the PowerShell scripts in scripts can be convenient too:

scripts/Run-Android.ps1 [-Verbose]

Prebuilt Binaries

Prebuilt binaries will be available here.

How to Interpret Outputs

A GPU hardware has many traits like GFLOPS and cache size. ArchProbe implements a bag of tricks to expose these traits and each implementation is called an aspect. Each aspect has its own configurations in ArchProbe.json, reports in ArchProbeReport.json, and data table of every run of probing kernels in [ASPECT_NAME].csv. Currently ArchProbe implements the following aspects:

• WarpSizeMethod{A|B} Two methods to detect the warp size of a GPU core;
• GFLOPS Peak computational throughput of the device;
• RegCount Number of registers available to a thread and whether the register file is shared among warps;
• BufferVecWidth Optimal vector width to read the most data in a single memory access;
• {Image|Buffer}CachelineSize Top level cacheline size of image/buffer;
• {Image|Buffer|ConstMem|LocalMem}Bandwidth Peak read-only bandwidth of image/buffer/constant/local memory;
• {Image|Buffer}CacheHierarchyPChase Size of each level of cache of image/buffer by the P-chase method.

If the -v flag is given, ArchProbe prints extra human-readable logs to stdout which is also a good source of information.

Experiment data gathered from Google Pixel 4 can be found here.

Tweaking ArchProbe

ArchProbe allows you to adjsut the sensitivity of the probing algorithms in each aspect. In the config file (by default the generated ArchProbe.json), each aspect has a Threshold for the algorithm to decide whether the difference in timing is significant enough to be identified as a numerical jump; Compensate helps the algorithm to smooth out step-like outputs, for example in cache hierarchy probing. In some of the aspects (like RegCount) ArchProbe also allows you to choose the parameter space, namely XxxMin, XxxMax and XxxStep. But in most of the cases, you can rely on the default ranges ArchProbe derived from the device.

Although ArchProbe has algorithms to conclude semantical data like bandwidth and computational throughput from execution timing, these algorithms are susceptible to noisy outputs especially under thermal throttling. So it's too recommended to plot the timing data in .csv files to have a better understanding of the architecture.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

archprobe's Issues

Is rooting required?

Facing permission denied copying to /data/local/tmp.

How to calculate ALUs?

Firstly, I think this is an amazing work! But I have some troubles in calculating the number of ALUs.

I have read code in detail, but I found that the warpsize method A has some difference from List3 illustrated in the paper. So I cannot get the number of ALUs in the log.txt.

Could you tell me how to modify program to get it?

Get stuck when run ./ArchProbe on Adreno 640 on Meizu 16s

Environment:

Device: Meizu 16s
Android version: 9
SoC: Qualcomm Snapdragon 825
GPU: Adreno 640

Have already compiled as https://github.com/microsoft/ArchProbe#how-to-use, and pushed ArchProbe into /data/local/tmp/ArchProbe of target device.

But when run ./ArchProbe, it get stuck after some while.

$ adb shell
16s:/ $ cd /data/local/tmp/ArchProbe
16s:/data/local/tmp/ArchProbe $ ./ArchProbe
[INFO] initialized opencl environment
[INFO] selected device #0: QUALCOMM Snapdragon(TM) (FULL_PROFILE) - QUALCOMM Adreno(TM) (4, OpenCL 2.0 Adreno(TM) 640)
[WARN] configuration file cannot be opened at 'ArchProbe.json', a default configuration will be created
[WARN] report file cannot be opened at 'ArchProbeReport.json', a new report will be created
[INFO] set-up testing environment
[INFO] fetched device report
[INFO]     (qualcomm extension) device page size is 4KB
[INFO]     2GB global memory with 128KB cache consists of 64B cachelines
[INFO]     images up to [16384, 16384] texels are supported
[INFO]     2 SMs with 1024 logical threads in each
[INFO] [Device]
[WARN]     aspect report ('Device') is invalid, a new record is created
[INFO]     reported 'SmCount' = '2'
[INFO]     reported 'LogicThreadCount' = '1024'
[INFO]     reported 'MaxBufferSize' = '2882611200'
[INFO]     reported 'CacheSize' = '131072'
[INFO]     reported 'CachelineSize' = '64'
[INFO]     reported 'MaxImageWidth' = '16384'
[INFO]     reported 'MaxImageHeight' = '16384'
[INFO]     reported 'PageSize_QCOM' = '4096'
[INFO]     reported 'Done' = '1'
[INFO] already know that 'LogicThreadCount' from aspect 'Device' is 1024
[INFO] [WarpSizeMethodA]
[INFO]     initialized table for aspect 'WarpSizeMethodA'
[WARN]     aspect report ('WarpSizeMethodA') is invalid, a new record is created
[INFO]     reported 'WarpThreadCount' = '128'
[INFO]     reported 'Done' = '1'
[INFO]     saved data table to 'WarpSizeMethodA.csv'
[INFO] [WarpSizeMethodB]
[INFO]     initialized table for aspect 'WarpSizeMethodB'
[WARN]     aspect configuration ('WarpSizeMethodB') is invalid, a new record is created
[WARN]     record entry ('Compensate') is invalid, a new record is created
[WARN]     record entry ('Threshold') is invalid, a new record is created
[INFO]     found minimal niter=100 to take 1000us
[WARN]     aspect report ('WarpSizeMethodB') is invalid, a new record is created
[INFO]     reported 'WarpThreadCount' = '64'
[INFO]     discovered the warp size being 64 by method b
[INFO]     reported 'Done' = '1'
[INFO]     saved data table to 'WarpSizeMethodB.csv'
[INFO] [Gflops]
[INFO]     initialized table for aspect 'Gflops'
[INFO]     already know that 'LogicThreadCount' from aspect 'Device' is 1024
[INFO]     already know that 'SmCount' from aspect 'Device' is 2
[WARN]     aspect configuration ('Gflops') is invalid, a new record is created
[WARN]     record entry ('Compensate') is invalid, a new record is created
[WARN]     record entry ('Threshold') is invalid, a new record is created
[INFO]     found minimal niter=100 to take 1000us
[WARN]     aspect report ('Gflops') is invalid, a new record is created
[INFO]     reported 'HalfArch' = 'SISD'
[INFO]     reported 'HalfVecComponentCount' = '1'
[INFO]     reported 'HalfGflops' = '890.102'
[INFO]     found minimal niter=100 to take 1000us
[INFO]     reported 'FloatArch' = 'SISD'
[INFO]     reported 'FloatVecComponentCount' = '1'
[INFO]     reported 'FloatGflops' = '889.921'
[INFO]     reported 'Done' = '1'
[INFO]     saved data table to 'Gflops.csv'
[INFO] [RegCount]
[INFO]     already know that 'Done' from aspect 'Device' is 1
[INFO]     initialized table for aspect 'RegCount'
[INFO]     already know that 'LogicThreadCount' from aspect 'Device' is 1024
[WARN]     aspect configuration ('RegCount') is invalid, a new record is created
[WARN]     record entry ('Compensate') is invalid, a new record is created
[WARN]     record entry ('Threshold') is invalid, a new record is created
[WARN]     record entry ('NRegMin') is invalid, a new record is created
[WARN]     record entry ('NRegMax') is invalid, a new record is created
[WARN]     record entry ('NRegStep') is invalid, a new record is created
[WARN]     record entry ('NGrpMin') is invalid, a new record is created
[WARN]     record entry ('NGrpMax') is invalid, a new record is created
[WARN]     record entry ('NGrpStep') is invalid, a new record is created
[INFO]     found minimal niter=21546 to take 1000us
[WARN]     aspect report ('RegCount') is invalid, a new record is created
[INFO]     testing register availability when only 1 thread is dispatched
[INFO]     183 registers are available at most
[INFO]     reported 'RegCount' = '183'
[INFO]     using 183 registers can have 12 concurrent single-thread workgroups
[INFO]     reported 'FullRegConcurWorkgroupCount' = '12'
[INFO]     using 91 registers can have 24 concurrent single-thread workgroups
[INFO]     reported 'HalfRegConcurWorkgroupCount' = '24'
[INFO]     all physical threads in an sm share 183 registers
[INFO]     reported 'RegType' = 'Pooled'
[INFO]     reported 'Done' = '1'
[INFO]     saved data table to 'RegCount.csv'
[INFO] [BufferVecWidth]
[INFO]     initialized table for aspect 'BufferVecWidth'
[INFO]     already know that 'LogicThreadCount' from aspect 'Device' is 1024
[WARN]     aspect configuration ('BufferVecWidth') is invalid, a new record is created
[WARN]     record entry ('Compensate') is invalid, a new record is created
[WARN]     record entry ('Threshold') is invalid, a new record is created
[INFO]     found minimal niter=4229 to take 1000us
[WARN]     aspect report ('BufferVecWidth') is invalid, a new record is created
[INFO]     optimal buffer read size is 32B
[INFO]     discovered the optimal vectorization for buffer access in 32b-words be 4
[INFO]     reported 'BufferVecSize' = '4'
[INFO]     reported 'Done' = '1'
[INFO]     saved data table to 'BufferVecWidth.csv'
[INFO] [ImageCachelineSize]
[INFO]     initialized table for aspect 'ImageCachelineSize'
[INFO]     already know that 'LogicThreadCount' from aspect 'Device' is 1024
[INFO]     already know that 'MaxImageWidth' from aspect 'Device' is 16384
[INFO]     already know that 'MaxImageHeight' from aspect 'Device' is 16384
[WARN]     aspect configuration ('ImageCachelineSize') is invalid, a new record is created
[WARN]     record entry ('Compensate') is invalid, a new record is created
[WARN]     record entry ('Threshold') is invalid, a new record is created
[WARN]     aspect report ('ImageCachelineSize') is invalid, a new record is created
[INFO]     testing image cacheline size along dim=0
[INFO]     found minimal niter=7974 to take 1000us
[INFO]     can concurrently access 64px with minimal cost along dim=0
[INFO]     reported 'ImgMinTimeConcurThreadCountX' = '64'
[INFO]     testing image cacheline size along dim=1
[INFO]     found minimal niter=7759 to take 1000us
[INFO]     can concurrently access 32px with minimal cost along dim=1
[INFO]     reported 'ImgMinTimeConcurThreadCountY' = '32'
[INFO]     reported 'ImgCachelineSize' = '32'
[INFO]     reported 'ImgCachelineDim' = 'X'
[INFO]     reported 'Done' = '1'
[INFO]     saved data table to 'ImageCachelineSize.csv'
[INFO] [BufferCachelineSize]
[INFO]     initialized table for aspect 'BufferCachelineSize'
[INFO]     already know that 'LogicThreadCount' from aspect 'Device' is 1024
[INFO]     already know that 'CacheSize' from aspect 'Device' is 131072
[WARN]     aspect configuration ('BufferCachelineSize') is invalid, a new record is created
[WARN]     record entry ('Compensate') is invalid, a new record is created
[WARN]     record entry ('Threshold') is invalid, a new record is created
[INFO]     found minimal niter=414 to take 1000us
[WARN]     aspect report ('BufferCachelineSize') is invalid, a new record is created
[INFO]     testing buffer cacheline size
[INFO]     top level buffer cacheline size is 64B
[INFO]     reported 'BufTopLevelCachelineSize' = '64'
[INFO]     reported 'Done' = '1'
[INFO]     saved data table to 'BufferCachelineSize.csv'
[INFO] [ImageBandwidth]
[INFO]     initialized table for aspect 'ImageBandwidth'
[INFO]     already know that 'MaxImageWidth' from aspect 'Device' is 16384
[INFO]     already know that 'LogicThreadCount' from aspect 'Device' is 1024
[INFO]     already know that 'SmCount' from aspect 'Device' is 2
[WARN]     aspect report ('ImageBandwidth') is invalid, a new record is created
[INFO]     reported 'MaxBandwidth' = '190.326'
[INFO]     reported 'MinBandwidth' = '67.9003'
[INFO]     discovered image read bandwidth min=67.9003; max=190.326
[INFO]     reported 'Done' = '1'
[INFO]     saved data table to 'ImageBandwidth.csv'
[INFO] [BufferBandwidth]
[INFO]     initialized table for aspect 'BufferBandwidth'
[INFO]     already know that 'LogicThreadCount' from aspect 'Device' is 1024
[INFO]     already know that 'SmCount' from aspect 'Device' is 2

question about "number of registers"

For the reg_count method, I have a doubt
The comments for Step1 are as follows:

     // Step 1. Probe for the most number of registers we can use by activating
     // different numbers of threads to try to use up all the register resources.
     // In case of register spill the kernel latency would surge.
     //
     // The group size was kept 1 to minimize the impact of scheduling.

But the code of Step1 shows that the currently measured nreg_max is the maximum number of registers available to the kernel when the number of threads remains 1

uint32_t nreg = NREG_MIN;
for (; nreg <= NREG_MAX; nreg += NREG_STEP) {
         double time = bench(1, 1, nreg);
         //....
}

In addition, according to the following sections, we can know that nreg_max will not exceed 512. According to the description in the paper, I guess that nreg_max is 181 measured on Adreno 640.

  const uint32_t NREG_MIN = env.cfg_num("NRegMin", 1);
  const uint32_t NREG_MAX = env.cfg_num("NRegMax", 512);
  const uint32_t NREG_STEP = env.cfg_num("NRegStep", 1);

---

My question is: Is the purpose of step1 to probe the maximum number of available registers on a single thread - nreg_max

If yes, I think the comments of step1 is not quite correct and will lead to misunderstanding.
And I want to know that on Adreno 640, since there is a 384 x 181 shared 32bit-register pool, why the maximum number of available registers for a single thread is 181, is it related to the allocation strategy of the pool?

If not, is the value of nreg_max 384 x 181?

Values between in GPU hierarchy images and output files

The two captured images from the document "Romou: Rapidly Generate High-Performance Tensor Kernels for Mobile GPUs" as below:

(1) The maximum number of registers for each work item
(2) The size, cacheline size, and bandwidth of memory hierarchy including all levels of unified and texture caches, local,
constant and global memory
(3) The number of threads in a warp
(4) The number of ALUs in a shader core

But after I output the .json and .csv files, I cannot make all connections between the values of properties and the values of GPUs hierarchy properties. Can you please figure out all of them, especially (2) and (4) ?

Besides, I have found some connections and make a list as below (just take Adreno 640 GPU for example), can you please check it right or wrong?

• Shader cores count: Value in ArchProbeReport.json - [Device] - SmCount?
• Execute engines count: Value in where?
• ALUs count: Value in where?
• Warp size: Value in [ArchProbeReport.json] - [WarpSizeMethod{A|B}] - WarpThreadCount?
• What's 384, value in where?
• Registers count: Value (is 181?) in ArchProbeReport.json - [RegCount] - RegCount?
• Register type (Pooled / Dedicated): Value in ArchProbeReport.json - [RegCount] - RegType?
• Register bits: Value (4B) in where?
• Texture L1 cache bandwidth: Value in ArchProbeReport.json - [ImageBandwidth] - MinBandwidth / MaxBandwidth?
• Local memory bandwidth: Value in ArchProbeReport.json - [BufferBandwidth] - MinBandwidth / MaxBandwidth?
• Unified L2 cache bandwidth: Value in ArchProbeReport.json - [BufferBandwidth] - MinBandwidth / MaxBandwidth?
• Constant memory bandwidth: Value in ArchProbeReport.json - [BufferBandwidth] - MinBandwidth / MaxBandwidth?
• Texture L1 cache size: Value in where?
• Local memory size: Value in where?
• Unified L2 cache size: Value in where?
• Constant memory size: Value in where?
• Global memory bandwidth: Value in where?

Continuous discussion about calculating the number of ALUs

This issue is continued from #6

Thanks for your reply and I am sorry for my late one.

I think the warp size calculated in this program is right. But I am still confused in how to get the number of parallel warps so that I can calculate the number of ALUs. Besides, I use 1 workgroup and (64, 6, 2) size to test. Delete the break you pointed. The output will become:

64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

I sincerely hope to get answer about the first question which can help me count the number of ALUs.

Does this contains the code to reproduce or generate the figures or data of the document?

Does this contains the code to reproduce or generate the figures or data of the document "Romou: Rapidly Generate High-Performance Tensor Kernels for Mobile GPUs"?