How Timeloop takes latency of different types of memories into account while measuring performance? about timeloop

[05Tushar]

There was an example in this exercise too timeloop-accelergy-exercises where we changed the type of DRAM from LPDDR4 to HBM2 but it had no effect on performance (number of cycles) although they have different latencies and bandwidths.

[angshuman-parashar]

Timeloop's performance modeling is bandwidth-based, and the user configures the bandwidth using the component attributes. LPDDR4 and HBM2 do not "have" an inherent baked-in bandwidth, the bandwidth depends on the number of channels, bits per channel, clock speed etc. All these are parameters abstracted into the user-specified "bandwidth" attribute in a Timeloop memory component, and as far as performance modeling is concerned this attribute dictates the component's capabilities. So if you provision lower DRAM bandwidth, workloads may begin to get bottlenecked by that bandwidth and throttle down. The technology (i.e., LPDDR4 vs. HBM2) is used to invoke the appropriate Accelergy plugin for energy modeling.

Regarding your small-vs-large RF example: Again, throughput-based modeling is sensitive to bandwidth, not latency. If your larger RF (despite its increased latency) is able to initiate the same #requests per cycle as your smaller RF, then in general your steady-state performance should remain unchanged. This steady-state performance is what Timeloop models using the bandwidth attribute. Yes, the initial pipeline fill/drain latency is not accounted for in today's model, but this is a minor tax for most realistically-sized network layers.

[05Tushar]

Thank you so much sir for the clarification!