speed comparison about graphless-neural-networks

[ShichangZh]

We did nothing fancy but using a timer to measure the model inference time. We mimic an online setting as in Figure 2 to infer one node at a time. We repeat the procedure 10 times to reduce the variance from randomly chosen nodes. The timing function we used is the following.
https://pytorch.org/docs/stable/benchmark_utils.html

[ShichangZh]

Below are the functions I wrote for time measurement. Note that the results could be different from our paper as they depend on your hardware.

def sage_inference(model, feats, dataloader):
    for step, (input_nodes, output_nodes, blocks) in enumerate(dataloader):
        model(blocks, feats[input_nodes])

def sage_measure(conf, model_name, L, prune_budget=0.5, fan_out=None):
    conf = copy.copy(conf)
    
    conf['model_name'] = model_name
    if model_name == 'SAGE':
        get_model = lambda conf: Model(conf).encoder
    elif model_name == 'SAGEQuantize':
        get_model = lambda conf: torch.quantization.quantize_dynamic(Model(conf).encoder)
    elif model_name == 'SAGEPrune':
        conf['hidden_dim'] = int(conf['hidden_dim'] * prune_budget)
        get_model = lambda conf: Model(conf).encoder
    gnn_dur = []
    for l in range(1, L+1):
        conf['num_layers'] = l
        set_seed(0)
        model = get_model(conf)
        sampler = dgl.dataloading.MultiLayerNeighborSampler([fan_out] * model.num_layers)
        model.eval()
        with torch.no_grad():
            dur = []
            for exp in range(num_exp):
                set_seed(exp)
                idx = torch.randperm(g.num_nodes())[0]
                dataloader = dgl.dataloading.NodeDataLoader(g, idx, sampler, batch_size=batch_size, shuffle=False, drop_last=False)
                t0 = benchmark.Timer(
                    stmt='sage_inference(model, feats, dataloader)',
                    setup='from __main__ import sage_inference',
                    globals={'model': model, 'feats': feats, 'dataloader': dataloader})
                measurement = t0.blocked_autorange(min_run_time=0.2)
                dur += [measurement.median * 1000]
            gnn_dur += [np.mean(dur)]
    return gnn_dur

[Angazenn]

Thx for your quick reply!