ray.put starts to lag after pushing too many weights about ray

[robertnishihara]

Interesting, I tried a simpler version

import ray
import time
import numpy as np

ray.init(num_workers=0)

x = np.zeros(10000000)

l = []
for i in range(500):
  start = time.time()
  ray.put(x)
  end = time.time()
  l.append(end - start)

And got a similar plot.

[richardliaw]

If you zoom into the first 200, there should be a upward tick in the median time taken right?

Here's what my machine is showing (from a replicated run of your script)

[robertnishihara]

The same plot but with 1GB object store memory (the previous plot was about 13GB).

[robertnishihara]

This time with x = np.zeros(10000000) so the objects are 1/10th the size (and 1GB object store memory).

[pcmoritz]

This is the same workload in linux (using half of the ram for the plasma store):

I think the large spikes we saw in mac os x with 13gb come from hitting the disk.

[robertnishihara]

Closing since I think we decided this was due to swapping.