Coder Social home page Coder Social logo

Comments (4)

double-u-a avatar double-u-a commented on June 23, 2024

Pandana already utilises parallel processing as it has a C++ backend.
I'm not too sure why, but it's always been quicker for me to not use network.precompute and just run the analyses.
My 16 core Ryzen can initiate a 2 million edge pdna.Network in around 2 minutes so it's definitely not something you should break up into chunks if you can help it.

from pandana.

wendy-ngan avatar wendy-ngan commented on June 23, 2024

Thank you for the reply. I removed the precompute line as suggested and ran just pdna.Network. It's been one hour and still computing, is it because I only have 4 cores 8 threads?

//Edit:
After I broke them down into provinces, the computation time was in matter of seconds, even with the precompute code added!
However, I have another question: is it possible to convert the final result from the plot into a shapefile? Or is there a way that I can save the data for each node (lat, long, accessibility distance computed) into a geodataframe?
There is an option save_hdf5, but it doesn't save the associated data with it.

from pandana.

double-u-a avatar double-u-a commented on June 23, 2024

Oh yes sounds like a cpu limitation there, breaking things up does create a lot of issues on the validity of analysis on the edge of provinces i.e. something like Ottawa and Gatineau perhaps.

Just join the results of the analysis back to the input geodataframe and export that, they should both be in the same order so it's just a straight join.

from pandana.

Fardin3303 avatar Fardin3303 commented on June 23, 2024

I am trying to create a network. With 480000 rows of my dataset which includes 482801 nodes and 960000 edges, it works (in 2 seconds), but by adding a bit more ( 490000rows) it gets stuck like:
Generating contraction hierarchies with 16 threads.
Setting CH node vector of size 492730
Setting CH edge vector of size 980000
Range graph removed 980000 edges of 1960000

and nothing happens. This is my system configuration:
Docker container
Windows 10, wsl2
Operating system: 64-bit operating system, x64-based processor
Processor: 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz
32GB RAM, 8 CPU core
Python version: 3.10.2
Pandana version: 0.6.1

Is it related to my machine resources?

from pandana.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.