Comments (4)
Pandana already utilises parallel processing as it has a C++ backend.
I'm not too sure why, but it's always been quicker for me to not use network.precompute and just run the analyses.
My 16 core Ryzen can initiate a 2 million edge pdna.Network in around 2 minutes so it's definitely not something you should break up into chunks if you can help it.
from pandana.
Thank you for the reply. I removed the precompute line as suggested and ran just pdna.Network. It's been one hour and still computing, is it because I only have 4 cores 8 threads?
//Edit:
After I broke them down into provinces, the computation time was in matter of seconds, even with the precompute code added!
However, I have another question: is it possible to convert the final result from the plot into a shapefile? Or is there a way that I can save the data for each node (lat, long, accessibility distance computed) into a geodataframe?
There is an option save_hdf5, but it doesn't save the associated data with it.
from pandana.
Oh yes sounds like a cpu limitation there, breaking things up does create a lot of issues on the validity of analysis on the edge of provinces i.e. something like Ottawa and Gatineau perhaps.
Just join the results of the analysis back to the input geodataframe and export that, they should both be in the same order so it's just a straight join.
from pandana.
I am trying to create a network. With 480000 rows of my dataset which includes 482801 nodes and 960000 edges, it works (in 2 seconds), but by adding a bit more ( 490000rows) it gets stuck like:
Generating contraction hierarchies with 16 threads.
Setting CH node vector of size 492730
Setting CH edge vector of size 980000
Range graph removed 980000 edges of 1960000
and nothing happens. This is my system configuration:
Docker container
Windows 10, wsl2
Operating system: 64-bit operating system, x64-based processor
Processor: 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz
32GB RAM, 8 CPU core
Python version: 3.10.2
Pandana version: 0.6.1
Is it related to my machine resources?
from pandana.
Related Issues (20)
- Support for M1 Macs HOT 3
- [Question] How to get different Key|Value combinations in osm.node_query
- [Feature Request] Aggreggation queries to subset of nodes
- Binary incompatibility error with older versions of NumPy HOT 1
- Shortest path length exists but shortest path is empty HOT 5
- shortest path distance between non connected nodes
- Allow specifying mapping distance for POIs
- Issue: the POI Accessibility computation is taking forever
- OSM loader - POIs that are ways in OSM HOT 1
- how to increase the performance of the aggregate when impedance is changed to time?
- net.shortest_paths doesn't work with multiprocessing
- tables sub-dependency versions conflict with python 3.10 compatibility when installing with pip HOT 1
- incorrect shortest paths solutions when impendance values are particularly small or large HOT 1
- Plotting with Pandana does not render title HOT 1
- osm.pdna_network_from_bbox error HOT 6
- Pulling network object from OSM bugged HOT 2
- New release ?
- Google Colab install pandana error HOT 1
- query the OSM API for the street network within the specified bounding box error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandana.