r-spatial / rqgis Goto Github PK

I am trying to run some calculus using RQGIS and calling GRASS functions, but I am getting some errors (it could be an install problem although my GRASS 7 seems to work fine):

> data(dem)
> find_algorithms(search_term = "slope")
Assuming that your root path is '/usr'!
 [1] "D-Infinity Upslope Dependence------------------------>taudem:dinfinityupslopedependence"                                                                                 
...
[20] "r.slope.aspect - Generates raster layers of slope, aspect, curvatures and partial derivatives from a elevation raster layer.--->grass:r.slope.aspect"                    
> params <- get_args_man(alg = "grass:r.slope.aspect")
Assuming that your root path is '/usr'!
> params$elevation = dem
> params$slope <- "slope.asc"
> params$aspect <- "aspect.asc"
> out2 <- run_qgis(alg = "grass:r.slope.aspect", params = params, load_output = params$slope)
Assuming that your root path is '/usr'!
Error in run_qgis(alg = "grass:r.slope.aspect", params = params, load_output = params$slope) : 
  Error: Wrong parameter value: None

Some of the main reasons for me to use RQGIS is to be able to access GRASS GIS functions without starting up GRASS. My session info is:

> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS

other attached packages:
[1] rgdal_1.1-10         raster_2.5-8         RQGIS_0.2.0.9000    
[4] GSIF_0.5-3           sp_1.2-3             RevoUtilsMath_10.0.0

Thank you!

Any RQGIS call returns

Traceback (most recent call last):
  File "py_cmd.py", line 3, in <module>
    from qgis.core import *
  File "/usr/local/Cellar/qgis2/2.18.3/QGIS.app/Contents/Resources/python/qgis/__init__.py", line 26, in <module>
    from qgis.PyQt import QtCore
  File "/usr/local/Cellar/qgis2/2.18.3/QGIS.app/Contents/Resources/python/qgis/PyQt/QtCore.py", line 26, in <module>
    import sip
ImportError: No module named sip

Related:

• http://stackoverflow.com/questions/30004778/no-module-named-sip-when-running-qgis-from-python

• http://gis.stackexchange.com/questions/150767/importerror-no-module-named-sip

• http://gis.stackexchange.com/questions/144908/can-we-avoid-using-sip-library-when-writing-standalone-qgis-python-script

SIP is not found by /usr/bin/python. Using my default homebrew python, SIP is found but I get

>>> from qgis.core import *
Segmentation fault: 11

@rkrug Are you getting the same error message?

We should set up unit testing to easen up future debugging - especially for code chunks which might be affected by QGIS version changes.

You probably have read Hadleys book chapter about unit testing using the testthat package:
http://r-pkgs.had.co.nz/tests.html

The test checks can be automated using codecov (https://codecov.io) during the Travis CI checks.

Error in py_run_string_impl(code, local, convert) :
NameError: name 'processing' is not defined

Detailed traceback:
File "", line 1, in

#The whole session
library(RQGIS)
data("dem", package = "RQGIS")
set_env(root="/Applications/QGIS.app")
$root
[1] "/Applications/QGIS.app"

$qgis_prefix_path
[1] "/Applications/QGIS.app/Contents"

$python_plugins
[1] "/Applications/QGIS.app/Contents/Resources/python/plugins"

Warning message:
In set_env(root = "/Applications/QGIS.app") :
We recognized that you are using the Kyngchaos QGIS binary.
Please consider installing QGIS from homebrew:'https://github.com/OSGeo/homebrew-osgeo4mac'. Run 'vignette(install_guide)' for installation instructions.
The Kyngchaos installation throws some warnings during processing. However, usage/outcome is not affected and you can continue using the Kyngchaos installation.

open_app()
QSqlDatabase: QSQLITE driver not loaded
QSqlDatabase: available drivers:
ERROR: Opening of authentication db FAILED
QSqlQuery::prepare: database not open
WARNING: Auth db query exec() FAILED
Failed CDLL(/opt/local/lib/libgeos_c.dylib)
Error in py_run_file_impl(file, local, convert) :
OSError: Could not find lib c or load any of its variants [].

Detailed traceback:
File "", line 2, in
File "/Applications/QGIS.app/Contents/Resources/python/plugins/processing/core/Processing.py", line 148, in initialize
Processing.addProvider(c())
File "/Applications/QGIS.app/Contents/Resources/python/plugins/processing/algs/qgis/QGISAlgorithmProvider.py", line 225, in init
from .Polygonize import Polygonize
File "/Applications/QGIS.app/Contents/Resources/python/qgis/utils.py", line 607, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "/Applications/QGIS.app/Contents/Resources/python/plugins/processing/algs/qgis/Polygonize.py", line 28, in
from shapely.ops import polygonize
File "/Applications/QGIS.app/Contents/Resources/python/qgis/utils.py", line 607, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "/Library/Frameworks/GEOS.framework/Versions/3/Pyth
find_algorithms("slope(.+)?aspect")
Error in py_run_string_impl(code, local, convert) :
NameError: name 'processing' is not defined

Detailed traceback:
File "", line 1, in

Would it make sense to add a progress bar to run_qgis()? I think it's possible through pyqgis, but no idea how.

@jannes-m Have you thought about adding some additional functions to this package? My idea is to have a function in R, where you specify a spatial vector data and could add a style to it (grouping column, color palette, etc.).
I've already have a very limited code for adding a fill color to polygons and could share it if you want.

Right now, RQGIS does not work with all QGIS installation methods (e.g. brew install qgis2 on macOS).

Two options:

1. If other installation approaches are used than specified in the install_guide vignette, throw ERROR and provide link to install_guide vignette.
2. Solve problems of problematic QGIS installation methods

(1) Should be preferred as long as (2) is not solved -> Stability > Convenience

Mac

• Solve RQGIS problems when QGIS is installed via brew install qgis2
• Check if QGIS was installed using brew cask install qgis

Windows

• OSGEO4W

Linux

• if Python web scraping "Error" is True, construct html file

Prevent README change when compiling either on Windows or Mac

previously svn -> change the documentation accordingly

When I run the following:

library(sf)
library(mapview)
library(RQGIS)
library(dplyr)
library(gdalUtils)
library(data.table)

setwd("test_union")

county = st_cast(franconia, "POLYGON")
county = data.frame(cbind(mapview:::sf2DataFrame(county, drop_sf_column = TRUE),
                          st_as_text(st_geometry(county), EWKT = FALSE)))
names(county)[ncol(county)] = "WKT"
fwrite(county, "franconia_county.csv", row.names = FALSE)

district = franconia %>%
  group_by(district) %>%
  summarize() %>%
  st_cast() %>%
  st_cast("POLYGON")

district = data.frame(cbind(mapview:::sf2DataFrame(district, drop_sf_column = TRUE),
                            st_as_text(st_geometry(district), EWKT = FALSE)))
names(district)[ncol(district)] = "WKT"
fwrite(district, "franconia_district.csv", row.names = FALSE)

ogr2ogr("franconia_county.csv", "franconia_county.shp", f = "ESRI Shapefile",
        oo = "KEEP_GEOM_COLUMNS=NO", overwrite = TRUE, nlt = "POLYGON",
        a_srs = "EPSG:4326")
ogr2ogr("franconia_district.csv", "franconia_district.shp", f = "ESRI Shapefile",
        oo = "KEEP_GEOM_COLUMNS=NO", overwrite = TRUE, nlt = "POLYGON",
        a_srs = "EPSG:4326")

params = get_args_man("grass7:v.overlay")

params$ainput = "/home/ede/test_union/franconia_county.shp"
params$atype = "area"
params$binput = "/home/ede/test_union/franconia_district.shp"
params$operator = "or"
params$output = "/home/ede/test_union/grass_or.shp"

run_qgis("grass7:v.overlay", params = params)

I get the following error:

Error in run_qgis("grass7:v.overlay", params = params) : 
  Error: Wrong parameter value: area

Any ideas why? When I run the algorithm in QGIS with those layers everything works fine.

sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.5 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=de_DE.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=de_DE.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] lubridate_1.6.0     plotly_4.5.2        ggplot2_2.2.1       latticeExtra_0.6-28 RColorBrewer_1.1-2 
 [6] lattice_0.20-35     cranlogs_2.1.0      data.table_1.10.4   gdalUtils_2.0.1.7   dplyr_0.5.0        
[11] RQGIS_0.2.0.9000    mapview_1.2.73      leaflet_1.1.0.9000  sf_0.4-2           

loaded via a namespace (and not attached):
 [1] purrr_0.2.2       colorspace_1.2-6  htmltools_0.3.5   stats4_3.3.3      viridisLite_0.1.3
 [6] yaml_2.1.14       base64enc_0.1-3   R.oo_1.20.0       withr_1.0.2       DBI_0.6-1        
[11] R.utils_2.4.0     sp_1.2-4          foreach_1.4.3     plyr_1.8.4        stringr_1.2.0    
[16] munsell_0.4.3     gtable_0.2.0      raster_2.5-8      R.methodsS3_1.7.1 devtools_1.12.0  
[21] htmlwidgets_0.8   codetools_0.2-15  memoise_1.0.0     knitr_1.15.1      httpuv_1.3.3     
[26] crosstalk_1.0.1   curl_2.5          Rcpp_0.12.10      xtable_1.8-2      udunits2_0.13    
[31] scales_0.4.1      satellite_0.3.0   webshot_0.3.2     jsonlite_1.4      mime_0.5         
[36] png_0.1-7         digest_0.6.12     stringi_1.1.5     shiny_1.0.0       grid_3.3.3       
[41] rgdal_1.2-7       tools_3.3.3       magrittr_1.5      lazyeval_0.2.0    tibble_1.3.0     
[46] tidyr_0.6.0       assertthat_0.2.0  httr_1.2.1        iterators_1.0.8   R6_2.2.0         
[51] units_0.4-4       git2r_0.18.0     

I am trying to intersect some SpatialPolygonsDataFrames with each other using the qgis:intersection algorithm. However, I am getting some unexpected results. Namely, all calls to run_qgis with alg = "qgis:intersection" after the first return the same thing.

A minimal example:

# some example SpatialPolygonsDataFrames
library(sp)
crds <- cbind(x=c(0, 0, 400, 400, 0), y=c(0, 400, 400, 0, 0))
Pl <- Polygon(crds)
ID <- "1"
Pls <- Polygons(list(Pl), ID=ID)
SPls <- SpatialPolygons(list(Pls))
df <- data.frame(value=1, row.names=ID)
SPDF1 <- SpatialPolygonsDataFrame(SPls, df)
crds <- cbind(x=c(0, 0, 400, 400, 0), y=c(0, 400, 400, 0, 0)) - 200
Pl <- Polygon(crds)
ID <- "2"
Pls <- Polygons(list(Pl), ID=ID)
SPls <- SpatialPolygons(list(Pls))
df <- data.frame(value=1, row.names=ID)
SPDF2 <- SpatialPolygonsDataFrame(SPls, df)
crds <- cbind(x=c(0, 0, 400, 400, 0), y=c(0, 400, 400, 0, 0)) + 200
Pl <- Polygon(crds)
ID <- "3"
Pls <- Polygons(list(Pl), ID=ID)
SPls <- SpatialPolygons(list(Pls))
df <- data.frame(value=1, row.names=ID)
SPDF3 <- SpatialPolygonsDataFrame(SPls, df)


library(RQGIS)
RQGIS::set_env("/Applications/QGIS.app")
test1 <- RQGIS::run_qgis("qgis:intersection", INPUT = SPDF1, INPUT2 = SPDF2, OUTPUT = file.path(tempdir(), "test1.shp"), load_output = TRUE)

test2 <- RQGIS::run_qgis("qgis:intersection", INPUT = SPDF1, INPUT2 = SPDF3, OUTPUT = file.path(tempdir(), "test2.shp"), load_output = TRUE)

identical(test1, test2)
# TRUE

But in reality SPDF1 intersect with SPDF2 is not equal to SPDF1 intersect with SPDF3.

My session info:

R version 3.4.0 (2017-04-21)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.5

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] RQGIS_1.0.1.9000 sp_1.2-4

loaded via a namespace (and not attached):
[1] Rcpp_0.12.11 lattice_0.20-35 sf_0.4-3 grid_3.4.0 R6_2.2.1 jsonlite_1.5
[7] DBI_0.6-1 magrittr_1.5 units_0.4-4 stringi_1.1.5 rlang_0.1.1 raster_2.5-8
[13] reticulate_0.9.0.9001 rgdal_1.2-7 tools_3.4.0 stringr_1.2.0 udunits2_0.13 readr_1.1.1
[19] hms_0.3 parallel_3.4.0 compiler_3.4.0 tibble_1.3.3

and I am using QGIS 2.18.7 from Kyngchaos.

sometimes RQGIS gets confused with working and temporary folders (run_qgis).
E.g., Manuel Spinolas v.split.length example. Link?

Hi,

tried to use get_args_man() on a win-machine from within RStudio.

 env <- set_env('C:\\Program Files\\QGIS Lyon')
 args <- get_args_man(alg = "qgis:joinattributesbylocation", 
                       options = TRUE,
                        qgis_env = env)

This produced following error:

 Error in file(file, "rt") : cannot open the connection
 In addition: Warning messages:
 1: running command 'batch_cmd.cmd' had status 1 
 2: In file(file, "rt") :
 cannot open file 'C:\Users\user\AppData\Local\Temp\RtmpOCWJmK/output.csv': No such file or # directory

Seems to be a forward/backslash problem '\RtmpOCWJmK/output.csv'

On my mac using:

env <- set_env()

produced

find: /usr/local/Cellar/: No such file or directory
Warnmeldung:
Ausführung von Kommando 'find /usr/local/Cellar/ -name 'QGIS.app'' ergab Status 1 

but

env <- set_env('/Applications/QGIS.app')

worked, but get_args_man() produced the following message

sh: sysctl: command not found
sh: grep: command not found
sh: sw_vers: command not found
sh: grep: command not found
Error in sys.excepthook:
Traceback (most recent call last):
  File "/Applications/QGIS.app/Contents/Resources/python/qgis/utils.py", line 171, in qgis_excepthook
    showException(type, value, tb, None, messagebar=True)
  File "/Applications/QGIS.app/Contents/Resources/python/qgis/utils.py", line 82, in showException
    open_stack_dialog(type, value, tb, msg)
  File "/Applications/QGIS.app/Contents/Resources/python/qgis/utils.py", line 117, in open_stack_dialog
    iface.messageBar().popWidget()
AttributeError: 'NoneType' object has no attribute 'messageBar'

Original exception was:
Traceback (most recent call last):
  File "py_cmd.py", line 33, in <module>
    vals.append(param.getValueAsCommandLineParameter())
  File "/Applications/QGIS.app/Contents/Resources/python/plugins/processing/core/parameters.py", line 593, in getValueAsCommandLineParameter
    ParameterString.ESCAPED_NEWLINE)) + '"'
AttributeError: 'NoneType' object has no attribute 'replace'
Fehler in file(file, "rt") : kann Verbindung nicht öffnen
Zusätzlich: Warnmeldungen:
1: Ausführung von Kommando 'sh batch_cmd.sh' ergab Status 1 
2: In file(file, "rt") :
  kann Datei '/var/folders/2j/jdxbpssn2075kpg880jv78w40000gn/T//Rtmpidy5o1/output.csv' nicht öffnen: No such file or directory

RQGIS functions will not run out-of-the-box with Homebrew Python 2.7.13 (I get "ImportError: No module named sip"). I have to manually add the following to build_cmds() in line 58/59 of helper_funs.R:

":/usr/local/lib/qt-4/python2.7/site-packages:/usr/local/lib/python2.7/site-packages"

Adding the same paths to $PYTHONPATH in ~/.bashrc and then appending a ":$PYTHONPATH" to line 59 of helper_funs.R does not work.

I've found a weird behaviour which is causing some issue. When I run a algorithm, it hardcodes parameters making the function get_args_man function useless.

library(RQGIS)
monqgis <- set_env("C:\\OSGeo4W64")
 
###  first run:
params <- get_args_man(alg = "gdalogr:rastercalculator", qgis_env = monqgis)
params$FORMULA <- "A+1"
params$INPUT_A <- "c:/temp/depth_D500.tif"
params$OUTPUT <- "c:/temp/test.tif"
run_qgis(alg = "gdalogr:rastercalculator", params=params, qgis_env = monqgis)
$OUTPUT
[1] "c:/temp/test.tif"

get_args_man(alg = "gdalogr:rastercalculator", qgis_env = monqgis)
$INPUT_A
[1] "\"c:/temp/depth_D500.tif\""

$BAND_A
[1] "\"\"1\"\""

$INPUT_B
[1] "\"None\""

$BAND_B
[1] "\"\"1\"\""

$INPUT_C
[1] "\"None\""

$BAND_C
[1] "\"\"1\"\""

$INPUT_D
[1] "\"None\""

$BAND_D
[1] "\"\"1\"\""

$INPUT_E
[1] "\"None\""

$BAND_E
[1] "\"\"1\"\""

$INPUT_F
[1] "\"None\""

$BAND_F
[1] "\"\"1\"\""

$FORMULA
[1] "\"A+1\""

$NO_DATA
[1] "\"\"\"\""

$RTYPE
[1] "0"

$EXTRA
[1] "\"\"\"\""

$OUTPUT
[1] "None"

 ###  second run in same session:
params <- get_args_man(alg = "gdalogr:rastercalculator", qgis_env = monqgis)
params$FORMULA <- "A+1"
params$INPUT_A <- "c:/temp/depth_D500.tif"
params$OUTPUT <- "c:/temp/test.tif"
run_qgis(alg = "gdalogr:rastercalculator", params=params, qgis_env = monqgis)
$OUTPUT
[1] "c:/temp/test.tif"

get_args_man(alg = "gdalogr:rastercalculator", qgis_env = monqgis)
$INPUT_A
[1] "\"c:/temp/depth_D500.tif\""

$BAND_A
[1] "\"\"\"1\"\"\""

$INPUT_B
[1] "\"'None'\""

$BAND_B
[1] "\"\"\"1\"\"\""

$INPUT_C
[1] "\"'None'\""

$BAND_C
[1] "\"\"\"1\"\"\""

$INPUT_D
[1] "\"'None'\""

$BAND_D
[1] "\"\"\"1\"\"\""

$INPUT_E
[1] "\"'None'\""

$BAND_E
[1] "\"\"\"1\"\"\""

$INPUT_F
[1] "\"'None'\""

$BAND_F
[1] "\"\"\"1\"\"\""

$FORMULA
[1] "\"A+1\""

$NO_DATA
[1] "\"\"\"\"\"\""

$RTYPE
[1] "0"

$EXTRA
[1] "\"\"\"\"\"\""

$OUTPUT
[1] "None"

 
###  third run in same session:
params <- get_args_man(alg = "gdalogr:rastercalculator", qgis_env = monqgis)
params$FORMULA <- "A+1"
params$INPUT_A <- "c:/temp/depth_D500.tif"
params$OUTPUT <- "c:/temp/test.tif"
run_qgis(alg = "gdalogr:rastercalculator", params=params, qgis_env = monqgis)
Error in run_qgis(alg = "gdalogr:rastercalculator", params = params, qgis_env = monqgis) : 
  Unable to execute algorithm
Wrong parameter value: None

 
get_args_man(alg = "gdalogr:rastercalculator", qgis_env = monqgis)
$INPUT_A
[1] "\"c:/temp/depth_D500.tif\""

$BAND_A
[1] "\"\"\"\"1\"\"\"\""

$INPUT_B
[1] "\"''None''\""

$BAND_B
[1] "\"\"\"\"1\"\"\"\""

$INPUT_C
[1] "\"''None''\""

$BAND_C
[1] "\"\"\"\"1\"\"\"\""

$INPUT_D
[1] "\"''None''\""

$BAND_D
[1] "\"\"\"\"1\"\"\"\""

$INPUT_E
[1] "\"''None''\""

$BAND_E
[1] "\"\"\"\"1\"\"\"\""

$INPUT_F
[1] "\"''None''\""

$BAND_F
[1] "\"\"\"\"1\"\"\"\""

$FORMULA
[1] "\"A+1\""

$NO_DATA
[1] "\"\"\"\"\"\"\"\""

$RTYPE
[1] "0"

$EXTRA
[1] "\"\"\"\"\"\"\"\""

$OUTPUT
[1] "None"

> 

You can see that for all unused arguments, there is some kind of arguments buildup... This is a weird and problematic behaviour if not a bug.

Thanks
Bastien Ferland-Raymond

The README.md suggests that an OSGeo4W folder + installation is necessary. I am on Windows and I do not have such an installation. I was able to run the README.md tutorial just fine by setting:

my_env <- set_env("C:/Program Files/QGIS Wien")

Is an OSGeo4W folder + installation strictly necessary? If not, I think the check_apps error message:

Please install qgis using the 'OSGEO4W' advanced installation routine.

would be more informative if it was changed to something like:

No qgis installation found at the specified path.

Any plans to support simple features objects as inputs to run_qgis()?

Hi everyone,

Is there a way to execute the qgis plot functions using RQGIS? The r options to plot sp or sf objects (even with ggplot2 with geom_sf) are too slow.

Thanks

Hi,

I'm having an error with RQGIS. When I ran open_app() I get the following error:

Error in py_run_string_impl(code, local, convert) :
  ImportError: No module named _gdal

Detailed traceback:
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/osgeo/__init__.py", line 21, in <module>
  File "/usr/local/lib/python2.7/site-packages/osgeo/__init__.py", line 13, in swig_import_helper

I also tried to run find_algorithms("slope(.+)?aspect"), and in this case I got the following error:

Error in py_run_string_impl(code, local, convert) :
  NameError: name 'processing' is not defined

Detailed traceback:
  File "<string>", line 1, in <module>

I'm running mac OSX and I installed QGIS stable 2.14.14 through homebrew
thanks!

• macOS (Travis)
• Linux (Travis)
• Windows (appveyor)

I've tried to create points inside of polygons using the qgis function. However, each time I get the output of the first calculation (after a new R session):

library(RQGIS)                                                                   
library(sf)                                                                      
library(tidyverse)                                                               
                                                                                 
nc = st_read(system.file("shape/nc.shp", package="sf"), quiet = TRUE)            
points_in_nc <- run_qgis(alg = "qgis:randompointsinsidepolygonsfixed",           
VECTOR = nc,                                                                     
OUTPUT = paste0(tempfile(), ".shp"),                                             
load_output = TRUE)                                                              
#> Assuming that your root path is '/usr'!
#> $OUTPUT
#> [1] "/tmp/RtmpBhGOY3/file4f416723ce66.shp"
nrow(nc)                                                                         
#> [1] 100
nrow(points_in_nc)                                                               
#> [1] 100
                                                                                 
nc2 = st_read(system.file("shape/nc.shp", package="sf"), quiet = TRUE)[c(1:10), ]
points_in_nc2 <- run_qgis(alg = "qgis:randompointsinsidepolygonsfixed",          
VECTOR = nc2,                                                                    
OUTPUT = paste0(tempfile(), ".shp"),                                             
load_output = TRUE)                                                              
#> $OUTPUT
#> [1] "/tmp/RtmpBhGOY3/file4f4166526e53.shp"
nrow(nc2)                                                                        
#> [1] 10
nrow(points_in_nc2)                                                              
#> [1] 100

This is probably related to #74.

Update: It works well in QGIS

In the future will it be possible to use Simple Features objects? At the moment I'm using RQGIS v 0.2.0.

https://cran.r-project.org/web/packages/sf/sf.pdf

I am trying to run grass7:r.slope.aspect, but I get the following error:

> library(RQGIS)
> qgis_env <- set_env()  
Assuming that your root path is '/usr'!
> data("dem")
> params <- get_args_man(alg = "grass7:r.slope.aspect", qgis_env = qgis_env, options = TRUE) #r.relief")
> params$elevation <- dem
> params$slope <- file.path(tempdir(), "slope.tif")
> out <- run_qgis("grass7:r.slope.aspect", params = params, qgis_env = qgis_env, load_output = TRUE)
$slope
[1] "/tmp/RtmpYTThpt/slope.tif"

$aspect
[1] "/tmp/processing411d87366b8046dc814e00d9fc1f0d2e/9ebb9053cf5f4f4d9bfb6f83823a5199/aspect.tif"

$dxy
[1] "/tmp/processing411d87366b8046dc814e00d9fc1f0d2e/232f604ada5749fa965f95cd37739343/dxy.tif"

$dxx
[1] "/tmp/processing411d87366b8046dc814e00d9fc1f0d2e/ac248b906ae440818095abcd08ef003b/dxx.tif"

$tcurvature
[1] "/tmp/processing411d87366b8046dc814e00d9fc1f0d2e/fea9ba1e625f4acea1822ba13808da44/tcurvature.tif"

$dx
[1] "/tmp/processing411d87366b8046dc814e00d9fc1f0d2e/2d82923584e34b79805ba41115220939/dx.tif"

$dy
[1] "/tmp/processing411d87366b8046dc814e00d9fc1f0d2e/fb5c7b158bf842df9b142e15cdac2264/dy.tif"

$pcurvature
[1] "/tmp/processing411d87366b8046dc814e00d9fc1f0d2e/c4738199463d4ae3833e8fea9fa95506/pcurvature.tif"

$dyy
[1] "/tmp/processing411d87366b8046dc814e00d9fc1f0d2e/68aa9d06fa3b4"/tmp/processing7ab996f38d97402daf0aac11fbe65342/1d5526631d1f4d1bb855a571a8179e17/pcurvature.tif"

$dyy
[1] "/tmp/processing7ab996f38d97402daf0aac11fbe65342/e913e5ab944c42808e602cbc9fab2992/dyy.tif"

Error in FUN(X[[i]], ...) : 
  Unfortunately, QGIS did not produce: /tmp/RtmpZxruXN/slope.tif

I'm using OSGeo-live 10.5 with QGIS 2.14.11 and GRASS GIS 7.2.0. My session info is:

> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

locale:
 [1] LC_CTYPE=en_US.UTF-8          LC_NUMERIC=C                  LC_TIME=en_US.UTF-8           LC_COLLATE=en_US.UTF-8       
 [5] LC_MONETARY=en_US.UTF-8       LC_MESSAGES=en_US.UTF-8       LC_PAPER=en_US.UTF-8          LC_NAME=en_US.UTF-8          
 [9] LC_ADDRESS=en_US.UTF-8        LC_TELEPHONE=en_US.UTF-8      LC_MEASUREMENT=en_US.UTF-8    LC_IDENTIFICATION=en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RQGIS_1.0.1

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.12    lattice_0.20-35 sf_0.5-2        grid_3.4.1      R6_2.2.2        DBI_0.7         magrittr_1.5    units_0.4-5    
 [9] stringi_1.1.3   rlang_0.1.1     raster_2.5-8    sp_1.2-5        reticulate_0.9  rgdal_1.2-8     tools_3.4.1     stringr_1.2.0  
[17] udunits2_0.13   readr_1.1.1     hms_0.3         parallel_3.4.1  compiler_3.4.1  tibble_1.3.3 

Thanks in advance for your help

• Leave old approach as backup for Windows when installed Python/Numpy is not sufficient for reticulate. Also, if someone has a running reticulate session we cannot overwrite the reticulate config -> R restarted needed. In this case, use the fall back option if the running reticulate config is not sufficient for QGIS
• No problems for Linux/Mac because we are using /usr/bin/python which should work in any case

• OTB
• GDAL
• TauDEM
• LiDAR

Since @eivindhammers has brought sf to RQGIS, we need also to support sf-objects in run_qgis:

https://github.com/jannes-m/RQGIS/blob/a7b5993ca4cf221778f30ccd0bee1e6171a884a8/R/processing.R#L820-L844

i.e. if somebody provides a sp-object, we should convert it into an sf-object. Equally, we should also use st_read for importing shapefiles (given that load_ouptput is specified in run_qgis), see:

https://github.com/jannes-m/RQGIS/blob/a7b5993ca4cf221778f30ccd0bee1e6171a884a8/R/processing.R#L930-L968

Both, saving the input and loading the output should go into its own helper_function.

Hey,

when I execute run_qgis() with your examples like
alg <- "grass7:r.slope.aspect"
data(dem, package = "RQGIS")
slope <- run_qgis(alg, elevation = dem, slope = "slope.asc", load_output = TRUE)
or
ger <- getData(name = "GADM", country = "DEU", level = 1)
params$INPUT_LAYER <- ger
params$OUTPUT_LAYER <- file.path(tempdir(), "ger_coords.shp")
out <- run_qgis(alg = "qgis:polygoncentroids", params = params, load_output = TRUE)

I get the following error:

Error in run_qgis(alg, elevation = dem, slope = "slope.asc", load_output = TRUE) :
Unable to execute algorithm
AVer\AppData\Local\Temp\Rtmps3f6yS/elevation.tif
or
Error in run_qgis(alg = "qgis:polygoncentroids", params = params, load_output = TRUE) :
Unable to execute algorithm
AVer\AppData\Local\Temp\Rtmps3f6yS/INPUT_LAYER.shp
, respectively.

It seems to me that the temp-path is not valid, because my computer name is rAVer and not AVer. After execution I also checked the temp directory, and the files exist!
When I use hard-coded paths instead (I saved the dem on a specific location, and give the exact location also of the "slope.asc"), it works.

I use Windows 10 Pro 64-bit, R version 3.4.0 (2017-04-21), RQGIS_1.0.0 and QGIS 2.18.7.

execute_cmds() has intern = FALSE as default, which is switched to TRUE when run on Darwin/Linux. However, in the actual calls to this function there are sometimes ifelse() statements that set intern = FALSE if system is Darwin. This seems both inconsistent and redundant. I guess it's enough to have the if/else in execute_cmds() definition?

https://github.com/jannes-m/RQGIS/blob/a7b5993ca4cf221778f30ccd0bee1e6171a884a8/R/helper_funs.R#L124

https://github.com/jannes-m/RQGIS/blob/a7b5993ca4cf221778f30ccd0bee1e6171a884a8/R/processing.R#L300

https://github.com/jannes-m/RQGIS/blob/a7b5993ca4cf221778f30ccd0bee1e6171a884a8/R/processing.R#L916

https://github.com/jannes-m/RQGIS/blob/a7b5993ca4cf221778f30ccd0bee1e6171a884a8/R/processing.R#L293

@eivindhammers

When running run_qgis() test for your sf implementation, I get the following error:

ERROR 4: `c(0, 0)list(list(c(0, 0, 1, 1, 0, 0, 1, 1, 0, 0)), list(c(2, 2, 3, 3, 2, 2, 3, 3, 2, 2)))' not recognized as a supported file format.

Wrong parameter value: c(0, 0)list(list(c(0, 0, 1, 1, 0, 0, 1, 1, 0, 0)), list(c(2, 2, 3, 3, 2, 2, 3, 3, 2, 2)))
Error in FUN(X[[i]], ...) : 
  Unfortunately, QGIS did not produce: /var/folders/5j/_1ts10x512sg_5q_3kzc_c1w0000gn/T//Rtmp55k8FL/coords.shp
In addition: Warning message:
In if (tmp == "RasterLayer") { :
  the condition has length > 1 and only the first element will be used

Haven't tried to debug yet - maybe you know what's going on and can fix it quickly?

There is one built-in qgis plugin called "Interpolation" that I would like to use. But I am unable to find it using find_algorithms. The result are attached below. I am new to RQGIS, so I was not sure if built-in plugins in QGIS are supported.

find_algorithms(search_term = "interpolation",  qgis_env = my_env)                          

 [1] "C:\\Users\\ZENANW~1\\AppData\\Local\\Temp\\RtmpYLOzwH"                                                                         
 [2] "r.fillnulls - Fills no-data areas in raster maps using spline interpolation.--->grass7:r.fillnulls"                            
 [3] "r.resamp.bspline - Performs bilinear or bicubic spline interpolation with Tykhonov regularization.--->grass7:r.resamp.bspline" 
 [4] "r.resamp.interp - Resamples raster map to a finer grid using interpolation.--->grass7:r.resamp.interp"                         
 [5] "r.surf.idw - Surface interpolation utility for raster layers.--->grass7:r.surf.idw"                                            
 [6] "v.surf.bspline - Bicubic or bilinear spline interpolation with Tykhonov regularization.--->grass7:v.surf.bspline"              
 [7] "v.surf.bspline.lambda - Bicubic or bilinear spline interpolation with Tykhonov regularization.--->grass7:v.surf.bspline.lambda"
 [8] "v.surf.bspline.sparse - Bicubic or bilinear spline interpolation with Tykhonov regularization.--->grass7:v.surf.bspline.sparse"
 [9] "v.surf.idw - Surface interpolation from vector point data by Inverse Distance Squared Weighting.--->grass7:v.surf.idw"         
[10] "Inverse distance weighted interpolation-------------->saga:inversedistanceweighted"                                            
[11] "Modified quadratic shepard interpolation------------->saga:modifiedquadraticshepard"                                           
[12] "Multilevel b-spline interpolation-------------------->saga:multilevelbsplineinterpolation"                                     
[13] "Multilevel b-spline interpolation (from raster)------>saga:multilevelbsplineinterpolationfromgrid"

Example: https://edzer.github.io/sfr/articles/sf1.html
R package pkgdown: https://github.com/hadley/pkgdown

Hi,

Thanks for the brilliant QGIS intro.
I followed your example in plotting the centroids in the ger spatial object. My final result though does not match your image with centroids in every region, but just one centroid.

When I run the run_qgis function I get the warining message:
running command 'batch_cmd.cmd' had status 1.
Maybe it is related.

Thanks for supporting,

Great work,

See #64 for more details

atype = "area" should work just as atype = "0"

I created an example on creating contours from a DEM raster (on windows) . This works see - see the R script below and screenshot. However I had to use readOGR (see line 54 in text file below) to assign the output contour shape file. This seems to be a bug to me (as it worked directly to another algorithm such as buffer e.g.). Can this be fixed ?

rqgis_contours_tz.txt

I get an error trying to retrieve parameters from QGIS. I run get_args_man("grass:v.clean") and get

C:\Users\eivinol\AppData\Local\Temp\RtmpwzBJdD
python: can't open file 'py_cmd.py': [Errno 2] No such file or directory
[1] 2
Warning message:
running command 'batch_cmd.cmd' had status 2

Now there may be something wrong with my installation, but I can't figure out what that would be.

It seems that calling o4w_env.bat changes the working directory to %OSGEO4W_ROOT%, and there is nothing that changes it back to the R temp directory before python py_cmd.py is called. I fixed this in my fork with a "set wd=%~dp0" before calling o4w_env.bat, and "pushd %wd%" after.

Replacing system("batch_cmd.cmd", intern = intern) with shell("batch_cmd.cmd") in `execute_cmds()' also works. Manually running the batch_cmd.cmd from cmd.exe also works.

Example when GRASS6 is missing:

qgis_session_info()

QSqlDatabase: QSQLITE driver not loaded
QSqlDatabase: available drivers: 
QSqlQuery::prepare: database not open
Warning 4: Failed to open /Applications/QGIS.app/Contents/Resources/python/plugins/processing/tests/data/points.shp, Permission denied.
Error in sys.excepthook:
Traceback (most recent call last):
  File "/Applications/QGIS.app/Contents/Resources/python/qgis/utils.py", line 196, in qgis_excepthook
    showException(type, value, tb, None, messagebar=True)
  File "/Applications/QGIS.app/Contents/Resources/python/qgis/utils.py", line 107, in showException
    open_stack_dialog(type, value, tb, msg)
  File "/Applications/QGIS.app/Contents/Resources/python/qgis/utils.py", line 142, in open_stack_dialog
    iface.messageBar().popWidget()
AttributeError: 'NoneType' object has no attribute 'messageBar'

Original exception was:
Traceback (most recent call last):
  File "py_cmd.py", line 26, in <module>
    g6 = GrassUtils.grassPath()
  File "/Applications/QGIS.app/Contents/Resources/python/plugins/processing/algs/grass/GrassUtils.py", line 110, in grassPath
    ProcessingConfig.setSettingValue(GrassUtils.GRASS_FOLDER, folder)
  File "/Applications/QGIS.app/Contents/Resources/python/plugins/processing/core/ProcessingConfig.py", line 216, in setSettingValue
    ProcessingConfig.settings[name].setValue(value)
  File "/Applications/QGIS.app/Contents/Resources/python/plugins/processing/core/ProcessingConfig.py", line 287, in setValue
    self.validator(value)
  File "/Applications/QGIS.app/Contents/Resources/python/plugins/processing/core/ProcessingConfig.py", line 271, in checkFileOrFolder
    raise ValueError(self.tr('Specified path does not exist:\n%s') % unicode(v))
ValueError: Specified path does not exist:
/Applications/GRASS-6.4.app/Contents/MacOS
Error in file(file, "rt") : cannot open the connection
In addition: Warning messages:
1: running command 'sh batch_cmd.sh' had status 1 
2: In file(file, "rt") :
  cannot open file '/var/folders/5j/_1ts10x512sg_5q_3kzc_c1w0000gn/T//Rtmp4tzNNy/out.csv': No such file or directory

What could be especially useful to have in the RQGIS is a functionality to load/display layers directly from R to QGIS. I think this is already possible via the PyQGIS package, so it should probably not be too much of effort to make functions to add raster / vector layers to an existing display. Just an idea for future enhancements.

At the moment I usually use plotKML and/or the leaflet package to display maps directly from R.

I am trying to perform the following call and it does not work -

out <- run_qgis(alg = "grass7:r.viewshed",
params = params,
load_output = params$OUTPUT_LAYER,
qgis_env = my_env)

Just after sending the call I get the following Warnings/Error Messages -

QSqlDatabase: QSQLITE driver not loaded
QSqlDatabase: available drivers: 
QSqlQuery::prepare: database not open

Is this an issue or should the call work and produce a result anyway?
Because there seems to be some process running but my out variable is always null.

Thanks for your help.
Cheers

I'm attempting to intersect census block groups with a city polygon layer using the intersection algo.

The call fails due to the inability to create a raster file.

find_algorithms(search_term = "intersection")
get_usage(alg = "qgis:intersection")

params <- get_args_man(alg = "qgis:intersection")

params$INPUT <- mult_cbg
params$INPUT2 <- pdx
params$OUTPUT <- "pdx_bg_2015.geojson"

out <- run_qgis(alg = "qgis:intersection",
                params = params,
                load_output = TRUE)```

The output error:

```$OUTPUT
[1] "C:\\Users\\elmue\\AppData\\Local\\Temp\\Rtmp0AW5LT/pdx_bg_2015.geojson"


Error in .local(.Object, ...) : 

Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer",  : 
  Cannot create a RasterLayer object from this file.```

Both input files are simple features from the census bureau and have the same projection. 

Option 1: Arg params (list) + arg ... (single function arguments)

Option 2: Arg ... which can take a list OR single function arguments

We should rewrite execute_cmds() so that it can also be used with qgis_session_info(), open_help() and get_args_man(). This would delete rendundancy and make the code more consistent. In this case, we could have had rewritten the python py_cmd.py to paste("python", file.path(tmp_dir), "py_cmd.py")) in execute_cmds()

I use RQGIS to perform zonal statistics with SAGA, via QGIS (2.14.10 on Windows 7 via OSGEO4W). However, the output returns the beginning of the path as field name of the statistic (C.Userseivi in the example below). If I'm not doing something wrong, I think processing.runalg('saga:gridstatisticsforpolygons') (and probably many other algs) automatically names the output field(s) like "pathtoraster [fun]", which would be "C:/Users/[...]temp/TEMPDIR/GRID.asc [sum]" in the example below.

Would it be an idea to optionally specify the output field name in run_qgis? That would be setting the name of the tempfile to something other than "GRID.asc", and then not provide the full path to that file in "py_cmd.py", instead relying on setting the working directory.

qgis <- set_env(root = "C:/OSGeo4W64/")
alg <- "saga:gridstatisticsforpolygons"
params <- get_args_man(alg, qgis_env = qgis)
params$GRIDS <- pop.afr # RasterLayer
params$POLYGONS <- grid.sp # SPDF
params$SUM <- "TRUE"
exclude <- c("COUNT", "MIN", "MAX", "RANGE", "MEAN", "VAR", "STDDEV")
params[exclude] <- "FALSE"
params$RESULT <- "output.shp"
grid.pop <- run_qgis(alg, params = params, qgis_env = qgis, 
                     load_output = params$RESULT)
# grid.pop now includes a column "C.Userseivi"

Hi Jannes,

just came across this error message in run_qgis while running the README.md example:

if (grepl("no features found", attr(test, "condition"))) {
   stop("The output-file ", fname, " is empty, i.e. it has no features.")
}

Error in if (grepl("no features found", attr(test, "condition"))) { : 
  Argument has length 0

If I comment these lines out, run_qgis executes properly.
I debugged the code up to these lines and found not attribute called "condition" in the tmp file test. That causes the length 0 error.

What interesting: Debugging in Windows causes the same error at this point. However, running run_qgis as a function does NOT produce the error in Windows, only for Mac.

Can we rewrite this check to avoid such problems? For now, I just commented the lines out.

QGIS 2.14.7 (Kyngchaos) Mac

bin\o4w_env.bat will take care of setting up all the paths that you need. Should be able to use it in the place of the stuff done at https://github.com/jannes-m/RQGIS/blob/master/R/helper_funs.R#L32

    cmd <- 
      c("@echo off",
        # defining a root variable
        paste0("SET OSGEO4W_ROOT=", qgis_env$root),
        # calling batch files from with a batchfile
        "call \"%OSGEO4W_ROOT%\"\\bin\\o4w_env.bat",
        paste0("call \"%OSGEO4W_ROOT%\"\\apps\\grass\\", grass, 
               "\\etc\\env.bat"),
        "@echo off",
        # adding QGIS and GRASS to PATH
        "path %PATH%;%OSGEO4W_ROOT%\\apps\\qgis\\bin",
        paste0("path %PATH%;%OSGEO4W_ROOT%\\apps\\grass\\", grass,
               "\\lib"),
        # setting a PYTHONPATH variable
        "set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\\apps\\qgis\\python;",
        # adding a few more python paths to PYTHONPATH
        paste0("set PYTHONPATH=%PYTHONPATH%;", 
               "%OSGEO4W_ROOT%\\apps\\Python27\\Lib\\site-packages"),
        # defining QGIS prefix path (i.e. without bin)
        "set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\\apps\\qgis")

Hi,

I've manage to install everything I think I needed to install rqgis, but when I execute qgis_session_info() R return me this:

> qgis_session_info()
Checking for homebrew osgeo4mac installation on your system. 

Error in py_run_string_impl(code, local, convert) : 
  NameError: name 'RQGIS' is not defined

Detailed traceback: 
  File "<string>", line 1, in <module>
> set_env()
Checking for homebrew osgeo4mac installation on your system. 

Error in if (no_homebrew[1] == TRUE) { : 
  missing value where TRUE/FALSE needed
> 

I think something is not correctly configured, but I think figure out what.

Can you give me a hand here?

thanks!

In helper_funs.R:

When QGIS is installed via homebrew from osgeo4mac, only a symlink exists in /Applications.

Hence, the second part of the following export statement in batch_cmd.sh points to nowhere:

export DYLD_LIBRARY_PATH=/usr/local/Cellar//qgis2/2.18.3/QGIS.app/Contents/MacOS/lib/:/Applications/QGIS.app/Contents/Frameworks/

and should be changed to /usr/local/Cellar//qgis2/2.18.3/QGIS.app/Contents/Frameworks/

For now, I do not know the implications of this empty export link but it might cause complex error messages related to missing imports of QGIS libs.

Additional note: A redundant slash is introduced between "Cellar" and "qgis2".