jeroen / curl Goto Github PK
View Code? Open in Web Editor NEWA Modern and Flexible Web Client for R
Home Page: https://jeroen.r-universe.dev/curl
License: Other
A Modern and Flexible Web Client for R
Home Page: https://jeroen.r-universe.dev/curl
License: Other
The curl
package for snow-leopard has not been updated since 0.9.1. Not sure why, it passes check on my snow-leopard server. @s-u
Hello,
I'm interested in using your rjade package, but had an issue installing the curl package on which it depends. Here's the output from my install attempt:
> install.packages("rjade")
also installing the dependencies 'curl', 'V8'
trying URL 'http://cran.mtu.edu/src/contrib/curl_0.9.1.tar.gz'
Content type 'application/x-gzip' length 242408 bytes (236 KB)
opened URL
==================================================
downloaded 236 KB
trying URL 'http://cran.mtu.edu/src/contrib/V8_0.6.tar.gz'
Content type 'application/x-gzip' length 110013 bytes (107 KB)
opened URL
==================================================
downloaded 107 KB
trying URL 'http://cran.mtu.edu/src/contrib/rjade_0.1.tar.gz'
Content type 'application/x-gzip' length 44597 bytes (43 KB)
opened URL
==================================================
downloaded 43 KB
* installing *source* package 'curl' ...
** package 'curl' successfully unpacked and MD5 sums checked
Found libcurl in /usr/include/curl/curl.h
** libs
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c callbacks.c -o callbacks.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c curl.c -o curl.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c download.c -o download.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c escape.c -o escape.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c fetch.c -o fetch.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c form.c -o form.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c getdate.c -o getdate.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c handle.c -o handle.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c ieproxy.c -o ieproxy.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c init.c -o init.o
/broad/software/free/Linux/redhat_5_x86_64/pkgs/gcc_4.8.2/bin/gcc -std=gnu99 -I/xchip/cogs/tools/opt/R-3.1.3/lib64/R/include -DNDEBUG -I/usr/include -I/usr/local/include -fpic -g -O2 -c options.c -o options.o
In file included from options.c:3:0:
options.h:7:27: error: 'CURLOPT_SOURCE_USERPWD' undeclared here (not in a function)
#define CINIT(a,b,c) {#a, CURLOPT_##a}
^
option_table.h:114:3: note: in expansion of macro 'CINIT'
CINIT(SOURCE_USERPWD, OBJECTPOINT, 123),
^
options.h:7:27: error: 'CURLOPT_SOURCE_PREQUOTE' undeclared here (not in a function)
#define CINIT(a,b,c) {#a, CURLOPT_##a}
^
option_table.h:115:3: note: in expansion of macro 'CINIT'
CINIT(SOURCE_PREQUOTE, OBJECTPOINT, 127),
^
options.h:7:27: error: 'CURLOPT_SOURCE_POSTQUOTE' undeclared here (not in a function)
#define CINIT(a,b,c) {#a, CURLOPT_##a}
^
option_table.h:116:3: note: in expansion of macro 'CINIT'
CINIT(SOURCE_POSTQUOTE, OBJECTPOINT, 128),
^
options.h:7:27: error: 'CURLOPT_SOURCE_URL' undeclared here (not in a function)
#define CINIT(a,b,c) {#a, CURLOPT_##a}
^
option_table.h:120:3: note: in expansion of macro 'CINIT'
CINIT(SOURCE_URL, OBJECTPOINT, 132),
^
options.h:7:27: error: 'CURLOPT_SOURCE_QUOTE' undeclared here (not in a function)
#define CINIT(a,b,c) {#a, CURLOPT_##a}
^
option_table.h:121:3: note: in expansion of macro 'CINIT'
CINIT(SOURCE_QUOTE, OBJECTPOINT, 133),
^
make: *** [options.o] Error 1
ERROR: compilation failed for package 'curl'
* removing '/xchip/cogs/tools/opt/R-3.1.3/lib64/R/library/curl'
ERROR: dependency 'curl' is not available for package 'V8'
* removing '/xchip/cogs/tools/opt/R-3.1.3/lib64/R/library/V8'
ERROR: dependency 'V8' is not available for package 'rjade'
* removing '/xchip/cogs/tools/opt/R-3.1.3/lib64/R/library/rjade'
The downloaded source packages are in
'/tmp/RtmpmVgwR6/downloaded_packages'
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
Warning messages:
1: In install.packages("rjade") :
installation of package 'curl' had non-zero exit status
2: In install.packages("rjade") :
installation of package 'V8' had non-zero exit status
3: In install.packages("rjade") :
installation of package 'rjade' had non-zero exit status
Also, some info about the version of curl on the system:
$ curl --version
curl 7.23.0 (x86_64-unknown-linux-gnu) libcurl/7.23.0 OpenSSL/0.9.8b zlib/1.2.7 libidn/0.6.5 libssh2/1.3.0
Please let me know if there's any other info I can provide. Thanks a lot!
Looks like the garbage collector is overwriting memory. Bit hard to reproduce, but in rstudio this usually works:
library(curl)
con <- curl("http://localhost/test.txt")
gc(F)
debug(readLines)
readLines(con)
I've got R and curl installed via MacPorts on OS X, and when I try to install curl (even after checking out the latest from github), it keeps failing at the very end of the install.
$ R CMD INSTALL .
WARNING: ignoring environment value of R_HOME
* installing to library ‘/Users/steevmi1/Library/R/3.2/library’
* installing *source* package ‘curl’ ...
Found pkg-config cflags and libs!
Using PKG_CFLAGS=-I/opt/local/include
Using PKG_LIBS=-L/opt/local/lib -lcurl
** libs
make: Nothing to be done for `all'.
installing to /Users/steevmi1/Library/R/3.2/library/curl/libs
** R
** data
*** moving datasets to lazyload DB
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/Users/steevmi1/Library/R/3.2/library/curl/libs/curl.so':
dlopen(/Users/steevmi1/Library/R/3.2/library/curl/libs/curl.so, 6): Symbol not found: _SSLv2_client_method
Referenced from: /opt/local/lib/libcurl.4.dylib
Expected in: /opt/local/lib/libssl.1.0.0.dylib
in /opt/local/lib/libcurl.4.dylib
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Users/steevmi1/Library/R/3.2/library/curl’
* restoring previous ‘/Users/steevmi1/Library/R/3.2/library/curl’
Which returns same information as curl_perform()
, plus a connection to listen to
I have a very strange use case that I would like to submit for consideration because I get different results with curl_download
than I do with curl
from the shell. Please feel free to close the issue straight away if the problem is not related enough to curl
.
I'm starting with redirect URLs like this one:
http://www.senato.it/loc/link.asp?tipodoc=CAM.DEP&leg=17&id=30350
By using httr
, I get the URL it redirects to:
http://www.camera.it/leg17/29?tipoAttivita=&tipoVisAtt=&tipoPersona=&shadow_deputato=30350&idLegislatura=17
Now, if I navigate to this page, I sometimes get a Web page for a male Italian MP, but I also sometimes get an empty page with content type application/json
that simply says "format json not implemented yet". Here's the output with httr
:
> GET("http://www.camera.it/leg17/29?tipoAttivita=&tipoVisAtt=&tipoPersona=&shadow_deputato=30350&idLegislatura=17")
Response [http://www.camera.it/leg17/29?tipoAttivita=&tipoVisAtt=&tipoPersona=&shadow_deputato=30350&idLegislatura=17]
Date: 2015-08-04 12:22
Status: 200
Content-Type: application/json; charset=utf-8
Size: 31 B
> GET("http://www.camera.it/leg17/29?tipoAttivita=&tipoVisAtt=&tipoPersona=&shadow_deputato=30350&idLegislatura=17") %>% content
Error in parse_string(txt, bigint_as_char) :
lexical error: invalid string in json text.
format json not implemented yet
(right here) ------^
So I turn to curl
, thinking that it might save me from that very strange issue. Why? Because every time that I try curl -v
on that last URL in the shell, I get HTML instead of the "JSON" thing:
curl -v http://www.camera.it/leg17/29?tipoAttivita=&tipoVisAtt=&tipoPersona=&shadow_deputato=30350&idLegislatura=17
My issue is, when I use curl_download
, it does not get the HTML like curl
in-the-shell does: it gets the wrong "JSON" thing.
Question, therefore, is: how do I emulate curl
in-the-shell's settings with curl
in R, in order to get HTML instead of "JSON" through these awful links?
The output of curl -v
for the above example is copied in this Gist. My curl
in-the-shell version is copied below.
curl 7.30.0 (x86_64-apple-darwin13.0) libcurl/7.30.0 SecureTransport zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM NTLM_WB SSL libz
I am currently using R version 3.2.1, and when I updated the "curl" package to version 0.9.2, I started getting following message after I run script that uses curl package (there is no such message with scripts that don't use curl):
"100% downloaded: rsession.exe-System Error"
The program can't start because MSVC80.dll is missing from your computer.
Try installing the program to fix this problem.
There are OK and cross (as in multiplication sign) options. If I chose one of them, program continues without any error. It's, however, annoying that I get this message whenever I run the script that uses functions from curl package.
The description file of curl package has following:
Version: 0.9.2
Depends: R (≥ 2.10.0)
But, on the closer look, I see this: "package ‘curl’ was built under R version 3.2.2". So, is this saying that I have to install new version of R (3.2.2) to run without any warning message? What if I don't want to update R version (now), but I do want to use new version of "curl" package (0.9.2)?
Note, that I have tried both with baseR and RStudio. Both gives the warning message when running scripts that use "curl" package. I would appreciate if you could take a closer look into this issue. I am using Windows 8.
This relates to r-lib/httr#352.
If I understand correctly httr::write_stream
is mostly offhanded to curl::curl_fetch_stream
which does not take into account the value that is returned by the callback function in such way as defined in the httr
docs.
As a reference the docs from ?httr::write_stream
:
write_stream(f)
Arguments
f: Callback function. It should have a single argument, a raw vector containing the bytes recieved from the server. This will usually be 16k or less. It should return the length of bytes processed - if this is less than the input length, the function will terminate.
If the callback function's return value would actually be checked to be the length of bytes processed, which it doesn't ATM, this would allow for terminating the stream when required.
This quick hack:
curl_fetch_stream <- function(url, fun, handle = new_handle()){
con <- .Call(R_curl_connection, url, "rb", handle, FALSE)
on.exit(close(con))
while(length(bin <- readBin(con, raw(), 8192L))){
fun_eval<-fun(bin)
if(!is.numeric(fun_eval) || fun_eval<length(bin)) break
fun_eval
}
handle_response_data(handle)
}
Would allow e.g. for:
foo<-1
bar<-5
GET("https://jeroenooms.github.io/data/diamonds.json",
write_stream(function(x) {
print(length(x))
foo<<-foo+1
if(foo > bar) return(0)
length(x)
})
)
Or maybe I'm taking this too far and there is a much easier way to terminate the handler when wanted? 😁
It would be nice to have a bit more documentation here (including example of uploading a file to a form)
I am using curl()
following usage I see in the official examples, like so:
x <- readLines(curl(SOME_URL)
but inside a loop over SOME_URL
. I then got lots of warnings about closing unused connections. Am I supposed to be more careful and always store then close the connection? Does it even matter?
con <- curl(SOME_URL)
x <- readLines(con)
close(con)
Would be useful to have variant of handle_setopt()
that lets you pass in a list of options
Hi, R 3.2.2 on Ubuntu 15.04 64 bit. Currently curl 0.92 is installed.
* installing *source* package ‘curl’ ...
** package ‘curl’ successfully unpacked and MD5 sums checked
Found pkg-config cflags and libs!
Using PKG_CFLAGS=
Using PKG_LIBS=-lcurl
** libs
"/usr/lib/R/bin/Rscript" "../tools/options.R"
Error in file(filename, "r", encoding = encoding) :
cannot open the connection
Calls: source -> file
In addition: Warning message:
In file(filename, "r", encoding = encoding) :
cannot open file '../tools/option_table.txt': No such file or directory
Execution halted
Makevars:7: recipe for target 'options' failed
make: *** [options] Error 1
ERROR: compilation failed for package ‘curl’
* removing ‘/usr/lib/R/library/curl’
* restoring previous ‘/usr/lib/R/library/curl’
The downloaded source packages are in
‘/tmp/RtmpOpqUvf/downloaded_packages’
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
Warning message:
In install.packages(update[instlib == l, "Package"], l, contriburl = contriburl, :
installation of package ‘curl’ had non-zero exit status
Read proxy settings in windows to mimic setInternet2
:
http://stackoverflow.com/questions/202547/how-do-i-find-out-the-browsers-proxy-settings
In the event of an HTTP error (particularly those in the 4XX series), it's useful to get error messages from the server to know what went wrong. In RCurl
this is accomplished by RCurl::getCurlInfo(handle)
after the error has occurred. Is there an equivalent in curl
?
From r-lib/httr#271:
This is not an issue, more of a question that I wanted to pose to the community. If there is already a method for achieving this, I apologize for the repeat and would appreciate being pointed in the right direction.
I use httr extensively and often find myself in situations where the needed aggregation of data requires hundreds of thousands of REST calls. This becomes performance limiting in R because it seems like every http call made using httr is a blocking call.
Is there any plan or path forward for enabling asynchronous io within curl similar to what exists in Python via aiohttp or Scala via Akka-http?
Dear Jeroen,
I am using curl for https://github.com/soodoku/clarifai
I am trying to pass multiple urls to the API, like so:
curl -H "Authorization: Bearer <access_token>" --data-urlencode "url=http://www.clarifai.com/img/metro-north.jpg" --data-urlencode "url=http://www.clarifai.com/img/metro-north.jpg" https://api.clarifai.com/v1/tag/
I can do that with:
handle_setform(h, url = "url1", url="url2")
But the number of urls being uploaded depends on user input. I am having trouble with building that.
I know I can't do: c("url = string1, url = string2") etc. There may be a do.call version of this but haven't found a way forward.
Any suggestions? Thanks.
When pointed at a libcurl compiled without libz support enabled, the package will install succesfully, but segfault when collecting version info here https://github.com/jeroenooms/curl/blob/master/src/version.c#L12 because the libz_version field of the struct is NULL.
If libz support is mandatory then the package configuration script should fail if the libcurl it finds doesn't have it.
I thought I would just pass you a note for homebrew/OSX users. In homebrew the curl headers are not put in any default search path to avoid conflicts with any system curl (unfortunately OS X does not ship with any headers for curl 😞 ).
So you might want to consider just adding an else if to search the homebrew location,
elif [ -r /usr/local/opt/curl/include/curl/curl.h ]; then CURL_INCLUDES="/usr/local/opt/curl/include"
Anyway, thanks for the package. If you want I could always make a pull request, but I probably won't get to that until Sunday night (EST).
I have the latest version of curl(0.9.4) and each time I am trying to load the library. It throws the following:
Error in .loadNamespaceImpl(package, path, keep.source, partial): error executing useDynLib for dynamic library 'curl' from package 'curl' loaded from C:/abc/xyz/library:
Error in library.dynam(chname = chname, package = package, lib.loc = ...: Foreign binary curl could not be loaded
Any idea on what could be causing it?
I use them in three places:
readfunction
)progressfunction
)debugfunction
)Implement R_CheckUserInterrupt();
.
Check that when the error
function is called, stuff is still cleaned up correctly.
h <- new_handle()
handle_setopt(h,
httpheader = c(Accept = "application/json, text/xml, application/xml, */*")
)
x <- curl_perform("http://google.com", h)
I'm not sure if you want to make this work or not, but it should at least throw an error.
Since you want to be able to check status etc.
Not sure what's going wrong here:
library(curl)
h <- new_handle()
handle_setopt(h, customrequest = "HEAD")
curl_fetch_memory("http://google.com", handle = h)
If I cmd + c in a debugger, the backtrace is
(lldb) bt 10
* thread #1: tid = 0x1457395, 0x00007fff838785c2 libsystem_kernel.dylib`poll + 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
* frame #0: 0x00007fff838785c2 libsystem_kernel.dylib`poll + 10
frame #1: 0x00007fff8b538378 libcurl.4.dylib`Curl_poll + 140
frame #2: 0x00007fff8b533791 libcurl.4.dylib`curl_multi_wait + 525
frame #3: 0x00007fff8b52ccba libcurl.4.dylib`curl_easy_perform + 245
frame #4: 0x0000000102fe9a79 curl.so`R_curl_fetch_memory + 169
frame #5: 0x000000010007b2c7 libR.dylib`do_dotcall(call=0x0000000106d20400, op=<unavailable>, args=<unavailable>, env=<unavailable>) + 327 at dotcode.c:1251
frame #6: 0x00000001000a546c libR.dylib`Rf_eval(e=0x0000000106d20400, rho=0x00000001038cd4b0) + 988 at eval.c:655
It doesn't seem to time out (I waited well over 10s)
While I appreciate the urge to keep this library very lightweight, curl/easy.h includes easy_escape(), which does what it says on the tin - really simple URL encoding. If the idea is for this to act as a replacement for RCurl in the context of httr and similar libraries, you might want to look at making this available (or including it automatically in the curl() function): escaping will be needed, and R's URLencode has many limitations that make it a pain.
See this file where I just switched to curl::curl()
because the site switched from http to https and url()
is no longer of use.
I can close(con)
in the first routine following readLines()
; in all the other ones following read.table()
I get an error. That seems odd. Is this a bug?
i get this error when i try to run gs_key function from googlesheets packages
temp= gs_key(ggdoc_2)
Authentication will be used.
Error in curl::curl_fetch_memory(url, handle = handle) :
Timeout was reached
How to slove it?
Seems that httr::progress()
now throws warnings when used on new httr
1.0 version - e.g., a discussion here traitecoevo/taxonlookup#11 (comment)
This is the warning message after using progress()
Warning messages:
1: In curl::curl_fetch_disk(url, x$path, handle = handle) :
progress callback must return boolean
...
The following functions result in segmentation faults:
curl::handle_setform(list(dummy=1))
curl::handle_setopt(list(dummy=1))
curl::handle_cookies(list(dummy=1))
I discovered this because I was getting a new error in curl::handle_setopt: "Unknown option: http.version", and in the process I forgot to feed handle_setopt the handle. I do not expect the code to work, but do expect it to error more gracefully.
I have a web service that has some clients with IPv6-only connectivity that need to use httr (and hence curl). On Mac we have had this work well, but we're were surprised to find on Windows it fails.
The error message is "unable to resolve host". While we ultimately expect DNS with AAAA record to be used, testing with IP is a good step:
con= curl(url="http://[2001:468:c80:a202:0:b074:0:c082]/~cgaylord/x.txt")
readLines(con)
gives "cannot resolve host [2001"
(I will leave that file there for a while so others can test.)
Results otherwise are variable, but we have had success with Mac at least.
My thought is: is this a deficiency of a statically linked libcurl that does not have proper IPv6 functionality? I'm happy to drop another library in, if there's a suggestion of how best to do this.
Thanks!
> install.packages("curl")
Installing package into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
trying URL 'http://cran.rstudio.com/src/contrib/curl_0.8.tar.gz'
Content type 'application/x-gzip' length 241067 bytes (235 KB)
==================================================
downloaded 235 KB
* installing *source* package ‘curl’ ...
** package ‘curl’ successfully unpacked and MD5 sums checked
Found libcurl in /usr/include/curl/curl.h
** libs
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/include -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c callbacks.c -o callbacks.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/include -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c curl.c -o curl.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/include -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c download.c -o download.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/include -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c escape.c -o escape.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/include -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c fetch.c -o fetch.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/include -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c form.c -o form.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/include -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c getdate.c -o getdate.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/include -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c handle.c -o handle.o
handle.c: In function ‘set_handle_defaults’:
handle.c:75: error: ‘CURLOPT_CONNECTTIMEOUT_MS’ undeclared (first use in this function)
handle.c:75: error: (Each undeclared identifier is reported only once
handle.c:75: error: for each function it appears in.)
make: *** [handle.o] Error 1
ERROR: compilation failed for package ‘curl’
* removing ‘/usr/lib64/R/library/curl’
The downloaded source packages are in
‘/tmp/Rtmp441QyQ/downloaded_packages’
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
Warning message:
In install.packages("curl") :
installation of package ‘curl’ had non-zero exit status
>
$ yum info curl-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.usc.edu
* epel: fedora-epel.mirror.lstn.net
* extras: repos.dfw.quadranet.com
* updates: mirrors.adams.net
Installed Packages
Name : curl-devel
Arch : i386
Version : 7.15.5
Release : 17.el5_9
Size : 643 k
Repo : installed
Summary : Files needed for building applications with libcurl.
URL : http://curl.haxx.se/
License : MIT
Description: cURL is a tool for getting files from FTP, HTTP, Gopher, Telnet, and
: Dict servers, using any of the supported protocols. The curl-devel
: package includes files needed for developing applications which can
: use cURL's capabilities internally.
Name : curl-devel
Arch : x86_64
Version : 7.15.5
Release : 17.el5_9
Size : 751 k
Repo : installed
Summary : Files needed for building applications with libcurl.
URL : http://curl.haxx.se/
License : MIT
Description: cURL is a tool for getting files from FTP, HTTP, Gopher, Telnet, and
: Dict servers, using any of the supported protocols. The curl-devel
: package includes files needed for developing applications which can
: use cURL's capabilities internally.
Let me know if I can provide any more info!
I am trying to install the curl package but keep running into the following error. I've run install.packages("curl")
in the R console and in RStudio and received this error both times. Running R CMD INSTALL curl_0.9.6.tar.gz
also returns the error.
I'm on a Mac with El Capitan and using a Homebrew-installed version of R. Both Homebrew and the R package are up to date. I've installed R on Homebrew with the default settings and with the --with-openblas
flag, both resulting in the same installation error when running install.packages("curl")
.
Some more information, with the error at the bottom:
brew info r
homebrew/science/r: stable 3.2.4 (bottled), HEAD
Software environment for statistical computing
http://www.r-project.org/
/usr/local/Cellar/r/3.2.4_1 (2,177 files, 54M) *
Built from source with: --with-openblas
From: https://github.com/Homebrew/homebrew-science/blob/master/r.rb
r --version
R version 3.2.4 Revised (2016-03-16 r70336) -- "Very Secure Dishes"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin15.4.0 (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
http://www.gnu.org/licenses/.
install.packages("curl")
Installing package into ‘/Users/tom/Library/R/3.2/library’
(as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session ---
HTTPS CRAN mirror
1: 0-Cloud [https] 2: Austria [https]
3: Belgium (Ghent) [https] 4: Chile [https]
5: China (Beijing 4) [https] 6: Colombia (Cali) [https]
7: France (Lyon 1) [https] 8: France (Lyon 2) [https]
9: France (Paris 2) [https] 10: Germany (Münster) [https]
11: Iceland [https] 12: Italy (Padua) [https]
13: Japan (Tokyo) [https] 14: Mexico (Mexico City) [https]
15: New Zealand [https] 16: Russia (Moscow) [https]
17: Spain (A Coruña) [https] 18: Spain (Madrid) [https]
19: Switzerland [https] 20: UK (Bristol) [https]
21: UK (Cambridge) [https] 22: USA (CA 1) [https]
23: USA (KS) [https] 24: USA (MI 1) [https]
25: USA (TN) [https] 26: USA (TX) [https]
27: USA (WA) [https] 28: (HTTP mirrors)
Selection: 26
trying URL 'https://cran.revolutionanalytics.com/src/contrib/curl_0.9.6.tar.gz'
Content type 'application/octet-stream' length 262545 bytes (256 KB)
==================================================
downloaded 256 KB
* installing *source* package ‘curl’ ...
** package ‘curl’ successfully unpacked and MD5 sums checked
Found pkg-config cflags and libs!
Using PKG_CFLAGS=
Using PKG_LIBS=-lcurl
** libs
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c callbacks.c -o callbacks.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c curl.c -o curl.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c download.c -o download.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c escape.c -o escape.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c fetch.c -o fetch.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c form.c -o form.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c getdate.c -o getdate.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c handle.c -o handle.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c ieproxy.c -o ieproxy.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c init.c -o init.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c interrupt.c -o interrupt.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c nslookup.c -o nslookup.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c utils.c -o utils.o
clang -I/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/include -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -fPIC -g -O2 -c version.c -o version.o
clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/usr/local/Cellar/r/3.2.4_1/R.framework/Resources/lib -L/usr/local/opt/openblas/lib -L/usr/local/opt/gettext/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/usr/local/lib -o curl.so callbacks.o curl.o download.o escape.o fetch.o form.o getdate.o handle.o ieproxy.o init.o interrupt.o nslookup.o utils.o version.o -lcurl -F/usr/local/Cellar/r/3.2.4_1/R.framework/.. -framework R -lintl -Wl,-framework -Wl,CoreFoundation
installing to /Users/tom/Library/R/3.2/library/curl/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** preparing package for lazy loading
Error in source("tools/option_table.txt", env) :
tools/option_table.txt:1:1: unexpected input
1:
^
Error : unable to load R code in package ‘curl’
ERROR: lazy loading failed for package ‘curl’
* removing ‘/Users/tom/Library/R/3.2/library/curl’
The downloaded source packages are in
‘/private/var/folders/v1/6x2gz2jj1257syfr2_p05nkw0000gn/T/RtmpOf07ZT/downloaded_packages’
Warning message:
In install.packages("curl") :
installation of package ‘curl’ had non-zero exit status
Any ideas why this might be happening and how I can fix it? Thanks.
@jeroenooms you added the anyNA()
function in commit ff76a61 but you don't require R >= 3.1.0 in the DESCRIPTION file. I get an error that anyNA()
is not found when using R < 3.1.0. Either you do the sensible thing and just use any(is.na(keys))
or you bump the minimum required version of R to 3.1.0.
Curl has hundreds of C symbols that identify option names or possible values for options names. The tricky part is that the available symbols depend on the version of libcurl. The libcurl author has included a plain-text table that lists which symbols were introduced or deprecated in which version. This script converts this table into a dataframe and looks up the actual integer value behind each symbol so we can set it at runtime.
Currently the table is not symbol exported: curl:::curl_symbols
. I am not sure if this should be exported, or if we should only test internally in the curl if a symbol is available.
Why do you turn it off by default? It seems like a really really bad idea.
This is fine:
repeat print(length(readLines(curl("https://code.jquery.com/jquery-2.1.3.js"))))
But this sometimes gives a memory error:
repeat print(length(readLines(curl("https://raw.githubusercontent.com/isaacs/sax-js/master/lib/sax.js"))))
After a while:
R(2199,0x7fff73c79300) malloc: *** error for object 0x7f8fbfb097c0: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
The problem seems to happen in curl_multi_perform
, but before the callback function is called. It seems like this was introduced in curl 0.3, I'm not seeing it in curl 0.2.
Any way to add easy switch for using curl over the TOR network?
in shell you can run curl over tor easily:
sudo apt-get install tor
curl icanhazip.com
usewithtor curl icanhazip # some warnings on the way but the different ip achieved
or any particular place in the code where I would need to look at?
Our shared cluster uses environment modules to install most programs and shared libraries to a non-standard location, and add them to the path as needed. I can tell the package configuration where curl.h is by setting the CURL_INCLUDES environment variable (in my case =/sw/curl/include), but the install eventually fails:
gcc -std=gnu99 -shared -L/sw/R-3.2.1/lib -L/usr/local/lib64 -o curl.so callbacks.o curl.o download.o escape.o fetch.o form.o getdate.o handle.o ieproxy.o init.o options.o utils.o version.o -lcurl -L/sw/R-3.2.1/lib -lR
/sw/binutils-2.25/bin/ld: cannot find -lcurl
collect2: ld returned 1 exit status
make: *** [curl.so] Error 1
ERROR: compilation failed for package ‘curl’
HI, I was trying to install R package 'culr' on a RHEL machine. I got the following errors.
"* installing source package âcurlâ ...
** package âcurlâ successfully unpacked and MD5 sums checked
Found pkg-config cflags and libs!
Using PKG_CFLAGS=-I/nz/export/ae/appsysroot/host64/include
Using PKG_LIBS=-L/nz/export/ae/appsysroot/host64/lib64 -lcurl
** libs
"/nz/export/ae/languages/r/3.1/host64/lib64/R/bin/Rscript" "../tools/options.R"
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c callbacks.c -o callbacks.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c curl.c -o curl.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c download.c -o download.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c escape.c -o escape.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c fetch.c -o fetch.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c form.c -o form.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c getdate.c -o getdate.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c handle.c -o handle.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c ieproxy.c -o ieproxy.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c init.c -o init.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c utils.c -o utils.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -I/nz/export/ae/languages/r/3.1/host64/lib64/R/include -DNDEBUG -I/nz/export/ae/appsysroot/host64/include -m64 -I/nz/export/ae/sysroot/host64/include -fpic -m64 -O1 -c version.c -o version.o
/nz/export/ae/sysroot/host64/bin/gcc -std=gnu99 -shared -m64 -Bdirect,--hash-style=both,-Wl,-O1 -L/nz/export/ae/core/thirdParty/intel_mkl/current/lib/em64t -L/nz/export/ae/sysroot/host64/lib64 -L/nz/export/ae/sysroot/host64/usr/lib64 -L/nz/export/ae/sysroot/host64/x86_64-rhel5-linux-gnu/lib64 -Wl,-rpath=/nz/export/ae/core/thirdParty/intel_mkl/current/lib/em64t:/nz/export/ae/sysroot/host64/lib64:/nz/export/ae/sysroot/host64/usr/lib64:/nz/export/ae/sysroot/host64/x86_64-rhel5-linux-gnu/lib64 -o curl.so callbacks.o curl.o download.o escape.o fetch.o form.o getdate.o handle.o ieproxy.o init.o utils.o version.o -L/nz/export/ae/appsysroot/host64/lib64 -lcurl -L/nz/export/ae/languages/r/3.1/host64/lib64/R/lib -lR
installing to /nz/export/ae/languages/r/3.1/host64/lib64/R/library/curl/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/nz/export/ae/languages/r/3.1/host64/lib64/R/library/curl/libs/curl.so':
libcurl.so.4: cannot open shared object file: No such file or directory
Error: loading failed
Execution halted
ERROR: loading failed
After this, I manually compiled the curl.so and ran ldd on it. The following is the result:
[nz@umabofanzd05h1 src]$ ldd curl.so
linux-vdso.so.1 => (0x00007fff1730c000)
libcurl.so.4 => not found
libR.so => not found
libc.so.6 => /nz/export/ae/sysroot/host64/lib64/libc.so.6 (0x00002b7b6769d000)
/lib64/ld-linux-x86-64.so.2 (0x0000003d91400000)
[nz@umabofanzd05h1 src]$ locate libcurl.so.4
/nz/export/ae/appsysroot/curl-7.44.0/lib/.libs/libcurl.so.4
/nz/export/ae/appsysroot/curl-7.44.0/lib/.libs/libcurl.so.4.4.0
/nz/export/ae/appsysroot/host64/lib64/libcurl.so.4
/nz/export/ae/appsysroot/host64/lib64/libcurl.so.4.4.0
/nz/export/ae/appsysroot/spu64/lib64/libcurl.so.4
/nz/export/ae/appsysroot/spu64/lib64/libcurl.so.4.4.0
[nz@umabofanzd05h1 src]$ locate libR.so
/nz/export/ae/languages/r/3.1/host64/lib64/R/lib/libR.so
/nz/export/ae/languages/r/3.1/spu64/lib64/R/lib/libR.so
Any idea why curl.so can't be loaded? Thanks for any help!
and it breaks the handle
library(curl)
h <- new_handle(verbose = TRUE, debugfunction = function(type, msg) NULL)
curl_fetch_memory("http://had.co.nz", handle = h)
curl_fetch_memory("http://had.co.nz", handle = h)
e.g.
curl_option_types <- function(opts = curl::curl_options()) {
type_name <- c("integer", "string", "function", "number")
type <- floor(opts / 10000)
type_name[type + 1]
}
On OpenBSD bash is installed (it is not available in the base system) at /usr/local/bin/, so https://github.com/jeroenooms/curl/blob/master/configure#L1 fails.
If you do not want to use /bin/sh, another solution would be "/usr/bin/env bash".
However, using bash at all is considered bad style: https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Writing-portable-packages states "Bash extensions also need to be avoided in shell scripts". So /bin/sh would be the best solution.
What would you recommend?
For the second time in a row, I've started working on a problem and Hadley or someone else has informed me you're already doing it ;p. I figure I should get involved in this (even if it means learning the C API. And, you know: if you'll have me).
Things I'd like to add, as a starter: bindings to easy_unescape and easy_escape, ideally ones that split the scheme out beforehand to avoid the :// getting encoded.
And, what are your thoughts on adding functionality that isn't found in the underlying curl library? URL composition and decomposition (as httr does it, but in C and vectorised), value extraction from parameters, that sort of thing.
This runs the preprocessor -E
over curl.h
to resolve symbols. We extract only option symbols:
CFLAGS=$(pkg-config --cflags libcurl)
echo '#include <curl/curl.h>' | cc $CFLAGS -std=gnu99 -E -xc - | grep "^[ \t]*CURLOPT_*"
Thereby we do not have to search for curl.h
but instead use cflags from pkg-config
.
It either succeeds or fails - if there's a long time out for some reason (e.g. curl::curl_fetch_memory("http://httpbin.org/delay/10")
) , there's no way to interrupt the transfer. Maybe we should instead use curl_multi_perform
, regularly checking for interrupts?
This problem only occurs before the server responds with headers or body, since we're already checking for interrupts in those places.
I'm having trouble installing the library on a RHEL 6.4 machine. I have libcurl-devel
installed. See the error message below.
> devtools::install_github("jeroenooms/curl")
Downloading github repo jeroenooms/curl@master
Installing curl
'/usr/lib64/R/bin/R' --vanilla CMD INSTALL \
'/tmp/RtmpAsJe5G/devtools23fc5c74c008/jeroenooms-curl-b92b20b' \
--library='/opt/Rapp/jrowen/R/x86_64-redhat-linux-gnu-library/3.1' \
--install-tests
* installing *source* package ‘curl’ ...
** libs
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c curl.c -o curl.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c download.c -o download.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c escape.c -o escape.o
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c options.c -o options.o
In file included from options.h:15,
from options.c:3:
option_table.h:168: error: ‘CURLOPT_GSSAPI_DELEGATION’ undeclared here (not in a function)
make: *** [options.o] Error 1
ERROR: compilation failed for package ‘curl’
* removing ‘/opt/Rapp/jrowen/R/x86_64-redhat-linux-gnu-library/3.1/curl’
Error: Command failed (1)
$ rpm -qa | grep libcurl
libcurl-7.19.7-40.el6_6.3.x86_64
libcurl-devel-7.19.7-40.el6_6.3.x86_64
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.4 (Santiago)
I had trouble tonight installing the curl
package under R 3.2.1 with a manually built and installed curl
library at 7.44.0, installed under a prefix $HOME/prefix
. Many other source packages could be successfully built with the same configuration, and in the end some unclear combination of changes made the error go away. I'm still puzzled what the problem was, and I'm wondering whether R and/or the Makefile
dealing correctly with my Makevars
file. I'm not very familiar with the Makevars
file in R, it could well be incorrect.
Can you look into this issue and tell me what has caused the problem and whether something should be fixed?
~/.R/Makevars
:
PKG_LIBS = -L${HOME}/prefix/lib
PKG_CPPFLAGS = -I${HOME}/prefix/include
PKG_CONFIG_PATH=$HOME/prefix/lib/pkgconfig
Linux *.nl 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux
Sys.getenv('LD_LIBRARY_PATH')
# [1] "/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/jre/lib/amd64/server:/home/myname/prefix/lib/"
gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.9.2-10' --with-bugurl=file:///usr/share/doc/gcc-4.9/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.9 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.9 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.9-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --with-arch-32=i586 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.9.2 (Debian 4.9.2-10)
install.packages('curl')
resulted in this failure:
* installing *source* package ‘curl’ ...
** package ‘curl’ successfully unpacked and MD5 sums checked
Found pkg-config cflags and libs!
Using PKG_CFLAGS=-I/home/myname/prefix/include
Using PKG_LIBS=-L/home/myname/prefix/lib -lcurl
** libs
"/usr/lib/R/bin/Rscript" "../tools/options.R"
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c callbacks.c -o callbacks.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c curl.c -o curl.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c download.c -o download.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c escape.c -o escape.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c fetch.c -o fetch.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c form.c -o form.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c getdate.c -o getdate.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c handle.c -o handle.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c ieproxy.c -o ieproxy.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c init.c -o init.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c utils.c -o utils.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG "-I/home/myname/prefix/include" "-I/home/myname/prefix/include" -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c version.c -o version.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-z,relro -o curl.so callbacks.o curl.o download.o escape.o fetch.o form.o getdate.o handle.o ieproxy.o init.o utils.o version.o -L/home/myname/prefix/lib -L/usr/lib/R/lib -lR
installing to /home/myname/R/x86_64-pc-linux-gnu-library/3.2/curl/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/home/myname/R/x86_64-pc-linux-gnu-library/3.2/curl/libs/curl.so':
/home/myname/R/x86_64-pc-linux-gnu-library/3.2/curl/libs/curl.so: undefined symbol: curl_easy_escape
Error: loading failed
Commenting out the Makevars, and building from the current master
revision with
R CMD INSTALL --configure-vars="INCLUDE_DIR=$HOME/prefix/include LIB_DIR=$HOME/prefix/lib" "$HOME/installers/curl"
is successful:
* installing *source* package ‘curl’ ...
** package ‘curl’ successfully unpacked and MD5 sums checked
Found pkg-config cflags and libs!
Using PKG_CFLAGS=-I/home/myname/prefix/include
Using PKG_LIBS=-L/home/myname/prefix/lib -lcurl
** libs
"/usr/lib/R/bin/Rscript" "../tools/options.R"
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c callbacks.c -o callbacks.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c curl.c -o curl.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c download.c -o download.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c escape.c -o escape.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c fetch.c -o fetch.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c form.c -o form.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c getdate.c -o getdate.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c handle.c -o handle.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c ieproxy.c -o ieproxy.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c init.c -o init.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c utils.c -o utils.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/home/myname/prefix/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c version.c -o version.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-z,relro -o curl.so callbacks.o curl.o download.o escape.o fetch.o form.o getdate.o handle.o ieproxy.o init.o utils.o version.o -L/home/myname/prefix/lib -lcurl -L/usr/lib/R/lib -lR
installing to /home/myname/R/x86_64-pc-linux-gnu-library/3.2/curl/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (curl)
On OSX:
> curl_perform("https://media.mongodb.org/zips.json", handle = new_handle(verbose=T))
* Hostname was NOT found in DNS cache
* Trying 54.230.69.49...
* Connected to media.mongodb.org (54.230.69.49) port 443 (#0)
* SSL peer handshake failed, the server most likely requires a client certificate to connect
* Closing connection 0
Error in curl_perform("https://media.mongodb.org/zips.json", handle = new_handle(verbose = T)) :
SSL connect error
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.