sgsokol / deriv Goto Github PK

Consider the following

Deriv::Deriv(quote(.e1*x), 'x')
Deriv::Deriv(quote(dnorm(x ** 2 - x)), 'x')
Deriv::Deriv(quote(.e1*x), 'x')
Deriv::Deriv(Deriv::Deriv(quote(dnorm(x ** 2 - x)), 'x'), 'x')
Deriv::Deriv(quote(.e1*x), 'x')

the last call to Deriv seems to be influenced by the previous one in that .e1 is somehow remembered. In my opinion this is a serious issue.

When I try to calculate the partial derivative of below function, there's an error caused as attached.

The code is as below:
func.2 <- function(thta) log(prod(1.5thta(x[i]^0.5)exp(-thta(x[i]^1.5)), i=1, n))
Deriv(func.2, "thta")

Please take a look at the following code. It seems there is something going wrong during simplification. My guess for the source of issue are the superfluous brackets enclosing pnorm and dnorm.

f = parse(text='(1 + ((pnorm(y1, beta1 + beta2 * x + beta3 * x^2, 1))^(-alpha) - 
    1 + (pnorm(y2, beta4 * x + beta5 * x^3 + beta6 * x^4, 1))^(-alpha) - 
    1))^(((-1/alpha) - 1) - 1) * (((-1/alpha) - 1) * ((pnorm(y2, 
    beta4 * x + beta5 * x^3 + beta6 * x^4, 1))^((-alpha) - 1) * 
    (-alpha))) * ((-1/alpha) * ((pnorm(y1, beta1 + beta2 * x + 
    beta3 * x^2, 1))^((-alpha) - 1) * (-alpha))) * (1 * (dnorm(y1, 
    beta1 + beta2 * x + beta3 * x^2, 1)) * (dnorm(y2, beta4 * 
    x + beta5 * x^3 + beta6 * x^4, 1)))')[[1]]
env = list(y1=0, y2=0, beta1=1, beta2=1, beta3=1, beta4=1, beta5=1, beta6=1, x=0, alpha=0.5)
eval(f, env)

d = Deriv::Deriv(f, 'beta1')
eval(d, env)
d

(1) I believe the code in the rule for differentiating dbinom(), found
in the drule environment, is incorrect. The attached script "demo01.txt"
provides evidence for my belief. (The derivative produced by Deriv()
is always negative, whereas it should be positive for prob < y/size
and negative for prob > y/size.)

The script demo01.txt also contains code for a proposed correction to the
rule.

Note in addition that "demo01.txt" raises the issue that if a rule is placed
in a new environment, rather than being placed in "drule", then an
error is thrown when one attempts to calculate a second derivative.
(Calculation of the first derivative seems to be OK.) This phenomenon
is illustrated in the attached script "demo02.txt".

(2) Incorrect results seem to be produced when second derivatives are
calculated when the derivative depends on a rule in "drule". This is
illustrated in the attached script "scr01.txt" This script also produces
correct values using calculations of first and second derivative done
"by hand". Note that the function value f1 agrees with the correct
function value f0, as does the first derivative df1 with the correct
value df0. However the second derivative value d2f1 is 1.769472 which
differs from the correct value, d2f0 = 2.064384. When derivatives are
calculated without applying a rule from "drule", as in the attached
script "scr02.txt", d2f1 agrees with the correct value.

Note that the problem is illustrated using a "local" version of dbinom()
called ldb() which has been simplified so as to clarify the issue. The
rule that I placed in "drule" for differentiating ldb() is also made
to be very simplistic so as to illustrate the problem clearly. This
rule is obviously not "robust" and would result in problems with
borderline cases.

demo01.txt
demo02.txt
scr01.txt
scr02.txt

Hi,

Thanks for the package. I am using version

Package: Deriv
Type: Package
Title: Symbolic Differentiation
Version: 3.5.6
Date: 2015-09-23
Authors@R: c(person(given="Andrew", family="Clausen", role="aut"), person(given="Serguei",
          family="Sokol", role=c("aut", "cre"), email="[email protected]"))
Description: R-based solution for symbolic differentiation. It admits user-defined
          function as well as function substitution in arguments of functions to be
          differentiated. Some symbolic simplification is part of the work.
License: GPL (>= 3)
Suggests: testthat
BugReports: https://github.com/sgsokol/Deriv/issues
NeedsCompilation: no
Packaged: 2015-09-24 13:50:15 UTC; sokol
Author: Andrew Clausen [aut], Serguei Sokol [aut, cre]
Maintainer: Serguei Sokol <[email protected]>
Repository: CRAN
Date/Publication: 2015-09-25 00:43:31
Built: R 3.2.2; ; 2015-10-26 06:31:56 UTC; unix

I used in a computation and run into a problem when the function to be differentiated contains a call to another function in which body there are more than one expression.
Here is a simple example

f<-function(x){ t<-x^2; log(t) }
g<-function(x) cos(f(x))
Deriv(g,"x")

returns the function

function (x) 
-(sin(f(x)) * {
    t <- x^2
    .t_x <- .t_x
    .t_x/t
})

Here t_x is not defined. I guess that this is somehow the result of some simplification step, but I could not really follow what Deriv does during the debugging. At some stage the expected code

dargs
$x
{
    t <- x^2
    .t_x <- 2 * x
    .t_x/t
}

was in the dargs variable within Deriv_.

During the differentiation of a more complicated function it happened that the variable .e1 was used before a value was assigned to it, and also many assignment of the type .esomething <- .esomething (the left hand side and the right hand side are the same) I do not know whether these issues are related.

Thanks
Vilmos Prokaj

Hi, when I tried the Deriv package, I met the following issue:

> library(Deriv)
> f <- function(x, y) x + y
> Deriv(f)
function (x, y, x = 1) 
1
> f <- function(x, y) x^2 + y^2
> Deriv(f, nderiv = 2)
function (x, y, x.x = 2, x.y = 0, y.x = 0) 
2

It seems that the results get into the function arguments.

This is my R session information:

R version 3.4.2 (2017-09-28)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] Deriv_3.8.1

loaded via a namespace (and not attached):
[1] compiler_3.4.2 tools_3.4.2    yaml_2.1.14   

Hi,

I found that two simplifications for log are obviously incorrect. log(a/b) = log(a) - log(b) and log(a*b) = log(a) + log(b) only for positive a and b. If both are negative, this simplification introduces NA.
Should be easy to fix (remove) ;)

Best regards, !evil

Hi Deriv authors,

I have a testing example that ran into problem:

    fun <- function(t,g,tau) c(1/(1+exp(t-g - tau[2])),1/(1+exp(-(t-g - tau[1]))) )
    require("Deriv")
    dp1 = Deriv(fun,c("t"))

How to solve this?

In the python package autograd, user can definite their own primitives.

Right now one can do something similar in Deriv package, like below

library(Deriv)
logsumexp <- function(x) {
   max_x = max(x)
   log(sum(exp(x-max_x))) + max_x
}

df_logsumexp <- function(x) {
  e_x = exp(x - max(x))
  e_x/sum(e_x)
}

drule[["logsumexp"]] = alist(x=df_logsumexp(x))

df <- Deriv(logsumexp)
df(c(1,2))

[1] 0.2689414 0.7310586

But above approach is hidden away from users (I only know this after I checked the R source code).

Hi,

I have a problem with applying the Deriv function to a custom R function I defined. Here is a MWE (forget about whether it does something useful or not) that throws the error for me:

f_test <- function (x,y) norm(acos(x%*%y), type="2")
Deriv(f_test, "x")
Error in missing(FALSE) : invalid use of 'missing'

It might be that this error is caused by something related to acos (auto-completion of missing arguments maybe?), a doublecheck for the simplest function I could come up with works, so it's not necessarily the installation:

f_test <- function (x,y) {x%*% y}
Deriv(f_test, "x")
function (x, y)
y

My R version is 4.2.0 for macOS, the installed Deriv package version is 4.1.3

When I use the following code as a package example

f = quote( (1 + alpha)^(1/alpha) )
d1 = Deriv::Deriv(f, 'alpha')
print(d1)
d2 = Deriv::Deriv(d1, 'alpha')
print(d2)

and run R CMD check I get the following error

Error in expr[[3]][[2]] : object of type 'symbol' is not subsettable
Calls: <Anonymous> ... Simplify -> Simplify_ -> Simplify.rule -> Numden -> Numden

at d2 = .... The first derivative works just as expected. What's wrong? Did I miss something crucial there?

(Don't know if this is the right venue. If not, just close this and I'll post it somewhere else (r-devel mailing list or StackOverflow ...)

I'm developing a package (with @parksw3 on github), https://github.com/bbolker/fitsir , that uses Deriv with some custom values in drule. We're having trouble figuring out how to export our custom drule values so that they work when our package is loaded.

We made a stripped-down version of the package here. There is a broken branch, which we initially thought would work but doesn't, and a master branch, which works but seems clunky to us.

Here's an example from an interactive session:

library(Deriv)
myfun <- function(x, y) x + y
mydfun <- function(x, y) NULL
drule[["myfun"]] <- alist(x=mydfun(x, x), y=mydfun(y, y))
Deriv(myfun(x, y), c("x", "y"))
## result:
## c(x = mydfun(x, x), y = mydfun(y, y))

In the broken branch, we put these definitions in our R package code. We export all of the functions in the package: even if we export the drule function, we get the following:

library(Derivtest)
myderiv ## computed within package: as in interactive session
## c(x = mydfun(x, x), y = mydfun(y, y))
Deriv(myfun(x, y), c("x", "y"))
## x=1, y=1
Deriv2(myfun(x, y), c("x", "y"))
## x=0, y=0

What seems odd is that find("drule") finds a copy of drule in the Derivtest environment (before the copy in Deriv, which has our custom drule in it. We can get this to work (master branch) if we add our new drule on the fly, within the function:

Deriv2 <- function(expr, var) {
    drule[["myfun"]] <- alist(x=mydfun(x, x), y=mydfun(y, y))
    Deriv(substitute(expr), var)
}

We've tried a lot of different solutions (assigning to global environment, playing around with the env argument), but haven't been able to figure out a better solution.

Suggestions?

This is not specific to Deriv but annoying. Whenever expressions get longer like "log(something long)" then assigns/gets from the cache environment fail.

Lists don't suffer from this. Would you consider switching to a list as cache?

Hi,

I made a few changes to Deriv.R so that matrix operations like %*%, det and solve work.

Please take a look at my fork, and in particular
https://github.com/notEvil/Deriv/blob/master/showcase.R

Its a proof of concept and certainly needs adjustment to be considered for a pull request.

KR, Andreas

f = quote(
(1) * (({
    .e3 <- exp(-(theta0 + theta2 * x))
    .e4 <- exp(-(theta0 + theta3 * x))
    .e5 <- .e4 - .e3
    .e6 <- gamma1^tau
    2 * ((1 - beta2/.e5) * exp(-(.e6 * .e4))) + 2 * (beta2 * 
        exp(-(.e6 * .e3))/.e5) - 1
}) * ({
    .e3 <- exp(-(theta0 + theta1 * x))
    .e4 <- exp(-(theta0 + theta3 * x))
    .e5 <- .e4 - .e3
    .e6 <- gamma2^tau
    2 * ((1 - beta1/.e5) * exp(-(.e6 * .e4))) + 2 * (beta1 * 
        exp(-(.e6 * .e3))/.e5) - 1
}))
)

Deriv::Deriv(f, 'beta1')

the issue is in the second statement of the result

.e6.e4 <- exp(-(.e3 * .e6))

The current cran version has a browser() command left uncommented on line 456 of the file Deriv.R.

If you run Deriv::Simplify(quote(y + c(1, 2))) it will fail due to as.character returning a vector with length 2 at Simplify.R:146. If replaced by toString, it works. I'm not sure if toString is the correct fix for this issue though ^^

Deriv::Simplify(quote(y + c(0, 1))) returns y because of b == 0 in Simplify.R:133 (and maybe 130 too). It will only consider the first value which turns out true in this case. Replacing the expression with identical(b, 0) (and identical(a, 0) on line 130) fixed the issue.

There seems to be an issue when using a long expression as f-argument in Deriv().

For example:

Deriv(function(x) eedddddddddddddlog(x)^(1-sig)*exp(x)*h, "x")

results in the following error "Error in drule[[fch]] : wrong arguments for subsetting an environment", while the one character shorter

Deriv(function(x) eeddddddddddddlog(x)^(1-sig)*exp(x)*h, "x")

works fine. The problem seems to be related to deparsing f [in particular line 262 in Deriv.R: fch <- deparse(substitute(f))]. The issue is that if the argument is too long deparse() splits its output and returns a vector of strings instead of returning a single string. Using deparse1() which wraps deparse() and uses a paste(... , collapse = " ") call on top should fix that issue.

By the way: Defining the function first, e.g.

myfun <- function(x) eedddddddddddddlog(x)^(1-sig)*exp(x)*h
Deriv(myfun, "x")

obviously works fine.

Hi Serguei,

I have an example that Deriv may not be able to get correct answer. See

P <- function(x,Theta){	
	xi_rho1 = exp( x[3] )
	sumTD = x[2]*(Theta-x[1])
	xi_td1 = exp(abs(sumTD))
	q1 = xi_td1/(xi_rho1 + xi_td1)
	p1 = xi_rho1/(xi_rho1 + xi_td1)
	
	xi_rho1 = exp( x[6] )
	sumTD = x[5]*(Theta-x[4])
	xi_td1 = exp(abs(sumTD))
	q1b = xi_td1/(xi_rho1 + xi_td1)
	p1b = xi_rho1/(xi_rho1 + xi_td1)
	
	p = 1/(1+ (q1*p1b)/(p1*q1b) )
	cbind(p,1 - p)				
}
require("Deriv")
dp1 = Deriv(P,c(x=1,x=2, x=3, x=4 ,x=5,x=6))
dp1(c(0,1,1,0,1,1), t(matrix(c(1))) )

# compared to
myenv <- new.env()
assign("rho", c(0,1,1,0,1,1), envir = myenv)
assign("theta", t(matrix(c(1))) , envir = myenv)
numericDeriv(quote(P(rho,theta)), c("rho", "theta"), myenv)

Dear Serguei, thank you very much for making Deriv and quick response! I met another problem in using Deriv.

> f <- function(x, y){
+     c(x ^ 2, y ^ 2)
+ }
> Deriv(f)
function (x, y) 
c(x = c(2 * x, 0), y = c(0, 2 * y))
> f <- function(x, y){
+     c(x, y) ^ 2
+ }
> Deriv(f)
Error in !ize 
> 

So things like c(x^2, y^2) are supported, and things like c(x, y)^2 are not. Is Deriv going to support this?

Dear Serguei,

I've noticed the following problem:

Deriv("1/(1+exp(-x))","x",nderiv=3)
[1] "{     .e1 <- exp(-x);     .e2 <- 1 + .e1;     .e3 <- .e1/.e2;     (1 + (2 * (2 * .e3 - 1) + 2 * (.e3 - 1) - 2) * .e1/.e2) * ;         .e1/.e2^2 }"

which has a syntax problem (extra ";").

When the input is an expression, it works:

Deriv(expression(1/(1+exp(-x))),"x",nderiv=3)

Dear Serguei,

Thanks for writing this package, it's very useful. There's a bug related to non-standard evaluation that prevents Deriv from being used inside of a function:

library(Deriv)
g = function(f) Deriv(f)
g(cos)

I'm trying to use Deriv to write a function that automatically produces Taylor approximations, but that requires passing a function as argument. Any ideas on how to fix this?

Thanks

Simon