error while analysis path analysis about metan

[TiagoOlivoto]

Dear @drschowdary , thanks for reporting this issue. This probably occurs because the correlation matrix between predictors (cor_mat) is not positive definite. Could you please provide a reproducible example? See how to use reprex to do that (https://reprex.tidyverse.org/)

Here, I show a similar error, forcing an exactly singular matrix (determinant = 0)

library(metan)
#> Registered S3 method overwritten by 'GGally':
#>   method from   
#>   +.gg   ggplot2
#> |=========================================================|
#> | Multi-Environment Trial Analysis (metan) v1.16.0        |
#> | Author: Tiago Olivoto                                   |
#> | Type 'citation('metan')' to know how to cite metan      |
#> | Type 'vignette('metan_start')' for a short tutorial     |
#> | Visit 'https://bit.ly/pkgmetan' for a complete tutorial |
#> |=========================================================|
mat <- 
  data_ge2 %>% 
  mutate(EH2 = EH) %>% # force a perfectly correlated predictor
  select_numeric_cols() %>% 
  corr_coef()

pcoeff <- path_coeff_mat(mat$cor, resp = EP)
#> Error in solve.default(cor.x, cor.y): Lapack routine dgesv: system is exactly singular: U[15,15] = 0

^{Created on 2021-12-27 by the reprex package (v2.0.1)}

[TiagoOlivoto]

Since this clearly is not a bug but an issue related to the data, I'm closing this issue. Feel free to re-open it if necessary.

[drschowdary]

Sir, but with the same correlation matrix the path analysis is done in GENES!

…

On Wed, 29 Dec, 2021, 7:00 pm TiagoOlivoto, ***@***.***> wrote: Closed #15 <#15>. — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AXB5YQTIBMVZC3N2GXQNDCTUTMEQDANCNFSM5K2BJLVQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[TiagoOlivoto]

@drschowdary, path analysis is done in GENES probably because GENES uses a different method of matrix inversion, which returns an approximate inverse of a matrix even when it is singular (see an example in metan at https://tiagoolivoto.github.io/metan/reference/solve_svd.html). I strongly suggest you check the multicollinearity of the matrix of predictor traits (use metan::colindiag()), because your path coefficients might be highly biased.

[drschowdary]

Yes, sir, There is severe multicollinearity found in the data. How to do path analysis under collinearity using metan?

…

On Thu, Dec 30, 2021 at 12:55 AM TiagoOlivoto ***@***.***> wrote: @drschowdary <https://github.com/drschowdary>, path analysis is done in GENES probably because GENES uses a different method of matrix inversion, which returns an *approximate* inverse of a matrix even when it is singular (see an example in metan at https://tiagoolivoto.github.io/metan/reference/solve_svd.html). I strongly suggest you check the multicollinearity of the matrix of predictor traits (use metan::colindiag()), because your path coefficients might be highly biased. — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AXB5YQW5AHCE5PH3DAR6N2LUTNOCNANCNFSM5K2BJLVQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- with warm regards *D. **Rajasekhar*[image: Flag India animated gif 120x90] *PhD Scholar [**Genetics & Plant Breeding** (SRF)]* *School of Crop Improvement,* *College of Post Graduate Studies in Agricultural Sciences,* *Central Agricultural University (Imphal)* *Umiam, Meghalaya-793103* 📱*: [phone]*

[TiagoOlivoto]

@drschowdary, I just changed the method of matrix inversion so that a pseudo-inverse is computed even with a singular matrix. A warning will alert you about this problem. To get this new feature, install the development version of metan with

devtools::install_github("TiagoOlivoto/metan")

After installing the development version, you'll be able to run the following examples. There are two main ways of dealing with multicollinearity (see our paper for more info). The first (and that I'd suggest) is removing the traits that are being the main cause of the problem. The second is including a correction factor (k) in the diagonal of the correlation matrix.

library(metan)
#> Registered S3 method overwritten by 'GGally':
#>   method from   
#>   +.gg   ggplot2
#> |=========================================================|
#> | Multi-Environment Trial Analysis (metan) v1.16.0        |
#> | Author: Tiago Olivoto                                   |
#> | Type 'citation('metan')' to know how to cite metan      |
#> | Type 'vignette('metan_start')' for a short tutorial     |
#> | Visit 'https://bit.ly/pkgmetan' for a complete tutorial |
#> |=========================================================|

df_colinear <- 
  data_ge2 %>% 
  select(PH, EP, KW, NKE) %>% 
  mutate(PH2 = PH)

# now it runs with a warning
pc1 <- path_coeff(df_colinear, resp = KW)
#> Warning: System is computationally singular. Check the collinearity of
#> predictors with `metan::colindiag()`.
#> Severe multicollinearity. 
#> Condition Number: 50796137447998448
#> Consider using a correction factor with 'correction' argument.
#> Consider identifying collinear traits with `non_collinear_vars()`
plot(pc1)

# include a correction factor, say, 0.05
pc2 <- path_coeff(df_colinear, resp = KW, correction = 0.05)
#> Weak multicollinearity. 
#> Condition Number: 56.782
#> You will probably have path coefficients close to being unbiased.
plot(pc2)

# identifying colinear traits
diagnosis <- colindiag(df_colinear)
print(diagnosis)
#> Severe multicollinearity in the matrix! Pay attention on the variables listed bellow
#> CN = 9413786837639034
#> Matrix determinant: 0 
#> Largest correlation: PH x PH2 = 1 
#> Smallest correlation: EP x NKE = 0.233 
#> Number of VIFs > 10: 0 
#> Number of correlations with r >= |0.8|: 1 
#> Variables with largest weight in the last eigenvalues: 
#> PH2 > PH > KW > NKE > EP

# removing the collinear trait PH2
# option 1
pc3 <- 
  df_colinear %>% 
  remove_cols(PH2) %>% 
  path_coeff(resp = KW)
#> Weak multicollinearity. 
#> Condition Number: 6.119
#> You will probably have path coefficients close to being unbiased.
plot(pc3)

# option 2
pc4 <- path_coeff(df_colinear, resp = KW, pred = -PH2)
#> Weak multicollinearity. 
#> Condition Number: 6.119
#> You will probably have path coefficients close to being unbiased.
plot(pc4)

^{Created on 2021-12-30 by the reprex package (v2.0.1)}

[drschowdary]

Thank you, sir

…

On Thu, Dec 30, 2021 at 5:22 PM TiagoOlivoto ***@***.***> wrote: @drschowdary <https://github.com/drschowdary>, I just changed the method of matrix inversion so that a pseudo-inverse is computed even with a singular matrix. A warning will alert you about this problem. To get this new feature, install the development version of metan with devtools::install_github("TiagoOlivoto/metan") After installing the development version, you'll be able to run the following examples. There are two main ways of dealing with multicollinearity (see our paper <https://acsess.onlinelibrary.wiley.com/doi/10.2134/agronj2016.04.0196> for more info). The first (and that I'd suggest) is removing the traits that are being the main cause of the problem. The second is including a correction factor (k) in the diagonal of the correlation matrix. library(metan)#> Registered S3 method overwritten by 'GGally':#> method from #> +.gg ggplot2#> |=========================================================|#> | Multi-Environment Trial Analysis (metan) v1.16.0 |#> | Author: Tiago Olivoto |#> | Type 'citation('metan')' to know how to cite metan |#> | Type 'vignette('metan_start')' for a short tutorial |#> | Visit 'https://bit.ly/pkgmetan' for a complete tutorial |#> |=========================================================| df_colinear <- data_ge2 %>% select(PH, EP, KW, NKE) %>% mutate(PH2 = PH) # now it runs with a warningpc1 <- path_coeff(df_colinear, resp = KW)#> Warning: System is computationally singular. Check the collinearity of#> predictors with `metan::colindiag()`.#> Severe multicollinearity. #> Condition Number: 50796137447998448#> Consider using a correction factor with 'correction' argument.#> Consider identifying collinear traits with `non_collinear_vars()` plot(pc1) <https://camo.githubusercontent.com/24589afd8c1fb003ffe424dfa3f58d5a3c100d3e2d73182ecbffa0f217dcc8db/68747470733a2f2f692e696d6775722e636f6d2f76304e67756b4f2e706e67> # include a correction factor, say, 0.05pc2 <- path_coeff(df_colinear, resp = KW, correction = 0.05)#> Weak multicollinearity. #> Condition Number: 56.782#> You will probably have path coefficients close to being unbiased. plot(pc2) <https://camo.githubusercontent.com/8c94c2f6830a856b76e7a7fcc66a62b787f64cfe57d071b5370f3c573e89233c/68747470733a2f2f692e696d6775722e636f6d2f43594d43596f612e706e67> # identifying colinear traitsdiagnosis <- colindiag(df_colinear) print(diagnosis)#> Severe multicollinearity in the matrix! Pay attention on the variables listed bellow#> CN = 9413786837639034#> Matrix determinant: 0 #> Largest correlation: PH x PH2 = 1 #> Smallest correlation: EP x NKE = 0.233 #> Number of VIFs > 10: 0 #> Number of correlations with r >= |0.8|: 1 #> Variables with largest weight in the last eigenvalues: #> PH2 > PH > KW > NKE > EP # removing the collinear trait PH2# option 1pc3 <- df_colinear %>% remove_cols(PH2) %>% path_coeff(resp = KW)#> Weak multicollinearity. #> Condition Number: 6.119#> You will probably have path coefficients close to being unbiased. plot(pc3) <https://camo.githubusercontent.com/4ec35e2789c0a7ee820adaef1319338308fef4c3c1de463d0095646396d664d8/68747470733a2f2f692e696d6775722e636f6d2f686d66425965432e706e67> # option 2pc4 <- path_coeff(df_colinear, resp = KW, pred = -PH2)#> Weak multicollinearity. #> Condition Number: 6.119#> You will probably have path coefficients close to being unbiased. plot(pc4) <https://camo.githubusercontent.com/a8c31c09cd2fd049ffb12a83903d7b02156190acefe1d62cde76b9d77e1ae03c/68747470733a2f2f692e696d6775722e636f6d2f743243385363332e706e67> Created on 2021-12-30 by the reprex package <https://reprex.tidyverse.org> (v2.0.1) — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AXB5YQWPCYTWNS2PYDEYJJLUTRBYZANCNFSM5K2BJLVQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- with warm regards *D. **Rajasekhar*[image: Flag India animated gif 120x90] *PhD Scholar [**Genetics & Plant Breeding** (SRF)]* *School of Crop Improvement,* *College of Post Graduate Studies in Agricultural Sciences,* *Central Agricultural University (Imphal)* *Umiam, Meghalaya-793103* 📱*: [phone]*