grosssbm / misssbm Goto Github PK

inferSBM should exploit multicore computing if possible (either with parallel or future).

Consider the following piece of code for sampling in a 300-node SBM with various sampling schemes.

## SBM parameters
N <- 300
Q <- 3
alpha <- rep(1,Q)/Q                     # mixture parameter
pi <- diag(.45,Q) + .05                 # connectivity matrix
directed <- FALSE
# Draw a SBM model (Bernoulli, undirected)
mySBM <- simulateSBM(N, alpha, pi, directed)
A <- mySBM$adjacencyMatrix

## network samplings 
dyad  <- samplingSBM(A, "dyad", parameters =.1)
node  <- samplingSBM(A, "node", parameters =.1)
block <- samplingSBM(A, "block", parameters =c(.1, .2, .7), clusters = mySBM$memberships)
double_standard <- samplingSBM(A,"double_standard", parameters =c(0.1, 0.5))
degree <- samplingSBM(A,"degree", parameters =c(0.01,0.01))
snowball <- samplingSBM(A,"snowball", parameters =.3)

To test if each sampling given a network with a proportion of NA corresponding to what's expected, I need the theoretical sampling rate for each sampling design.

For instance, for the dyad samplign rate, it is simply the value of the parameters (0.1).

I need the theoretical expectation for the other ones as a function of the vector of parameters.

Error: netSampling %in% available_samplings is not TRUE

When compiling JSS paper with knitr.

We should use covr, in complement with testthat in #2

Required by JSS pre-reviewing

See e.g., test_consistency:

• fittedSBM$covarParam
• fittedSampling$parameters

The networkSampling_fit should be splitted into two subclasses, with/without covariates.

The same for the class networkSampling_sampler.

See the structure of SBM_fit for a reference.

See vignette for an astonishing estimate for the covariate effect... (comparison with SBM)

So we should never store neither covarMatrix nor covarSimilarity in it.

There are recurrent warnings that appears during the smoothing of ICL curve with degree sampling : "In log(1 - prob) : NaNs produced"

Once covariates will be available, maybe we could use network ~ covariates to specify our model.

Think also about how specifying smartly the sampling model.

This can be as simple as the karate club network with some randomnly sampled edges and dyad.

All function (even internal) should have a basic documentation to help other developers to understand what's going on...

We should start to write a basic vignette.

The correct argument should be an array of covariates.

See file inst/lostinICL.R for reproductible example.

ICL inconsistency when comparing with direct computation. I mean ICL computed in the class SBMfit

Probably in file networkSampling_fit-Class.R

Maybe related to #21, the following code for first estimation of pi in missSBM-fit is relevant for problem without covariates (when private$pi is indeed the mathematical matrix of connectivity between blocks)

https://github.com/jchiquet/missSBM/blob/18c3959d60ae4cf12039e37492987e89a8702253/R/missingSBM_fit.R#L32-L37

However, when private$pi represent gamma, as it is the case for the model with covariatess, we should adapt this first initialization and imputation. It has been show to be crucial in order to reproduce properly the resuts found with @TabouyT 's implementation.

So @TabouyT , comment initialises-tu les pi/gamma dans le modèles avec covariables ?

At this stage of the development, we perform imputation even in the MAR case to keep the same framework, whatever the underlying sampling process (MAR or NMAR).

Not only it would save some time to eprform the inference only on the observed part of the surrogate loglikelihood, but it would also be more correct in the LMAR case with covariates.

I will create a branch for that, as it changes a bit the interface with the C++ code and also the structure of the R6 object. A elegant solution would be to handle NA in the C++, by only looping over the no-NA value of the network adjacency matrix.

Also related to #8

Error: netSampling %in% available_samplings is not TRUE

when sampling = "block-node", same code

At this stage, printing the result of the inferSBM is not informative at all...

If the adjacency matrix is symmetric, does missSBM use the symmetric version of the lik?

The smooth function doesn't always improve the estimation, it makes it even worst and it shouldn't be ... Example with R code in JSS paper (.Rnw code)... See the ggplot attached with this issue

Rplot01.pdf

Indeed, this function can perform sampling of a network even if the network is not drawn under a SBM.

This will save time in the tests, I only manage to fit on "up to" a required number of clusters 👍

BM <- blockmodels::BM_bernoulli_covariates("SBM_sym", A, covariates_BM, verbosity = 0, explore_max = Q, plotting = "", ncores = 1)

THe following code does not work properly

A <- matrix(rbinom(100,1,.2),10,10)
  type <- "simple"
mySBM <- SimpleSBM_fit$new(A, "poisson",directed=FALSE)

This is important and should avoid long waste of time when code has not been checked for a while. This is also important for

• targeting a release on CRAN
• preparing Timothée's end of PhD

Moreover, it does not make sense since we should not store the array of covariates in there.

With basic output like: number of nodes, dyads, sampling rates.

Required for resubmission to JSS

see inst/covariates.R
In the particular case of a node sampling when we want to provide a matrix for the covariate on dyad

Hi

We are preparing the next version of ggplot2 and our reverse dependency tests shows an issue with missSBM. The issue revolves around tighter checks of theme settings in facet rendering and means that free scales in facets will error if the theme has a specified aspect ratio. This change results in an error when running the examples in the estimateMissSBM documentation.

The next release is available in the v3.3.4-rc branch if you need to test against it. We plan on releasing in the next week.

best
Thomas

Hi Julien !

using SBM and LBM functions we would like to explore storedModels using the setModel method. But setModel does not allow exploring the last model, the one with the highest Index value. Is it possible to correct for that?

Also, I have a question about the storedModels method. Are they all here or are there more than that stored but not accessible through the method storedModels ?

Virginie and Benoit

This should simplify the writing of the main function estimate, avoid multiple copies and decouple the problem of smoothing.

So contrarely to #16, we must include both covariates array and matrix to handle both sampling on nodes and dyad.

Most of the functions and classes should not be call by the user.

At this stage of the package, a lot of classes are exported and should not.

Once they are no more exported, note that they are still reachable by missSBM::: to perform test.

This is a popular sampling so we should add the possibility to sample against this snowball thing, with a single or several waves.

Now that @TabouyT solves #4 we need to to complete

• test-network-data-samplings.R (block, double standard and degree)
• test-network-data-samplings-with-covariates.R (node and dyad)

Recurrent Error : "Error: number of cluster centres must lie between 1 and nrow(x)".
Probably due to K-means algorithm when splitting a class into two when only one node is in the class.
Solution : prevent these cases with a "if"

We explore all possible models at each iteration of forward/backward smoothing. We can do better than that.

At some point, the smoothing process should even be integrated in the standard fitting procedure.

This will impact at least dyadSampling_fit_covariates and nodeSampling_fit_covariates

Required for JSS resubmission.