Coder Social home page Coder Social logo

valentinitnelav / bootstrapnet Goto Github PK

View Code? Open in Web Editor NEW
4.0 2.0 2.0 5.8 MB

R package for bootstrapping indices of bipartite ecological networks

Home Page: https://valentinitnelav.github.io/bootstrapnet/

License: MIT License

R 100.00%
r bipartite-network bipartite resample bootstrap network-metrics ecological-network pollination ecological-indices ecological-modelling

bootstrapnet's Introduction

bootstrapnet

DOI Licence

I will not have time to maintain this package anymore. If anyone wants to take over, please let me know.

Overview

R package for resampling network metrics/indices (bootstrapping without replacement).

GitHub site at: https://valentinitnelav.github.io/bootstrapnet/index.html

It wraps around the bipartite package functions networklevel and specieslevel.

Assuming a network/web like Safariland from the bipartite package:

library(bipartite)
data(Safariland)

One can sample interactions without replacement from the network until all interactions have been used. The sampling procedure starts with a small sample size to which interactions are added until all are consumed. Every time we sample interactions, a smaller version of the entire network can be built and a network index/metric can be computed. The sampling procedure can be repeated as many times as needed, giving the possibility to compute mean values with quantile-based confidence intervals. The mean values across sample sizes can be plotted and indices of different networks can be visually compared. See examples below.

Below is an animation of the sampling method (one iteration). A detailed explanation of the method can be found here.

Installation

You can install bootstrapnet from GitHub with:

# install.packages("devtools")
devtools::install_github("valentinitnelav/bootstrapnet")

Examples

See more examples here

Resample two networks with computing "nestedness".

library(bootstrapnet)
library(bipartite)
library(magrittr)
data(Safariland)

# Generate two fictive networks to compare

set.seed(321)
Safariland_1 <- Safariland[, sort(sample.int(ncol(Safariland), 20))]
set.seed(123)
Safariland_2 <- Safariland[, sort(sample.int(ncol(Safariland), 20))]


# Resample the two networks with computing "nestedness". The computation is
# carried in parallel.

lst_nest <- list(s1 = Safariland_1, s2 = Safariland_2) %>%
  lapply(web_matrix_to_df) %>%
  boot_networklevel(col_lower = "lower", # column name for plants
                    col_higher = "higher", # column name for insects
                    index = "nestedness",
                    level = "both", # here, nestedness is not affected by level
                    start = 20,
                    step = 10,
                    n_boot = 100,
                    n_cpu = 3)
# approx. 1 min of CPU time

gg_networklevel(lst_nest)

spl_size represents the number of interactions sampled (without replacement) from each network/web (here, s1 and s2). The first sample is set to start = 20 interactions. These 20 interactions form a small web for which a nestedness value is computed. Then we add 10 more randomly sampled interactions (step = 10) that where not sampled yet. The new network has now 30 interactions and a new nestedness is computed for spl_size = 30, and so on until all interactions were sampled (that is, the entire web).

So, after the first start = 20 sample, at each step of 10 sampled interactions (step = 10), a metric is computed (here nestedness). The last computed index value (right most tip of a bootstrap mean line) corresponds to the index value of the entire network (because at this point, as mentioned above, the entire network was sampled).

These operations are repeated n_boot = 100 times in parallel on n_cpu = 3 CPUs for each web (s1 & s2). For each web, each thinner line represents one of the 100 iterations. Having 100 nestedness values at each spl_size, then we can compute an average and the 95% quantile-based confidence intervals (CI) around it. Therefore, we get the mean thicker line and its 95% CI dashed lines. Is normal to see wider 95% CIs at smaller spl_size since there is high variation in the networks constructed from the few sampled interactions. At the other end, the CIs have to converge at the final index value, which is the index of the entire network (see values below).

The same resampling / bootstrapping procedure is applied for species level indices.

# The last computed index value (right most tip of a bootstrap mean line)
# corresponds to the index value of the entire network:

networklevel(Safariland_1, index = "nestedness")
## nestedness
##  17.36351

networklevel(Safariland_2, index = "nestedness")
## nestedness
##  23.23792

Discussion

Such accumulation/rarefaction curves allow comparison of networks/webs with different number of interactions. Ideally the indices/metrics will be compared if the curves display a trend of reaching an asymptote. That means that if we keep on investing effort to sample interactions (observe plant-pollinator in the field) we will not gain much further information, so network comparison is already possible.

Here, the web s2, even though has fewer interactions, seems to produce a stable, asymptotic resampled nestedness (blue line) as the one of web s1 (red line). The two networks are not much different in terms of nestedness, since the 95% CI overlap considerably.

How to contribute to the package?

You could fork the github repository, apply your changes, test if things work properly and then submit a pull request. Moreover, feel free to check or open issues about code and documentation suggestions, bugs, etc.

I am open to code improvements for speed, readability, better modularity, practical simplifications, etc. Please, always comment as much as possible your code and reasoning. Better comment more than less :)

How to cite the package?

Releases of the package are constantly archived on Zenodo at this link where you find also their suggestion for citation. Inspired from Zenodo’s suggestion, you could cite as:

Valentin Ștefan, & Tiffany Marie Knight. (2020). R package for bootstrapping indices of ecological networks. Zenodo. http://doi.org/10.5281/zenodo.3997559

Example of scientific papers using bootstrapnet

Zoller, L., Bennett, J., & Knight, T. M., 2023. Plant–pollinator network change across a century in the subarctic. Nature Ecology & Evolution, 1-11. Link to paper here

Rakosy, D., et al. 2022. Intensive grazing alters the diversity, composition and structure of plant-pollinator interaction networks in Central European grasslands. PloS one, 17(3), e0263576. Link to paper here

Motivans Švara, Elena, et al., 2021. Effects of different types of low‐intensity management on plant‐pollinator interactions in Estonian grasslands. Ecology and evolution 11.23: 16909-16926. Link to paper here

Kelly, T. and Elle, E., 2020. Effects of community composition on plant–pollinator interaction networks across a spatial gradient of oak-savanna habitats. Oecologia, 193(1), pp.211-223.

bootstrapnet's People

Contributors

valentinitnelav avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

bootstrapnet's Issues

Cannot remove prior installation of package ‘ps’

I try to install the package from github with devtools::install_github("valentinitnelav/bootstrapnet"), and I get this error message:

Downloading GitHub repo valentinitnelav/bootstrapnet@master
Installing 2 packages: ps, cpp11
Installing packages intoC:/Users/.../Documents/R/win-library/4.0’
(aslibis unspecified)
trying URL 'https://cran.rstudio.com/bin/windows/contrib/4.0/ps_1.3.4.zip'
Content type 'application/zip' length 701948 bytes (685 KB)
downloaded 685 KB

trying URL 'https://cran.rstudio.com/bin/windows/contrib/4.0/cpp11_0.2.1.zip'
Content type 'application/zip' length 214288 bytes (209 KB)
downloaded 209 KB

packagepssuccessfully unpacked and MD5 sums checked
Error: Failed to install 'bootstrapnet' from GitHub:
  (converted from warning) cannot remove prior installation of packageps

Give informative error message in `boot_networklevel` when `lst` is not a list

Below, the error message is not that informative:

library(bootstrapnet)
library(bipartite)
data(Safariland)

nest <- Safariland %>% 
  web_matrix_to_df() %>%
  boot_networklevel(col_lower = "lower", # column name for plants
                    col_higher = "higher", # column name for insects
                    index = "nestedness",
                    level = "both", # here, nestedness is not affected by level
                    start = 100,
                    step = 50,
                    n_boot = 30,
                    n_cpu = 3)

Gives

Error in function_list[[k]](value) : 
  `data` must be a data.frame or data.table.  Maybe check out the function web_matrix_to_df() for examples.

Should actually inform that the user didn't provide a list object (in this case it can be a list of one element containing the needed data).

Installation problem

Hi Valentin,

Tiffany sent me the link to your package and I look forward to see the bootstrapping you have done for the network metrics.

Im having one problem with the installation. When I tried to install it its telling me that I need parallel 3.5 or higher = "namespace ‘parallel’ 3.5.1 is already loaded, but >= 3.6 is required".

I have R version 3.5.1 and I can't update just parallel because it is part of base R. I think you should allow parallel to have version 3.5 if you are allowing R to be version 3.5

Thanks!

Melissa

Allow user to set the seed

Currently in boot_networklevel the seed is set internally and maybe that is not a good practice. Reproducibility is assured, but maybe should let the user be a bit more in charge of this.
In boot_networklevel there are two cases when seed is employed:

  1. In the usage of boot_networklevel_once, when computing a network index for the sampled interactions. Is set internally to set.seed(42);
  2. Then in boot_networklevel_n, when calling sample_indices;
  3. Again in boot_networklevel_n, when running boot_networklevel_once in parallel an the iterator is passed as seed to boot_networklevel_once.

I think point 3 should stay unchanged because each iteration should generate different samples, otherwise we take the same sample every time. Or could generate a sequence of integers of the same length as the amount of iterators based on a user given seed and then pass those as seeds to boot_networklevel_once. However, that sounds unnecessarily complicated...

Nevertheless, the seed at point 2 could be a parameter that the user may want to change. Would this be worth implementing? If yes, then set the default value of 42 so that users don't snap if they test old results with new ones.

This should also be applicable for boot_specieslevel.

Throw error message when wrong index is provided

When using a species level index (e.g betweenness) as index in boot_networklevel (or boot_networklevel_n, boot_networklevel_once) it still tries to compute something. It should break the code with error message.

I think has to do with how I set the try in boot_networklevel_once, because networklevel(Safariland_sort, index = "betweenness", level = "both") throws an error. That error should be passed to boot_networklevel_n, which should pass it to boot_networklevel.

Note that, this will stand valid for the species level functions as well.

Fix `data.table::melt` warning from `get_stats_single`

Need to fix this warning:

Warning messages: 1: In data.table::melt(., id.vars = "spl_size", variable.name = "simulation_id") : The melt generic in data.table has been passed a data.frame and will attempt to redirect to the relevant reshape2 method; please note that reshape2 is deprecated, and this redirection is now deprecated as well. To continue using melt methods from reshape2 while both libraries are attached, e.g. melt.list, you can prepend the namespace like reshape2::melt(.). In the next version, this warning will become an error.

It is triggered in boot_networklevel or boot_specieslevel and most probably by the data.table::melt in the helper function get_stats_single as it gets a data.frame instead of a data.table object.

In sample_indices() function, the stop event doesn't pass the correct number of rows

doubtful_data <- web_matrix_to_df(Safariland) %>% unique()
boot_networklevel(lst = list(s1 = doubtful_data),
                  col_lower = "lower",   # column name for plants
                  col_higher = "higher", # column name for insects
                  index = "nestedness",
                  level = "both",
                  start = 50,
                  step = 1,
                  n_boot = 4,
                  n_cpu = 2)

# Error in sample.int(length(x), size, replace, prob): cannot take a sample larger than the population # when 'replace = FALSE'

# Error in value[[3L]](cond) : 
# Possibly because your `start` value, 50, is bigger than the number of interactions in data, 1130.
# If this is the case, then consider to reduce `start` to maybe 10 % of your interactions.

boot_networklevel(lst = list(s1 = doubtful_data),
                  col_lower = "lower",   # column name for plants
                  col_higher = "higher", # column name for insects
                  index = "nestedness",
                  level = "both",
                  start = 10,
                  step = 50,
                  n_boot = 4,
                  n_cpu = 2)

# Error in cut.default(x = seq_along(x), breaks = n, labels = FALSE): invalid number of intervals

# Error in value[[3L]](cond) : 
# Possibly because your `step` value, 50, is bigger than the number of interactions in data, 1130.
# If this is the case, then consider to reduce `step` to maybe 5-10 % of your interactions.

It should not be 1130 interactions in data, but 39 (nrow(doubtful_data)). Not sure what is happening?

Expected `NA` values and therefore, warnings when plotting

Make it clear via a note that is expected to see lots of warning messages when plotting because some bootstrap values are NA (during the sampling procedure, some interactions are rare , so at small sample sizes, they are not sampled, they only appear later).

This happens particularly for species level indices. It can also happen for network levels indices when maybe the start sample size (first sampled interactions) is too small and metrics cannot be computed. Then because of how try is set, an NA is assigned instead of failing (which should stay like this).

Consider using fortify for custom graphs

I am not pleased with the idea of providing ggplot2 wrappers (the ones in bootstrapnet/R/boot_gg.r). I think is wiser to let the user build their own graphs as they please/need. I need to put more thought into this. Get inspiration from the section Draw R/E curves by yourself, here.

Allow sampling with replacement

There is demand from the community for allowing sampling with replacement. At the moment only sampling without replacement was implemented.
Not sure when i will get the time to work on this, but I welcome help.

Give informative error message for too big `start` and `step` arguments

When the start or step arguments (in boot_networklevel_n or boot_specieslevel_n) are bigger than the total number of unique interactions in a web, then the error triggered is not informative.

Make informative error message and also should stop the processing before the parallel computation starts.

boot_networklevel_n typo bug in get_list_of_matrices?

Why isn't there any difference in the two if cases in boot_networklevel_n:

  if (n == 1) {
    results <- get_list_of_matrices(boot_lst, metric_names, n, iter_spl_size)
    # Convert data back to data frame if applicable
    if (! "data.table" %in% cls_data) data.table::setDF(data)
    return(results)

  } else if (n == 2) {
    results <- get_list_of_matrices(boot_lst, metric_names, n, iter_spl_size)
    if (! "data.table" %in% cls_data) data.table::setDF(data)
    return(results)
  }

Misuse of `table` function

I should give a more realistic example in documentation about the fact that boot_specieslevel or boot_networklevel expect a list of one or more data frames of interactions. Each interaction (row in the data frame) must be repeated as many times as it was observed. E.g. if the interaction species_1 x species_2 was observed 5 times, then repeat that row 5 times within the data frame.

One misleading workflow in data preparation now is to build a web matrix with the table function from the row data, that most likely contains an enumeration of interactions. The table function will not consider the abundance of the interactions, unless they are repeated within the raw data. So, there is high risk to lose data. Then, once that web is build with the table function, the user goes to using web_matrix_to_df, which kind of completes a vicious data processing circle.

So, the user tends to build the web matrix from a data frame and then transform the matrix back into an expanded data frame. This is an unnecessary journey and I guess was inspired somehow from how I constructed the example with Safariland from bipartite. But a more real case is to take the raw data with the interactions, enumerate/explode the rows based on some abundance column and then that data is already good to use directly for boot_specieslevel or boot_networklevel. No need to use table, especially that creates a misleading way towards data loss.

So, try to make a more realistic simple usage example of boot_specieslevelandboot_networklevel, without needing to use web_matrix_to_df`, which seems to be reserved rather in rare cases. The user tends to have the raw data more as data frame from Excel than as a web matrix/community matrix.

Error: 'attempt to select less than one element in get1index' when data has only unique interactions

Need to check why this error happens. Bootstrapping should not fail even if unique interactions, or? If the case, then set an informative error message.

library(dplyr)
library(bipartite)
data(Safariland)

unique_interactions_data <- web_matrix_to_df(Safariland) %>% unique()

boot_networklevel(lst = list(s1 = doubtful_data),
                  col_lower = "lower",   # column name for plants
                  col_higher = "higher", # column name for insects
                  index = "nestedness",
                  level = "both",
                  start = 10,
                  step = 10,
                  n_boot = 6,
                  n_cpu = 2)

## Error in FUN(X[[i]], ...) : 
##  attempt to select less than one element in get1index

Not getting error message from web_matrix_to_df function when object is a table and not a matrix

Elena discovered that when one passes a table object (constructed with the table function) to the web_matrix_to_df function, then the expected error is not triggered. An error message should inform the user about the expected class of the object ( a matrix).

Example of reproducible error:

library(bootstrapnet)
library(bipartite)

test <- table(letters[1:5], letters[1:5])
web_matrix_to_df(test)

# Error in rep(1:n(), times = counts) : invalid 'times' argument
# In addition: Warning messages:
# 1: attributes are not identical across measure variables;
# they will be dropped 
# 2: In ~rep(1:n(), times = counts) : NAs introduced by coercion

Expected result should be:

# Error in web_matrix_to_df(Safariland_df) : 
#  `web` must be a matrix, e.g. `bipartite::Safariland`

Must throw error when typo in `level`

For example:

boot_networklevel_n(data = web_matrix_to_df(Safariland_1),
                    col_lower = "lower", 
                    col_higher = "higher", 
                    index = "all",
                    level = "nice_typo", # should be only "lower", "higher" or "both"
                    start = 10,
                    step = 10,
                    n_boot = 6,
                    n_cpu = 2)

gives:

$V1
   [,1] [,2] [,3] [,4] [,5] [,6]
10    1    1    1    1    1    1
24    1    1    1    1    1    1
38    1    1    1    1    1    1

It should give an error instead.

Implementing test statistics based on bootstrapnet confidence intervals

The example below is code for comparing robustness between network plots in a nature preserve and network plots on adjacent land of neighbors. In this situation the confidence intervals provide evidence that the neighbor plots are more robust than the preserve plots, particularly when considering the higher level (pollinators). I am curious if I could take this a step further and implement a test statistic based on the confidence intervals generated or another type of statistical test. Sample size is a limitation for the specific example I provide, but I am interested in any recommendations based on the example provided or future analysis where I compare the preserve plots in 2021 (n=32) to preserve plots from 2016 (n=32).

Where "preserve" (n=32) and "neighbor" (n=8) are matrices of plant-pollinator interactions across different plots

robust.compare <- list(Preserve.Plots = preserve, Neighbor.Plots = neighbor) %>%
lapply(web_matrix_to_df) %>%
boot_networklevel(col_lower = "lower", 
                    col_higher = "higher", 
                    index = "robustness",
                    level = "both",
                    start = 50,
                    step = 20,
                    n_boot = 50,
                    n_cpu = 3)

robust_graph<- gg_networklevel(robust.compare)
robust_graph

Robustness HL
Robustness LL

Examples of data preparation

Create more example of data processing. See for example the CIPI paper, data processing section of the network_properties.r where I make use of frame2webs. Also check the R notes doc , the chapter abut ecology specific data processing.

Running boostrapnet on a cluster

Hi Valentin,

Thank you for fixing the previous issue. I was able to install your package and run it perfectly on my laptop.

However, I am trying to run the package on a cluster and I get the following error:

Error in checkForRemoteErrors(lapply(cl, recvResult)) : one node produced an error: object '.doSnowGlobals' not found

I have loaded and re-installed the packages doParallel and doSNOW.

Some googling seems to suggest that I need to do set up the cluster in a different way:

cl<-makeCluster(type="MPI", 4)

Have you come across something like this before?

Melissa

Update R version in DESCRIPTION file

I think I should update to R (>= 3.6) in the DESCRIPTION file at:

Depends:
    R (>= 3.5)

because there were some new bugs that had to be addressed due to updates from R 3.5 to R 3.6, like #22

Push to CRAN

I think for the community is more convenient to install a package from CRAN than from GitHub, but since I will not have time to maintain this, then I am not sure who would keep the development up to date with whatever changes appear along the life cycle of R in general.

If anyone wants to take over, please let me know. I will not have time to maintain this package anymore.

Installation error - Unsupported proxy ... libcurl is built without the HTTPS-proxy support

When I tried to install the package on a Windows terminal server of our university, I get this:

devtools::install_github("valentinitnelav/bootstrapnet")
## Error: Failed to install 'unknown package' from GitHub:
##   Unsupported proxy 'https://proxy.bla-bla', libcurl is built without the HTTPS-proxy support.

Note that in the error message above, the bla-bla in the URL will be different from case to case.

Note also that the same error is given also by remotes::install_github

remotes::install_github("valentinitnelav/bootstrapnet")

Fit model to estimate asymptote

Maybe a user could be interested in fitting a nonlinear asymptotic model to the bootstrap values to estimate the asymptote for example.
Then include an example in documentation.

Unexpected warning message in web_matrix_to_df function

I get the following warning message when I run web_matrix_to_df:

library(bootstrapnet)
library(bipartite)

data(vazarr)

test <- bootstrapnet::web_matrix_to_df(web = vazarr)

# Warning message:
# In if (class(web) != "matrix") stop("`web` must be a matrix, e.g. `bipartite::Safariland`") :
#   the condition has length > 1 and only the first element will be used

Behavior of `gg_specieslevel_web_by_web` when not having shared species among the webs

If you have multiple webs/networks and want to run gg_specieslevel_web_by_web in one go, but do not have shared species among the webs, then this will fail gracefully :)
In such cases, run boot_specieslevel() for each web separately. Make an example to clarify this behavior. Adapting the code do deal with this might be too time consuming and not a good investment right now.

Change color of graphics (function boot_specieslevel)

metric[[i]] = list(Past_low = com_matPast_Fam_L,
Past_med = com_matPast_Fam_M,
Past_high = com_matPast_Fam_H,
Present_low = com_matPresent_Fam_L,
Present_med = com_matPresent_Fam_M,
Present_high = com_matPresent_Fam_H) %>%
lapply(web_matrix_to_df) %>%
boot_specieslevel(col_lower = "lower", # column name for plants # https://valentinitnelav.github.io/bootstrapnet/examples.html
col_higher = "higher", # column name for insects
level = "higher",
index = i, # each index takes ~ 1min of CPU time
start = 20,
step = 10,
n_boot = 30, # number of bootstraps
n_cpu = 4) # number of CPU-s to use

cols <- c("Past_low" = "grey75",
"Past_med" = "grey55",
"Past_high" = "gray40",
"Present_low" = "skyblue3",
"Present_med" = "dodgerblue2",
"Present_high"= "darkblue")

Apidae <- metric[[i]] %>%
get_stats_multi() %>%
gg_specieslevel_compare_webs(sp_higher = "Apidae")

Apidae$higher_level +
scale_color_manual(values = cols)

Are there weighted versions of the network level indices?

I am not familiar with the network metrics. The network level functions do not deal with weighted versions of the indices (if by any chance is applicable). I noticed that the specie level indices have weighted variants.
At a fast search I couldn't find weighted variants in bipartite for the network level indices, but if anybody knows something about this, then please let me know.

Document the sampling procedure

I think is a useful idea to illustrate the sampling procedure. Maybe use the Safariland dataset from bipartite.
Show how a first and few subsequent samples are taken from the web until all interactions are consumed. At each step an index is computed. Repeating this n_boot times allows to compute some statistics like: mean and 95% confidence intervals.

Error in FUN(X[[i]], ...) : subscript out of bounds in boot_specieslevel

Error appeared in the Poland dataset when computing closeness and betweenness

dt <- read.csv2("../data/Interactions_PL18_ver2.csv")
setDT(dt) # convert to data.table; to convert back just run setDF(dt_raw)

# Prepare a table with all observed interactions, replicated by the number of
# visits observed.
set.seed(123)
dt_interactions <- 
  dt[, .(abund = sum(Individuals, na.rm = TRUE)),
     by = c("Meadow.type", "Plant", "Insect.species")] %>%
  .[rep(1:.N, times = abund)] %>% # expand/explode
  .[sample(1:nrow(.))] # shuffle

dt_meadow <- dt_interactions[Meadow.type == "Meadow"]
dt_pasture <- dt_interactions[Meadow.type == "Pasture"]

# Get an idea about the total number of interactions
dt_meadow$abund %>% unique %>% sum
dt_pasture$abund %>% unique %>% sum
start_spl <- 30
step_spl <- 20

system.time({
  lst_closeness <- 
    boot_specieslevel(lst = list(meadow = dt_meadow,
                                 pasture = dt_pasture),
                      col_lower = "Plant",           # column name for plants
                      col_higher = "Insect.species", # column name for insects
                      index = "closeness",
                      level = "both", 
                      start = start_spl,
                      step = step_spl,
                      n_boot = 100,
                      n_cpu = 3)
})

## Error in FUN(X[[i]], ...) : subscript out of bounds

Replace all try() with tryCatch() when bootstrapping

Would it be better to replace all the try statements with tryCatch, because we can assign return values like NA in case of error?

So, for example in boot_networklevel_once(), instead of having this:

  for (i in 1:length(ids_lst)){
    metric_lst[[i]] <- try({
      web <- data[ids_lst[[i]], table(get(col_lower), get(col_higher))]
      set.seed(42)
      bipartite::networklevel(web = web, index = index, level = level, ...)
    })
  }

we could better have this:

  for (i in 1:length(ids_lst)){
    metric_lst[[i]] <- tryCatch({
      web <- data[ids_lst[[i]], table(get(col_lower), get(col_higher))]
      set.seed(42)
      bipartite::networklevel(web = web, index = index, level = level, ...)
    },
    error = function(e) NA)
  }

This will return NA-s in case of errors, though it might be dangerous because try can be still informative because it can return the error message ...
Though tryCatch can also be 'instructed' to pass the error message, but is just getting more verbose than try in that case.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.