dillonhammill / dataeditr

An Interactive R Package for Viewing, Entering Filtering and Editing Data

DataEditR

Manual data entry and editing in R can be tedious, especially if you have limited coding experience and are accustomed to using software with a Graphical User Interface (GUI). DataEditR is an R package built on shiny and rhandsontable that makes it easy to interactively view, enter, filter and edit data. If you are new to DataEditR visit https://dillonhammill.github.io/DataEditR/ to get started.

Installation

DataEditR can be installed from CRAN:

install.packages("DataEditR")

The development version of DataEditR can be installed directly from GitHub:

library(devtools)
install_github("DillonHammill/DataEditR")

To ensure that DataEditR works as expected, you will also need to install my fork of rhandsontable:

devtools::install_github("DillonHammill/rhandsontable")

Usage

DataEditR ships with a series of shiny modules, namely dataInput, dataSelect, dataFilter, dataEdit and dataOutput which have been wrapped up into a single function called data_edit() to create an interactive data editor. You can use data_edit() as a standalone application, or include the relevant modules within your own shiny applications. Alternatively, DataEditR also ships with an RStudio add-in should you prefer to interact with it in this way.

General features:

• RStudio add-in
• flexible display options (either dialog box, browser or RStudio viewer pane)
• fast rendering to quickly view datasets
• ability to interactively create data.frames from scratch
• load tabular data saved to file using any reading function (e.g. read.csv())
• save edited data to file using any writing function (e.g. write.csv())
• return appropriately formatted data as an R object for downstream use
• code required to create edited data can be optionally printed to the console or saved to a file
• support for custom themes through bslib package
• customisable user interface (title, logo and modules)
• row indices are always displayed for easy navigation
• switch between datasets or files without having to leave the application

Data editing features:

• column selection using the dataSelect module
• row selection using the dataFilter module
• edit row or column names
• addition or removal of rows or columns
• manual column resizing
• drag to fill cells
• copy or paste data to and from external software
• custom column types to simplify user input (e.g. checkboxes and dropdown menus)
• support for readonly columns to prevent users from editing certain columns
• control over which column names can be edited
• stretch columns horizontally to fill available space
• programmatically add columns or rows to data prior to loading into the data editor

A quick demonstration of some of these features can be seen below, where we use data_edit() to make changes to the mtcars dataset and save the result to a new csv file:

# Load required packages
library(DataEditR)

# Save output to R object & csv file
mtcars_new <- data_edit(mtcars,
                        save_as = "mtcars_new.csv")

Credits

DataEditR is built using the fantastic rhandsontable package. DataEditR makes use of many features for entering and editing data, but rhandsontable has support for much more sophisticated interactive representations of data should you need them. The user interface of DataEditR has been inspired by the editData package which is a great alternative to DataEditR.

Code of Conduct

Please note that the DataEditR project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

Citation

If you use DataEditR in your work, please cite the package as follows:

citation("DataEditR")
#> 
#> To cite package 'DataEditR' in publications use:
#> 
#>   Dillon Hammill (2022). DataEditR: An Interactive Editor for Viewing,
#>   Entering, Filtering & Editing Data. R package version 0.1.5.
#>   https://github.com/DillonHammill/DataEditR
#> 
#> A BibTeX entry for LaTeX users is
#> 
#>   @Manual{,
#>     title = {DataEditR: An Interactive Editor for Viewing, Entering, Filtering & Editing Data},
#>     author = {Dillon Hammill},
#>     year = {2022},
#>     note = {R package version 0.1.5},
#>     url = {https://github.com/DillonHammill/DataEditR},
#>   }

dataeditr's Issues

bug if used inside a modal

Hi again,
I've found a bug if I use the package inside a modal. Here's the code where you can reproduce the error. The bug appears if you try to change the column names or the row names.

if (interactive()) {
  ui <- fluidPage(
    actionButton("upaziendedata", "Aggiorna i dati"),
    shinyBS::bsModal("upaziendetab", "Tabella", trigger = "upaziendedata", size = "large",
    dataInputUI("input-1"),
    dataOutputUI("output-1"),
    dataEditUI("edit-1")
  ))
  
  server <- function(input, output, session) {
    data_to_edit <- dataInputServer("input-1")
    data_to_edit2 <- dataEditServer("edit-1",
                                    data = data_to_edit
    )
    dataOutputServer("output-1",
                     data = data_to_edit2
    )
  }
  
  shinyApp(ui, server)
}

I notice that you use the {rhandsontable} package. Maybe this can be useful:
https://stackoverflow.com/questions/52697624/handsontable-in-shiny-modal-does-not-render-properly

[feature request] filter, order and arrange options

Thanks for the great project.

I was wondering if you would be interested in implementing any manipulations of the data. View() allows you to reorder and filter rows by a particular column. This would be quite useful to allow users to better explore and edit the data. For example, I want to edit the values of column 1 only where the value of column 2 is greater than x.

Potentially re-ordering columns might also be useful.

If you think this would be useful, the question would be, should the filtering/ordering/arranging be saved or not. It would be easy to filter a column, do some editing and click save without remembering to remove the filter. This would end up with you losing filtered out rows. Equally, however, there would no doubt be use cases where you want the user to be able to filter rows and then return the smaller filtered data frame...

Dialog viewer not appropriate when calling data_edit() inside another function

If you call data_edit() inside another function and set viewer = "dialog", the parental function exits early when the data editor is closed. See example:

fun <- function() {
  mtcras_new <- data_edit(mtcars)
  for(i in 1:30){
    print(i)
  }
  return(mtcars_new)
}
fun()

This issue has been reported (rstudio/shiny#1723) but it looks like the only solution is to switch to using viewer = "browser" or viewer = "pane" in data_edit().

Add find/replace module

It would be useful to have a find/replace feature within data_edit(). Perhaps this could be implemented in the next update as a dataReplace module.

[Feature request] Output rcode of all edit actions

It would be nice if you could output an rscript or code that replicates all actions and edits needed to replicate the data. While this would make editing faster and easier, recode output would help new r users learn and keep the process reproducable

FIlter rows multiple times

Hi,
I use your package inside a shinyApp but the problem exists also with the data_edit() function so I'll write an example based on that function.

So after launched the function with my data, I have a problem with the filtering on rows. What I did is applying a filter on rows (i.e. EXP_ID equal to 101) and then clicking on the Close button. Everything works fine. Now imagine that I made a mistake and I want to change the filtered EXP_ID to 105. So I open again the filter module, and inside "Levels" 101 is still there. I click on the levels but I can't see the other levels of my original data (even if I remove 101). I think the problem is that the choices of the levels selectInput is based on the edited data.
However I've found a workaround: I changed the column (instead of EXP_ID I put something else) so the levels resets, then I select again the EXP_ID column and now I can choose again between all the levels. I don't know why I have to do this, but it works though is not an elegant solution especially in a ShinyApp.

data_edit function does not work without RStudio IDE

Hi Dillon Hammil,

I am not a user of RStudio IDE, but I would like to test and use your package.
Unfortunately, when I try to use the function data_edit, I receive the error message that RStudio not running.

I try to change the viewer to the browser, but this change did not solve the problem. Looking through the vignette, I did not find that RStudio is a requirement or dependency for the package. Maybe I am doing something wrong. Could you help me with that?

Bellow, I send the Code that I am testing.

library(DataEditR)
data(mtcars)
data_edit(mtcars, viewer = "browser")
sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.3 LTS

Matrix products: default
BLAS: /usr/local/lib/R/lib/libRblas.so
LAPACK: /usr/local/lib/R/lib/libRlapack.so

locale:
[1] LC_CTYPE=pt_BR.UTF-8 LC_NUMERIC=C
[3] LC_TIME=pt_BR.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=pt_BR.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=pt_BR.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=pt_BR.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] grid stats graphics grDevices utils datasets methods
[8] base

other attached packages:
[1] DataEditR_0.1.3 RColetum_0.2.1 ggspatial_1.1.5
[4] rnaturalearthdata_0.1.0 rnaturalearth_0.1.0 paletteer_1.4.0
[7] grImport_0.9-3 XML_3.99-0.6 rgdal_1.5-23
[10] sp_1.4-5 tmaptools_3.1-1 tmap_3.3-2
[13] sf_1.0-2 ggforce_0.3.3 viridis_0.6.1
[16] viridisLite_0.4.0 DataExplorer_0.8.2 janitor_2.1.0
[19] readxl_1.3.1 ggrepel_0.9.1 patchwork_1.1.1
[22] cowplot_1.1.1 forcats_0.5.1 stringr_1.4.0
[25] dplyr_1.0.7 purrr_0.3.4 readr_2.0.0
[28] tidyr_1.1.3 tibble_3.1.3 ggplot2_3.3.5
[31] tidyverse_1.3.1

loaded via a namespace (and not attached):
[1] leafem_0.1.6 colorspace_2.0-2 ellipsis_0.3.2
[4] class_7.3-19 leaflet_2.0.4.1 snakecase_0.11.0
[7] base64enc_0.1-3 fs_1.5.0 dichromat_2.0-0
[10] rstudioapi_0.13 proxy_0.4-26 farver_2.1.0
[13] fansi_0.5.0 lubridate_1.7.10 xml2_1.3.2
[16] codetools_0.2-18 extrafont_0.17 knitr_1.33
[19] shinythemes_1.2.0 polyclip_1.10-0 jsonlite_1.7.2
[22] Rttf2pt1_1.3.9 broom_0.7.9 dbplyr_2.1.1
[25] png_0.1-7 shiny_1.6.0 compiler_4.1.0
[28] httr_1.4.2 backports_1.2.1 fastmap_1.1.0
[31] assertthat_0.2.1 cli_3.0.1 later_1.2.0
[34] tweenr_1.0.2 htmltools_0.5.1.1 tools_4.1.0
[37] igraph_1.2.6 gtable_0.3.0 glue_1.4.2
[40] Rcpp_1.0.7 cellranger_1.1.0 raster_3.4-13
[43] vctrs_0.3.8 extrafontdb_1.0 leafsync_0.1.0
[46] crosstalk_1.1.1 lwgeom_0.2-7 xfun_0.24
[49] networkD3_0.4 rvest_1.0.1 mime_0.11
[52] miniUI_0.1.1.1 lifecycle_1.0.0 MASS_7.3-54
[55] scales_1.1.1 shinyBS_0.61 promises_1.2.0.1
[58] hms_1.1.0 parallel_4.1.0 rematch2_2.1.2
[61] RColorBrewer_1.1-2 curl_4.3.2 gridExtra_2.3
[64] stringi_1.7.3 e1071_1.7-8 rlang_0.4.11
[67] pkgconfig_2.0.3 evaluate_0.14 lattice_0.20-44
[70] labeling_0.4.2 htmlwidgets_1.5.3 tidyselect_1.1.1
[73] magrittr_2.0.1 R6_2.5.0 generics_0.1.0
[76] DBI_1.1.1 pillar_1.6.2 haven_2.4.1
[79] withr_2.4.2 units_0.7-2 stars_0.5-3
[82] abind_1.4-5 modelr_0.1.8 crayon_1.4.1
[85] KernSmooth_2.23-20 utf8_1.2.2 rhandsontable_0.3.8
[88] tzdb_0.1.2 rmarkdown_2.9 data.table_1.14.0
[91] reprex_2.0.0 digest_0.6.27 classInt_0.4-3
[94] xtable_1.8-4 httpuv_1.6.1 munsell_0.5.0
[97] shinyjs_2.0.0

slow to update

Hi,

Great package. very intuitive and simple to use.

My issue however is when I load a large (ish) dataframe of ~1000 rows and 50 columns, when I use the complex user inputs such as check boxes or dropdown menus, it can take a while to register the input. It is only about a second but this is for each input and so makes its unrealistic to use for large datasets. Is there a way this could be sped up? hopefully there is a better fix but i suppose one quick fix would be to incorporate pagination?

Also, a fixed horizontal scroll bar would be nice (currently it is at the bottom of page).

Thanks!

My computer specs are:
Windows 10
intel xeon gold 6128, 6 core @3.4GHz
128GB RAM

Paste table also with headers

When pasting a table with headers into DateEdiR, the headers go to the first row. To move the headers to the headers of DateEditR may be too long, specially when there are many columns. So, it would be very convenient if some way of pasting with headers was implemented. Maybe with SHIFT+CONTROL+V ?

Thanks!

DataEdit as a function is not working

Hello,
When i try to save the result of the function, it does not work.

for example:

mtcars2 = DataEditR::data_edit(mtcars)

Will not save any changes in mtcars2.

Here is my session info:
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=Spanish_Peru.1252 LC_CTYPE=Spanish_Peru.1252 LC_MONETARY=Spanish_Peru.1252
[4] LC_NUMERIC=C LC_TIME=Spanish_Peru.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] forcats_0.5.1 stringr_1.4.0 dplyr_1.0.7 purrr_0.3.4 readr_2.1.1 tidyr_1.1.4
[7] tibble_3.1.6 ggplot2_3.3.5 tidyverse_1.3.1 shiny_1.7.1 editData_0.1.8

loaded via a namespace (and not attached):
[1] httr_1.4.2 sass_0.4.0 jsonlite_1.7.2 modelr_0.1.8 bslib_0.3.1
[6] assertthat_0.2.1 cellranger_1.1.0 yaml_2.2.1 pillar_1.6.4 backports_1.4.0
[11] glue_1.5.1 digest_0.6.29 promises_1.2.0.1 rvest_1.0.2 colorspace_2.0-2
[16] htmltools_0.5.2 httpuv_1.6.3 pkgconfig_2.0.3 broom_0.7.10 haven_2.4.3
[21] xtable_1.8-4 scales_1.1.1 fontawesome_0.2.2 openxlsx_4.2.4 later_1.3.0
[26] rio_0.5.29 tzdb_0.2.0 generics_0.1.1 ellipsis_0.3.2 DT_0.20
[31] cachem_1.0.6 withr_2.4.3 cli_3.1.0 magrittr_2.0.1 crayon_1.4.2
[36] readxl_1.3.1 mime_0.12 fs_1.5.1 fansi_0.5.0 xml2_1.3.3
[41] foreign_0.8-81 tools_4.1.2 data.table_1.14.2 hms_1.1.1 lifecycle_1.0.1
[46] munsell_0.5.0 reprex_2.0.1 zip_2.2.0 compiler_4.1.2 jquerylib_0.1.4
[51] rlang_0.4.12 grid_4.1.2 rstudioapi_0.13 htmlwidgets_1.5.4 crosstalk_1.2.0
[56] miniUI_0.1.1.1 shinyWidgets_0.6.2 gtable_0.3.0 DBI_1.1.1 curl_4.3.2
[61] R6_2.5.1 lubridate_1.8.0 fastmap_1.1.0 utf8_1.2.2 stringi_1.7.5
[66] Rcpp_1.0.7 vctrs_0.3.8 dbplyr_2.1.1 tidyselect_1.1.1

data_edit() does not appropriately handle empty datasets

We cannot set rownames on a zero row dataset, hence why the following example causes data_edit() to crash:

library(DataEditR)
test <- matrix(ncol = 10, 
               nrow = 0, 
               dimnames = list(NULL, paste0("A", 1:10)))
data_edit(test)

Need to do a check on row numbers when annotating rownames, perhaps we can add an empty row so that rownames can be set.

Using data_edit() to Modify Data in Databases

This is not yet documented on the website, but data_edit() can be used to modify data stored in databases. The trick is that most database writing functions expect a connection as their first argument, whilst DataEditR passes arguments to these functions in the following order: 1. data -> 2. save_as -> 3. write_args. The workaround is to write a wrapper function that fixes the order of the arguments before passing them to the writing function - naming the arguments is the best way to do this. See example below:

# Load required packages
library(DataEditR)
library(DBI)

# Create database connection
con <- dbConnect(
  RSQLite::SQLite(),
  dbname = ":memory:"
)

# Function to update database
dbWrite <- function(...) {
  # DataEditR argument order - data -> save_as -> write_args
  args <- list(...)
  names(args)[1:2] <- c("value", "conn")
  do.call("dbWriteTable", args)
}

# Edit Data
data_edit(
  mtcars,
  write_fun = "dbWrite",
  write_args = list("name" = "mtcars"),
  save_as = con
)

# Check database is updated
dbListTables(con)

# Read data from database
dbReadTable(con, "mtcars")

Print edited table in shiny app

Hello,
First, thanks for this beautiful package, it will help me a lot!
I'm building a Shiny App using your package and I'm struggling to figure out how to extract and save the table once edited.
I reproduced below a quick Shiny App:

library(DataEditR)

if(interactive()) {
  ui <- fluidPage(
    dataOutputUI("output-1"),
    dataEditUI("edit-1"),
    tableOutput("tab")
  )

  server <- function(input, output, session) {
    x <- data.frame(Age = c(10, 20, 1000), Weight = c(120, 131, 111))
    data_edit <- dataEditServer("edit-1",data = x)
    edited <- dataOutputServer("output-1",data = data_edit)
    output$tab <- renderTable(edited())
  }
  shinyApp(ui, server)
}

In this example, there is no table output and it throws the following error: Warning: Error in renderFunc: argument "shinysession" is missing, with no default. I tried the following alternative: output$tab <- renderTable(edited) but it was not working either.
Do you know how to deal with that?

[Feature Request] Text on icon mouseover?

Would it be possible to have text appear when you mouse over one of the buttons on the editor?

[Feature Request] Date picker as complex user input

Entering dates as text is hard and unreliable, it would be nice to have a calendar that the user can click on to select dates as with shiny::dateInput()

Adding Rows After Columns - FALSE

It seems that there may be an issue when rows are added after columns have been added - the values in the added columns change to FALSE. I suspect that this is a rhandsontable issue that is caused by the new row setting an NA in the created columns where a handsontable column type has been set. This doesn't seem to happen all the time, for example in my docker images with rhandsontable 0.3.7 everything works as expected.

If this becomes an issue for users I will need to look into the rhandsontable code to figure out why this is happening.

yeti is not an available theme

When I try to run data_edit() I get the error message:

Error in shinytheme(theme) :
yeti is not an available theme. Valid themes are: .

I cannot find any guidance to fix this error.

Are you able to help?

Changing row name crashes shiny

Try creating a duplicate row name to reproduce. Output:

open_toos_new = DataEditR::data_edit(open_tools)
Warning: non-unique value when setting 'row.names': ‘4’
Warning: Error in .rowNamesDF<-: duplicate 'row.names' are not allowed
  78: stop
  77: .rowNamesDF<-
  76: row.names<-.data.frame
  74: rownames<-
  73: observeEventHandler
   2: runApp
   1: DataEditR::data_edit

Loading Data from Within a Function

Looks like there are some environment issue with dataInput() when data_edit() is called inside another function. For example, no data is displayed with the following:

data_test <- function(){
  data <- mtcars
  data_edit(data)
}

I suspect this is because dataInput is searching for data in the wrong environment. I will need to set the search environment to the parent.frame() of data_edit() in order for this to work.

Exception if row add

test my example, right click, and try add any rows.

mytb <- mtcars[1:10,1:2]
mytb$colorcyl <- c("orange","green","red","blue", "black", "orange","green","red","blue", "black")[mytb$cyl]
data_edit(mytb,
col_options = list(colorcyl = c("red","blue","orange","green","black")))

Column editing is turned off to add dropdowns or checkboxes...
Предупреждение: Error in seq.default: argument 'length.out' must be of length 1
80: stop
79: seq.default
77: genRowHeaders
76:
74: hot_to_r
73: observeEventHandler
2: runApp
1: data_edit

[Save & Close] button only closes but does not save any changes

Hi Dillon,

# Make a backup DF m2,  to keep it safe... :-)
 m2  <- mtcars  

# Open m2  in Rstudio Viewer 
# and we change some values .ie: mpg from 21 to 2199 
data_edit( m2 )  

# Click on the [Save & Close] button
# and then do:
head(m2)

See?
No values have been changed
in m2 ...

The [Save & Close] button
will close the Rstudio Viewer,
but the m2 data frame
has not been changed any values after the edit...
The m2 values are the same as before.

(I would have expected my value edits
in m2 to persist after clicking [Save...]).

pls,
tell me if I misunderstood something basic.

thanks, Dillon!
SFd99

type.convert turns in certain cases numerics to integer, so that only integers are allowed when editing

Hello,
thank you for your package, which I started using recently.

I noticed the following problem: data.frames containing numeric columns are converted to integer if the values are not fractional.
A little bit of digging around showed that this is due to the usage of type.convert:

lst1 <- list(a = 4.0)
str(lst1)
# List of 1
# $ a: num 4

lst2 <- utils::type.convert(lst1)
str(lst2)
# List of 1
# $ a: int 4

In case of mtcars for example, this means that some columns, although initially of type numeric, are converted to type integer:

str(mtcars)
# 'data.frame':	32 obs. of  11 variables:
#   $ mpg : num  21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
# $ cyl : num  6 6 4 6 8 6 8 4 4 6 ...
# $ disp: num  160 160 108 258 360 ...
# $ hp  : num  110 110 93 110 175 105 245 62 95 123 ...
# $ drat: num  3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
# $ wt  : num  2.62 2.88 2.32 3.21 3.44 ...
# $ qsec: num  16.5 17 18.6 19.4 17 ...
# $ vs  : num  0 0 1 1 0 1 0 1 1 1 ...
# $ am  : num  1 1 1 0 0 0 0 0 0 0 ...
# $ gear: num  4 4 4 3 3 3 3 4 4 4 ...
# $ carb: num  4 4 1 1 2 1 4 2 2 4 ...

str(type.convert(mtcars))
# 'data.frame':	32 obs. of  11 variables:
# $ mpg : num  21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
# $ cyl : int  6 6 4 6 8 6 8 4 4 6 ...
# $ disp: num  160 160 108 258 360 ...
# $ hp  : int  110 110 93 110 175 105 245 62 95 123 ...
# $ drat: num  3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
# $ wt  : num  2.62 2.88 2.32 3.21 3.44 ...
# $ qsec: num  16.5 17 18.6 19.4 17 ...
# $ vs  : int  0 0 1 1 0 1 0 1 1 1 ...
# $ am  : int  1 1 1 0 0 0 0 0 0 0 ...
# $ gear: int  4 4 4 3 3 3 3 4 4 4 ...
# $ carb: int  4 4 1 1 2 1 4 2 2 4 ...

As a consequence, for example when the cyl column is edited with DataEditR, only integer values are accepted.

I fixed this locally by replacing the following lines of code from helpers.R:

    # MATRIX - SAME COLUMN CLASS
    if("matrix" %in% data_class) {
      data <- as.matrix(data)
    # DATA.FRAME - DIFFERENT COLUMN CLASSES
    } else {
      for (z in colnames(data)) {
        data[, z] <- type.convert(data[, z], as.is = !col_factor)
      }
    }

with

   # MATRIX - SAME COLUMN CLASS
    if("matrix" %in% data_class) {
      data <- as.matrix(data)
    # DATA.FRAME - DIFFERENT COLUMN CLASSES
    } else {
      for (z in colnames(data)) {
        if (!is.numeric(data[, z][1]))
        {
          data[, z] <- type.convert(data[, z], as.is = !col_factor)
        }
        else
        {
         #do not convert
        }
      }
    }

(I apologize, for technical reasons I'm not able to create a pull request.)

Would it be possible for you to consider this workaround, or do you see some possible issues with it?

I also had a look in the list of R bugs, to check if anybody posted something about type.convert. I found this:
https://bugs.r-project.org/show_bug.cgi?id=17979
which describes some questionable conversions of dates, timestamps and tseries. Not sure what the impact is for rhandsontable,
maybe adding a word of caution in the description would help the users.

Kind regards,
Angela

data_edit() from within a function?

df <- data_edit(df) works on the console (really nicely!). I would like to be able to call df <- data_edit(df) from within a function, but when I do that I get:

Warning in file(file, "rt") :
cannot open file 'df': No such file or directory
Warning: Error in file: cannot open the connection
62: file
61: read.table
60: read.csv
58: module
53: callModule
52: moduleServer
51: dataInputServer
50: server
Error in file(file, "rt") : cannot open the connection

This seems like a Shiny thing, but I don't know Shiny. Is there a way to use data_edit under program control?

Cannot hide upload/download options in Shiny

Firstly - thank you for creating this package, I have been searching for something like this for a long time! It is so easy to use with Shiny!

I have encountered the following minor issue:

hide = TRUE argument does not seem to hide the download/upload options when running dataEditServer() directly from shiny.

My app uses a separate download/upload option in a side panel, which works with the DataEditoR to show the uploaded data but I can't seem to disable/suppress the default view from above the table (see attached image).

data_edit crashes when synchronizing edited filtered data

When I load a data set into data_edit, create a filter and make an edit, if I then click synchronize data_edit crashes emiting 2 warnings:
Warning: non-unique values when setting 'row.names'
Warning: Error in .rowNamesDF<-: duplicate 'row.names' are not allowed

Making an edit and synchronizing without filtering does not cause a crash.

I want use it on my shiny project, but shiny-ini-shiny impossible..

what about a guided tour?

Hi, I'm building a very complex shiny app and I have some internal data that has to be editable in the future (like adding more rows). My first question is: can I upload and use an "updated .csv file" in order to update my internal data? (some .rda files that came from csv files) I didn't see an "upload button".

Then I have a suggestion for you. Your package looks amazing, but since my app will be used by people with no knowledge of R (and perhaps non-tech savvy people), your package could be a bit difficult to understand without an explanation. I also had to watch some youtube videos to understand some buttons. So what about adding a guided tour? Like with {cicerone} or {rintrojs} packages. With {rintrojs} you can also add only some "help buttons" if you don't like the guided tour idea.

[Feature Request] Alow to select which columns should be visible

It would be useful to have the option to hide some columns from the user but keep them available for posterior use, a use case for this is Id columns that will be used to interact with databases but they take unnecessary space in the UI

How can I fix "Error in type.convert.default"?

Hi, Mr. Hammill,

I installed DataEditR through the CRAN. Actually, at first it worked fine, but at some point I started receiving errors. My data frame (usually tibble format) is not displayed at the pop-up dialog window when I run the data_edit function. Though the pop-up appears a few seconds, it always fleetingly is shut with the following error message:

data_edit(my_data)
필요한 패키지를 로딩중입니다: shiny

Listening on http://127.0.0.1:7744
Warning: Setting row names on a tibble is deprecated.
Warning: Error in type.convert.default: invalid multibyte string at '<eb><85><b8>
  68: type.convert.default
  66: type.convert.data.frame
  64: data_format
  63: <reactive>
  47: data_update
  46: <observer>
   3: shiny::runApp
   2: runGadget
   1: data_edit

My session information is as followings:

> sessionInfo()
R version 4.1.1 (2021-08-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=Korean_Korea.949  LC_CTYPE=Korean_Korea.949    LC_MONETARY=Korean_Korea.949 LC_NUMERIC=C                 LC_TIME=Korean_Korea.949    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] shiny_1.6.0      optimx_2021-6.12 lmerTest_3.1-3   lme4_1.1-27.1    Matrix_1.3-4     hypr_0.2.2       EnvStats_2.4.0   bit64_4.0.5      bit_4.0.4       
[10] broom_0.7.9      psycho_0.6.1     psych_2.1.6      pastecs_1.3.21   gt_0.3.1         mmtable2_0.1.3   DataEditR_0.1.3  writexl_1.4.0    readxl_1.3.1    
[19] forcats_0.5.1    stringr_1.4.0    dplyr_1.0.7      purrr_0.3.4      readr_2.0.1      tidyr_1.1.3      tibble_3.1.4     ggplot2_3.3.5    tidyverse_1.3.1 

loaded via a namespace (and not attached):
 [1] nlme_3.1-152        fs_1.5.0            lubridate_1.7.10    httr_1.4.2          numDeriv_2016.8-1.1 bslib_0.3.0         tools_4.1.1         backports_1.2.1    
 [9] utf8_1.2.2          R6_2.5.1            DBI_1.1.1           colorspace_2.0-2    withr_2.4.2         tidyselect_1.1.1    mnormt_2.0.2        compiler_4.1.1     
[17] cli_3.0.1           rvest_1.0.1         xml2_1.3.2          shinyjs_2.0.0       rhandsontable_0.3.8 sass_0.4.0          scales_1.1.1        digest_0.6.27      
[25] minqa_1.2.4         shinyBS_0.61        pkgconfig_2.0.3     htmltools_0.5.2     dbplyr_2.1.1        fastmap_1.1.0       htmlwidgets_1.5.3   rlang_0.4.11       
[33] rstudioapi_0.13     jquerylib_0.1.4     generics_0.1.0      jsonlite_1.7.2      magrittr_2.0.1      Rcpp_1.0.7          munsell_0.5.0       fansi_0.5.0        
[41] lifecycle_1.0.0     yaml_2.2.1          stringi_1.7.4       MASS_7.3-54         grid_4.1.1          parallel_4.1.1      promises_1.2.0.1    crayon_1.4.1       
[49] miniUI_0.1.1.1      lattice_0.20-44     splines_4.1.1       haven_2.4.3         hms_1.1.0           tmvnsim_1.0-2       pillar_1.6.2        boot_1.3-28        
[57] reprex_2.0.1        glue_1.4.2          modelr_0.1.8        nloptr_1.2.2.2      vctrs_0.3.8         tzdb_0.1.2          httpuv_1.6.2        cellranger_1.1.0   
[65] gtable_0.3.0        assertthat_0.2.1    cachem_1.0.6        mime_0.11           xtable_1.8-4        pracma_2.3.3        later_1.3.0         shinythemes_1.2.0  
[73] ellipsis_0.3.2 

As long as I change my locale to "Sys.setlocale(category = "LC_ALL", locale = "English")", the data_edit function seems to work fine. However, the problem is then, that I can't see any Korean character strings in my data frame any more because my locale was set to English! How can I fix this problem? I greatly appreciate if you give me any useful advice. Thank you for your time.

Best,
Galaxy

[Bug] - When "Inserting column left/right" other column data is removed

1. Column data to the right is removed when I add more than one Column. It seems to only happen after I've already added one Column to the left or right. Pictured below with mtcars dataset

mtcars_new <- DataEditR::data_edit(mtcars,
                        save_as = "mtcars.csv",
                        write_fun = "write.table",
                        write_args = list(sep = ",",
                                          row.names = TRUE))

2. If I rename a column after entering in data, that variables' data is removed and I have to re-enter the data all over again.

Fantastic work on the package too and such great progress. Very excited by it. Keep up the great work.

Error: "cannot open the connection" when dataframe name contains a "."

The first line of the the following code works as expected.

data_edit(mpg)
mpg.1 <- mpg
data_edit(mpg.1)

However, when the dataframe name contains a "." the following error gets triggered:

Listening on http://127.0.0.1:6197
Warning in file(file, "rt") :
cannot open file 'mpg.1': No such file or directory
Warning: Error in file: cannot open the connection
61: file
60: read.table
59: read.csv
57: module
52: callModule
51: moduleServer
50: dataInputServer
49: server
Error in file(file, "rt") : cannot open the connection

Admittedly, I didn't check other special characters or such in the dataframe name, but I discovered in my own syntax, where I will sometimes use MyData, MyData.1, etc.... Not familiar enough with R, or reading library syntax to even begin to troubleshoot what might be going on here. Sorry.

Really like the package by the way.

No UNdo correction possible...

Hi Dillon,

DataEditR = GREAT and useful pkg!.
Very much needed...

But No UNdo correction is possible...

ie: (in mtcars DF example),

if I change the wt value of the 1st row
from: 2.62
to: 2.99 and press Enter.

and... -oops!- I made a mistake!

But there is no way to "undo"
the wrong value I entered,
(the word "undo" is grayed out
in the right-click dropdown menu).

Ctrl-Z
also does not correct the value(s).
Would be a nice shortcut to have
to UNdo mistakes...it's quite a popular, universal shortcut!.

Am I missing something?
Thanks!

SFd99
Rstudio 1.3.1056
R 4.0.2
Ubuntu Linux 20.04.4 64bits
==================

Bug if used within a model

Hi, rhandsontable is very useful for the types of models I work with. And in combination with modal dialogue, it makes the model input process very user friendly.

However, when an rhandsontable is rendered in a modal box, depending on the circumstance only a part of the table is rendered until the user clicks on the partially rendered table. Is this a bug, or am I using rhandsontable incorrectly? Please see super simple MWE code below. It renders the initial table just fine, but when the user makes a change to the table (for example inserting a row with data) and tries re-rendering it by clicking the "Show" action button, only part of the table is rendered until the user clicks on the table.

The first image shows the rendered table when first invoking the app - looks good. The second image shows the rendered table after having inserted a row/data and then clicking "Show" -- I only get a partial rendering of the table until clicking on it, and then the complete table pops up (as it should have done after the "Show" click). The third images shows the completely and correctly rendered table after inserting a 3rd row, after clicking "Show", and after clicking on the partially rendered table.

library(shiny)
library(rhandsontable)

ui <- fluidPage(actionButton("show","Show"), 
                actionButton("reset","Reset"))

server <- function(input,output,session){

  dat <- reactiveVal(data.frame(x=runif(2),y=runif(2)))
  
  observeEvent(input$show,{showModal(modalDialog(rHandsontableOutput("hot")))})
  
  observeEvent(input$reset,dat(data.frame(x=runif(2),y=runif(2))))
  
  output$hot <- renderRHandsontable(rhandsontable(dat()))

  } # close Server

shinyApp(ui,server)

Shiny Based Installation Error

Hi. I tried to install the library from CRAN and got this error:

Error: object ‘moduleServer’ is not exported by 'namespace:shiny'
Execution halted

I checked and that object is not in shiny 1.4.0. Are you using another version? I'm also running R 3.6.0 on Ubuntu 20.04.

Feature Request: "Data Validation" functionality

Hi @DillonHammill ,

I recently stumbled across your package in my search for an R solution for data entry/data capture. It seems like your package would largely fit the bill.

However, I don't see any capability for restricting the range of numerical input values as part of a "Data Validation" workflow when entering data (see e.g. https://datacarpentry.org/spreadsheet-ecology-lesson/04-quality-control/index.html for what I mean by "Data Validation"). The col_options argument seems mostly to address data validation for known lists of discrete values, but I see no way to restrict column type to integer or min/max values for numeric input.

Did I miss this somewhere in the documentation?

Where would you suggest I begin looking in your code to add this capability? I'm happy to collaborate/contribute code for this capability as it would greatly facilitate my own workflow.

Cannot hide upload fields in shiny

Unfortunately I have exactly the same problem as someone described before. The input menus cannot be hidden.

library(DataEditR)
library(shiny)

ui <- fluidPage(
  splitLayout(
    dataInputUI("input1"),
    dataOutputUI("output1"),
    actionButton("done", "Done!"),
    cellWidths = c("60%", "20%", "20%")
  ),
  dataEditUI("edit1")
)

server <- function(input, output, session) {
  
  # DATA INPUT
  data_input <- dataInputServer("input1", hide = TRUE)
  
  # DATA EDIT
  data_update <- dataEditServer("edit1",
                                data = data_input)
  
  # DATA OUTPUT
  dataOutputServer("output1",
                   data = data_update,
                   write_fun = "write.csv",
                   write_args = list(row.names = FALSE))
  

  # DATA RETURN
  observeEvent(input$done, {
    stopApp(data_update())
  })
  
}

shinyApp(ui, server)

Edits do not save

I updated from v. 0.0.5 and the edits I make using the "data_edit" function are no longer reflected in the data frame upon clicking the "done" button (previously the "Save and Close" button). With some testing, I was able to save the edits by clicking the refresh icon, then clicking "done", but that was unintuitive. I have R version 4.0.2 and tested the package alone with no other packages loaded. I tried using data_edit in the browser and in the standard pop-up.

Error in shinytheme(theme):

I have just installed the package and i tried using the data_edit function on my dataset. It keeps returning this error

Error in shinytheme(theme):
yeti is not an available theme. Valid themes are: .

PS: This package worked fine on a computer i was using previously. I am using a new one and i get this error message. I am not sure if it is a package issue or computer.

DataEditR aborts when one tries to paste more columns than the available

Suppose one removes all columns but one and, subsequently, tries to paste 3 columns from, say, a spreadsheet, DataEditR simply aborts. Should not DataEditR add more columns in order to accommodate the whole pasted data?

Writing to googlesheets

Hi @DillonHammill Thank you for creating this wonderful package! I wanted to know if it was possible to write and save the data from the shiny app to googlesheets instead of having to save it on our local storage.

data_edit() crashes when code is highlighted before running

This works:
Drag select entire line and run.

mydata <- data_edit(code = "D:/data_recreate2.R", read_fun = "read.csv2")

But this doesn't (i.e. breaking the line):
Drag select all lines and run.

mydata <- data_edit(code = "D:/data_recreate2.R", 
                    read_fun = "read.csv2")

It returns with error:

Warning in file(file, "rt") :
  cannot open file 'mydata <- data_edit(code = "D:/data_recreate2.R", 
read_fun = "read.csv2")': Invalid argument
Warning: Error in file: cannot open the connection
  61: file
  60: read.table
  59: read.csv2
  57: module
  52: callModule
  51: moduleServer
  50: dataInputServer
  49: server
Error in file(file, "rt") : cannot open the connection

it does run if I place the cursor on the first line (so not selecting anything) and then run.

Is that to be expected?

System:
windows 10
R 4.1.2
Rstudio 2021.9.1.372
DataEditR 0.1.4

Calling process terminates 30ms after exiting from DataEditR

Approximately 30ms after returning from the DataEditR window, the calling process terminates with no exception or apparent cause.

If I insert a browser() step immediately after return this does not happen.
If I step through in debug mode this does not happen.

iris <- iris
ro_cols <- c('Species')
edited_iris  <- DataEditR::data_edit(iris,
                                     col_edit = FALSE,
                                     col_readonly = ro_cols,
                                     col_names = FALSE,
                                     row_edit = FALSE,
                                     title = 'Iris data Set')
#browser()
for (i in seq(50)){
  print(sprintf('%d. %s', i, format(Sys.time(),'%H:%M:%OS3')))
  Sys.sleep(.01)
}

The loop at the end stops after 3 or 4 iterations. But if I enable the browser() then execution continues.
Any help on how to prevent this much appreciated.

Can not load the data.frame

I had installed the add-in with no problem. But when I tried to open a data frame this message appears:

Warning: Error in sub: input string 1 is invalid UTF-8
  51: sub
  50: mysub
  47: trimws
  46: <observer>
   3: shiny::runApp
   2: runGadget
   1: data_edit

So, I tried with another one and another similar message appears:

Warning: Error in warn: entrada en evaluacion: recursivo por defecto o problemas anteriores?
  67: warn
  66: row.names<-.tbl_df
  64: rownames<-
  63: <reactive>
  47: data_to_edit
  46: <observer>
   3: shiny::runApp
   2: runGadget
   1: data_edit

Different return value for cancel button

Hi @DillonHammill - I really want to know whether the user exited using the Done or the Cancel button. At present there doesn't seem to be any way to do this if they haven't actually changed some data.
Would it make sense to not return the data when the Cancel is pressed?
Or is there another way of determining this from the calling process?

The scenario for me is:

1. Get some data
2. Make some proposed changes
3. Use DataEditR for user to review or adjust
4. Either proceed with the (adjusted) changes (Done) or back out and start again (Cancel).

At present I can't distinguish 'Done' with no change from 'Cancel' - meaning back out.

Originally posted by @sch56 in #3 (comment)

Data Not Saving

When trying to save edits to a new dataframe nothing saves. Are you supposed to just hit done?

How to use all modules inside a shiny app

Hi Dillon, I'm developing another shiny app and now I need all the modules (filter,select,sync and so long). Could you explain me how I can use all the modules together? For example the correct order of the functions in the server part. I searched in every help and tried some combinations but nothing seems to work. I wrote this example but it doesn't work.

  library(shiny)
  library(rhandsontable)
  library(shinyjs)
  library(DataEditR)
  library(DT)

  ui <- fluidPage(
    useShinyjs(),
    DTOutput("test"),
    DTOutput("test2"),
    actionButton("editinternal", icon("edit")),
    tags$head(tags$style("#upinternalmodal .modal-dialog{ width:1300px}")),
    tags$head(tags$style("#upinternalmodal .modal-body{ min-height:1000px}")),
    shinyBS::bsModal("upinternalmodal", "Edit data", trigger = "editinternal", size = "large",
        dataFilterUI("filter1"),
        dataSyncUI("sync1"),
        DataEditR::dataOutputUI("output_internal_edit"),
        DataEditR::dataEditUI("edit1")
    )
  )
  
  server <- function(input,
                     output,
                     session) {
    
    data = reactive(return(ggplot2::mpg))
    
    output$test = renderDT(data())

    
    data_edit  <- DataEditR::dataEditServer("edit1", data = data(), col_names = FALSE)
    
    filtered = dataFilterServer("filter1", data = data_edit())
    
    data_sync <- DataEditR::dataSyncServer(
      "sync1",
      data = data(),
      data_subset = filtered,
    )
    
    output$test2 = renderDT({
      req(data_sync())
      data_sync()
    })
    
    DataEditR::dataOutputServer("output_internal_edit", data = data_sync())

  }
  
  shinyApp(ui, server)

[Feature Request] sliderInput as complex user input

Hi, if possible it would be nice to be able to use sliderInput as user input, this could work for example as a way to enforce a Likert scale.

Data validation

It would be great if you could

1. fix the data type for each variable before you begin entering data, or repair them later if guessed wrong
2. set valid ranges for each variable. i.e. systolic blood pressure in healthy adults, 70-160. Out of range values challenged, suggesting you change the range if this value is correct.
3. Set up allowed values for factor variables - drop-down to limit to only these.

Standard paid data entry expects errors on 1-2% of fields.
For every 100 fields, 1-2 errors. It adds up.
It is a big deal.
Lots of these could be prevented with fixed data types and value ranges.
Common to get race :
white, White, Caucasian, Black, black, African-American, African-american, aa, etc.

Option to not display rownames in editor

Based on use cases within CytoExploreR, it would be handy to have an option to control whether the rownames are displayed in the editor. Ideally the rownames would be present in the output but just not displayed to the user.

However, I suspect that a feature would only work if row_edit = FALSE - as adding/removing rows would require addition/removal of unique rownames from the original data.

Synchronise Data with Rows/Columns Added/Removed

At the moment, the sync button only works when the dimensionality of the data subset remains unchanged. For example, if you subset the data and add a new column this will throw an error when you attempt to sync the data to the original dataset. There is some complexity in adding support for this operation as we will need to know which of the original columns/rows remain and which ones have been added. This would require changing the return of dataEdit to include the row/column indices.