ropenscilabs / qcoder Goto Github PK

Is there any reason why there's a sudo: required on the travis build?
Each build is taking 20 minutes to complete, and they're spending over 90% of the time just installing packages, and these could be cached using Travis' options easily.

I am testing this on a fork, will create a PR if I see no problems with it.

For good documentation for users and submission to CRAN we will need to have examples for each exported function. This is part of the functions being able to run in the console. These are the same functions we need tests for.

The list of these functions matches the list of files in the docs folder.

• [BUG] 'Add data'>'Add new document'>'Select File' crashes upon choosing a document and hitting select.
  Error message: Warning: Error in <-: replacement has length zero reference an issue found in app.R line#350

• [SUGGESTION] 'Add data' tab allows for input of units and codes before user has selected 'my_qcoder_project' folder. Upon choosing the project folder, the codes and units are amended to the project. Should rather have the user add their project folder first before being able to do anything within 'Add data' tab.

• [SUGGESTION] 'Add new unit' field should not be allowed to be added when left blank.

This function does not work, it is only a small '/' that is missing somewhere

Add support of coding images ... you should be able to grab a section of an image and tag it.

Make a way to close qcoder that is more obvious and has a success message in the console.

This is supported by textreadr but it will require a different format than other documents.

Currently you can't display or download the document table in the App. Add a tab for this ... but make sure to subset so you don't try to display the whole text, just the ID and the code path.

The current version of parse_qcodes() will not parse a coded entry properly if the entry text contains the "]" character.

While cloning this repo, I was at a loss how to properly install and test it. R CMD check would not execute the tests as they should be, so I ended up copying a Makefile from an old project and just using it. It's relying on devtools for the steps, however it'd be helpful if this was on the project.

I'm opening this issue with a PR for that. It might not be the way that you're developing it currently however, so if anyone could point me as to how it'd be really helpful.

Thanks in advance!

Right now when setting up a project an empty data frame is created for codes. Let's do the same thing for the other data frames.

create_qcoer_project() should not overwrite existing files.

Shiny ace has a bug that prevents it from using shiny variables, fix it and sent them a PR.

Currently when a new code is added to the codes data frame the id numbers for codes can sometimes be changed. Fixe this.
There are examples of this in other update functions.

Right now codes that are not in the code data frame are updated when the coded data is created, but the data frame is not refreshed in the views of Codes and Summary.
Likely this needs to be fixed as part of moving the extraction of codes to on save.
(At least now the discovered codes are saving.)

This is more of a suggestion in case you have more Rmd files being read from github.
The only difference might be the need to use a slightly more updated version of rmarkdown, as per the docs.

I have used this locally to alter the README, as it creates a HTML preview for the file, so it's rendering correctly for it too.

We might want to add a document title in addition to the unique id so that people actually know what document we are selecting on the shiny drop down etc.

Think about good short cut keys for adding codes.

Currently the qcoder project defaults to being created in the working directory. Users should be required to specify the directory instead.

We need tests for our current code, which is not that many functions.

Such as

• codes + definitions

• summary

Here is some sample code.
https://shiny.rstudio.com/gallery/download-knitr-reports.html

There are two sample reports in the inst/rmarkdown/templates folder of the repository. They could be good to start with.

Nested codes with non same strings seem to be righly parsed :

"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. (QCODE) Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. (QCODE) Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. (/QCODE){#A}(/QCODE){#B}"

test.OK <- data.frame(doc_id = 1, document_text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. (QCODE) Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. (QCODE) Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. (/QCODE){#A}(/QCODE){#B}")

parse_qcodes(test.OK)

----------------------------------------------
 doc   qcode                text              
----- ------- --------------------------------
  1      B     Ut enim ad minim veniam, quis  
                nostrud exercitation ullamco  
               laboris nisi ut aliquip ex ea  
               commodo consequat.  Duis aute  
                irure dolor in reprehenderit  
               in voluptate velit esse cillum 
                   dolore eu fugiat nulla     
                  pariatur. Excepteur sint    
                   occaecat cupidatat non     
                proident, sunt in culpa qui   
                officia deserunt mollit anim  
                      id est laborum.         

  1      A        Duis aute irure dolor in    
                 reprehenderit in voluptate   
                velit esse cillum dolore eu   
                   fugiat nulla pariatur.     
                  Excepteur sint occaecat     
                cupidatat non proident, sunt  
               in culpa qui officia deserunt  
                mollit anim id est laborum.   
----------------------------------------------

But not when the strings are exactly the same (in the beginning) :

Here, the pasing function seems to pass QCODE and take too much text.

"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. (QCODE)(QCODE) Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. (/QCODE){#A}(/QCODE){#B}"

test.NOK <- data.frame(doc_id = 1, document_text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. (QCODE)(QCODE) Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. (/QCODE){#A}(/QCODE){#B}")

parse_qcodes(test.NOK)

----------------------------------------------
 doc   qcode                text              
----- ------- --------------------------------
  1      B      Lorem ipsum dolor sit amet,   
                consectetur adipiscing elit,  
                   sed do eiusmod tempor      
               incididunt ut labore et dolore 
               magna aliqua. Ut enim ad minim 
                    veniam, quis nostrud      
                exercitation ullamco laboris  
               nisi ut aliquip ex ea commodo  
                consequat.  Duis aute irure   
                 dolor in reprehenderit in    
                voluptate velit esse cillum   
                   dolore eu fugiat nulla     
                  pariatur. Excepteur sint    
                   occaecat cupidatat non     
                proident, sunt in culpa qui   
                officia deserunt mollit anim  
                      id est laborum.         

  1      A        Duis aute irure dolor in    
                 reprehenderit in voluptate   
                velit esse cillum dolore eu   
                   fugiat nulla pariatur.     
                  Excepteur sint occaecat     
                cupidatat non proident, sunt  
               in culpa qui officia deserunt  
                mollit anim id est laborum.   
----------------------------------------------

THis would involve capturing specific time points in a video and applying codes to them.

Implement auto pairing for (QCODE)(/QCODE).
https://ace.c9.io/#nav=higlighter

Add more statistics or visualizations to the summary tab.

Create a button to save the data frame of codes with coded text. (Possibly eventually in other formats also.)

Currently codes are read in from a csv file and new codes (but not descriptions) can be added while coding a document. This will add a way to add or edit a description and a way to create codes from a table view.

I have a problem when launching qcode (after import data). The program seems to try to find a file : "/data_frames/qcoder_documents_.rds" (without taking my project name ?).
Shiny app is launching, but it stay gray and I cannot select any folder.

Listening on http://127.0.0.1:7580
Warning in gzfile(file, "rb") :
impossible d'ouvrir le fichier compressé '/data_frames/qcoder_codes_.rds', cause probable : 'Aucun fichier ou dossier de ce type'
Warning: Error in gzfile: impossible d'ouvrir la connexion
75: gzfile
74: readRDS
73: observeEventHandler [/home/fred/R/x86_64-pc-linux-gnu-library/3.4/qcoder/shiny/qcoder/app.R#146]
2: shiny::runApp
1: qcode
Warning in gzfile(file, "rb") :
impossible d'ouvrir le fichier compressé '/data_frames/qcoder_documents_.rds', cause probable : 'Aucun fichier ou dossier de ce type'
Warning: Error in gzfile: impossible d'ouvrir la connexion
116: gzfile
115: readRDS
114: reactive:my_choices [/home/fred/R/x86_64-pc-linux-gnu-library/3.4/qcoder/shiny/qcoder/app.R#116]
98: my_choices
93: renderUI [/home/fred/R/x86_64-pc-linux-gnu-library/3.4/qcoder/shiny/qcoder/app.R#123]
92: func
79: origRenderFunc
78: output$choices
2: shiny::runApp
1: qcode

If you start the app and then select a folder that is not a qcoder project the application will crash. Probably the thing to do is to check for the data frames directory and then the needed files in the directory.

Parsing of the documents has now been pulled out to a separate function, parse_one_document(). However there are still many levels of loops which are not the ideal approach for many situations in R. The code is also very complex.

What should happen is to simplify this code through a combination of creating some separate functions and by using lapply() and its relatives when appropriate.

When there are warnings like the one where you try to import a document with an existing name, display them in the UI instead of/in addition to the console. Also an issue for adding duplicate named units, maybe some other places.

As of now the user would need to put in the path to the data to run the code to read in the raw data, also we would have the path to the created data. So we will want to add that to the shiny app.

Create options to put data into formats for use in Quantenda and TidyText .

Hi there,
trying the package on windows, it seems that the create_qcoder_project does not generate the qcoder_documents_my_qcoder_project.rds in the data-frame folder. I get this error in terminal when selecting the project in the app.

cannot open compressed file 'C:/Users/fvanw/Projects/qcoder-test/my_qcoder_project/data_frames/qcoder_documents_my_qcoder_project.rds', probable reason 'No such file or directory'

After loading a project, my app freeze when I try to add a new document. I saw a problem in the parameters of the call in the app, that are not as excepted by the function 'add_new_documents'.

292 qcoder::add_new_documents(files, doc_folder, docs_df_path)

The two last parameters have to be inverted.

292 qcoder::add_new_documents(files, docs_df_path, doc_folder)

I tried rapidly this, but it did not work. I used an alternative way, I will propose a new PR with it, more simple, but it works on my computer.

When in the application if you click the add selected code button but there is no code in the field, the application crashes. @laderast Any thoughts? Maybe we could test for an empty field and return or else make the button inactive if there is no code.

Ace allows us to implement our own syntax highlighters. We'd like to highlight text within (QCODE)(/QCODE) blocks.
https://ace.c9.io/#nav=higlighter

Gives some details.

In the qcoder shiny application I don't think that there is a way for creating a new folder to even be meaningful because we provide a structured project. Am I wrong?

Say a user has 3 different projects, how do we keep them straight and usable?
What does a project consist of?

• text files (aka the data) that all need to be in the same folder.
• code file, possibly stored as csv?
• data frames such as the coded data, data on units, mapping files.
• other?

We need to decide on a way to handle these and maybe have a function to initialize a project that creates all the folders, empty data sets and so on.

While aiming for MVP we should think of some basic analysis tools that we might want to include. As discussed at the unconf we don't want or need to have every bell and whistle but some items would be useful.

Ideas from me, please add more in comments.

• Dashboard with current status
• code-document matrix
• report template that shows all the text for a given code
• report template that shows some summary data like frequencies of codes.

Currently nested codes in a document don't work.

Some tabs show a table/data frame. Use DT or other add on to make the display dynamic by allowing filtering and sorting as well as possibly updating.

after qcode(), the user HOME folder is displayed for selection of qcoder projects. This is a fail for me because projects are not loaded within my HOME folder, they are on a server share in another location.

Hi there,
As stated in the readme : 'You must restart the qcode application to update those displays. This is a high priority development item.'

For me, it was linked to the adding of a new code, but during the process of coding. I think it could be great to have more reactivity. I had to re-open the project (select....) for seeing changes in tables / in the coded text.
I proposed a pull request linked to this issue. It may be not the ideal, but a statring point : an 'update' button right beside the 'select button'.

Twitter thread on the topic https://twitter.com/judell/status/999817415390216192

Overlapped coding is not supported.

Maybe need lot of codes : new adding_code / parsing_code functions.
It is treated by the parse_code as nested code.
Maybe use the marking of the code for the starting point can help.

Schematic issue :

xxxxxxxxxxx
\begin-code-A aaaaaaaaaaaaaaaa
\begin-code-B abababababababababab
\end-code-A bbbbbbbbbbbbbbbbbbb
\end-code-B
xxxxxxxxxxxxxx

give :
--> code-A : abababababababababababab - bbbbbbbbbbbbbbbbbbbbb
--> code-B : aaaaaaaaaaaaa abababababab bbbbbbbbbbbbbbbbbb

Qcoder is currently structured so that only one person can be coding a given project at a given time. However if two people have access to the same instance (or if the same person started two sessions) it would be possible for this to happen. We should prevent this.

One idea, when a project is selected in the shiny app temporarily rename the file(s) and then when qcode() is ended change the name back.

Hi there,
I started a project with sample as in Readme. I can select a project/file (sample)
I don't see how to add a 'code', there is no more 'sub-tab' in 'Codes' (I saw it existed before in the help).
I can added one in editing the text adding a new free text after the '#' during the 'coding' process.
I saw a new 'add data' for new doc and unit, but no for 'new code'

Right now we are using verbatim text and it's pretty ugly and unreadable. Switch to use html (will mean adding at least <p></p> tags and also consider converting QCODE tags to <mark></mark>. This is ideally a gsub use case I would think.

Currently if you try to add a new code to the field and press the button to add the codes, the code is not rendered after the #. Ideally we should be able to add a code on the fly. @laderast Any thoughts on this?