borismattijssen / information-retrieval Goto Github PK

To research:

• Check if source code is available from original paper
• Woosh IR Toolkit
• Dataset (TREC 2005 Robust Retrieval Track)
• APONE
• ANOVA
• Borlund's guidelines for the four topic setup. (A framework for evaluation of interactive information retrieval systems)
• Our level of reproducibility (https://www.acm.org/publications/policies/artifact-review-badging).

To ask:

• How much time do we have in class for the experiment?

• We should refer to our source code
• Dataset topics http://trec.nist.gov/data/robust/05/05.50.topics.txt
• Dataset relevance judegement: http://trec.nist.gov/data/robust/05/TREC2005.qrels.txt
• Dataset AQUAINT description: https://catalog.ldc.upenn.edu/LDC2002T31
• Elastic search tutorial: https://www.sitepoint.com/search-engine-node-elasticsearch/

• Can we use this source code from them? They even put up a live version of the system for us, which we could use: http://leifos.pythonanywhere.com. This would kinda take away the need to build any IR system.
• In the important dates section it says that week 3.6 - 3.8 are for participating in each other's experiments. When will ours be?
• Why would we use AQUAINT?
• Can we get your data set?
• Can we get some details about our participants? Primarily age.
• Wat do you think of our approach?

• Search back-end
  • Setup elastic instance
  • Index documents
    • Check docs: make sure we use the right IR techniques, or understand which techniques are used by default such that we can report on this
  • Setup autocomplete capabilities (low priority)
• oTree
  • Page for introduction
  • Page for search task
    • Keep track of the following measures:
      • Total task time
      • Queries issued per minute
      • Documents viewed per minute
      • Marked as relevant per minute
      • SERPs viewed per query
      • Documents viewed per query
      • View depth per query
      • Hover depth per query
      • Total time per document
      • Total SERP time per query
    • Explanation of task
    • Search bar
    • Query elastic with AJAX
      • Show results in page
    • Use pagination for multiple SERPs
    • Search result page
      • Title
      • Full text
      • Mark as relevant button