uberspot / opentriviaqa Goto Github PK

View Code? Open in Web Editor NEW

186.0 186.0 61.0 6.64 MB

A creative commons dataset of trivia questions and answers

License: Creative Commons Attribution Share Alike 4.0 International

Ruby 38.63% Python 61.37%

opentriviaqa's People

Contributors

Stargazers

Watchers

Forkers

davidmarek bbqchickenrobot pachacamac nke001 illperipherals dimpu codeniko radi-ratlh lokling davemachado fredefox a4vi2r ahtm cmbkla iwan-rg sandeepkurne cadelaney3 tagyro rohanthacker omaryasserali nikhil-vinove rococtz amygourlay geekytime shawenyao sfoster84 elesleon noatcards manuelliebchen fmbuthia subhadeepch eternalmasterjoe pyrocean anmolrattan007 a7madsab lozanobosch shuntgit ntls duplaja nht178 dgcacique planadecu cliffweng gabeweng muchaz crista pablo-cs iamprime nahtnam sumit-coder arshannon9 lisa-fink alasslbz leastchaos benknox9 lrp-viorise faztdm-yt poatty matthieudebeaucorps maelys-buhler stuartkehoe

opentriviaqa's Issues

information source

Is there a way to only get the information source?
For eg : From this question :
#Q In 1972 a company named Waco released the first handheld electronic game. What was its name?
^ Tic Tac Toe
A Cross
B Tic Tac Toe
C Tetris
D Microvision

If I want this text : In 1972, a company named Waco released the first handheld electronic game called Tic Tac Toe

Converting the .txt files to .csv

Hey, I required the data for my research work to build effective distractors.
I have written a fully functional code in python that could transform the txt to csv in any required format.
I want to contribute the code
Can you please open a pull request for the same
Thanks.
@uberspot @pachacamac @fredefox @codeniko @manuelliebchen

Missing Answer

https://github.com/uberspot/OpenTriviaQA/blob/master/categories/brain-teasers#L419-L424

Issue converting to json with converter.rb

Ruby script gives this error when trying to convert the data. I'm using ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]

converter.rb:24:in `strip!': invalid byte sequence in UTF-8 (Encoding::CompatibilityError)
        from converter.rb:24:in `stripAndEncode'
        from converter.rb:40:in `block (2 levels) in <main>'
        from converter.rb:38:in `open'
        from converter.rb:38:in `block in <main>'
        from converter.rb:27:in `each'
        from converter.rb:27:in `<main>'

So rather than try to get the ruby script to work I made a Python script which does effectively the same thing, so I thought I'd share it for anyone coming across the same issue.

import sys
import string
import json

ALPHABET = list(string.ascii_uppercase)

for file in sys.argv[1:]:
    print(file)
    with open(file, "r", encoding='ISO-8859-1') as f:
        lines = f.readlines()

    questions = []
    question = {}
    for line in lines:
        if line.startswith("#Q "):
            question["question"] = line[3:-1]
            question["category"] = os.path.basename(file)

        elif line.startswith("^ "):
            question["answer"] = line[2:-1]

        elif line[0] in ALPHABET:
            if not question.get("choices"):
                question["choices"] = []
                
            question["choices"].append(line[2:-1])

        elif line.strip() == "" and question != {}:
            questions.append(question)
            question = {}

    with open(f"{file}.json", "w", encoding="utf-8") as f:
        json.dump(questions, f, indent=4)

Unsupported characters encoding

When converting the md files to json symbols, like dollars, pounds, cents, power and more along with spanish, latin words are not in correct format:

Praseing problem

I have a problem with praising the text file s

do you have a JSON dump I can pay too

Missing Answer

https://github.com/uberspot/OpenTriviaQA/blob/master/categories/movies#L5201-L5206

Missing Answer

https://github.com/uberspot/OpenTriviaQA/blob/master/categories/celebrities#L18475-L18480

Questions in multiple lines not working with the text2csv.py

A few questions, found in the "brain-teasers" category have the questions text take multiples lines, like for exemple:

#Q There are two kinds of people who live on a mysterious island. They are the so-called Honestants, who always speak the truth, and the others are the Swindlecants, who always lie.

A visitor to the island went to a local pub and had a few drinks. When he was ready to leave he asked the bartender how much he owed. The bartender told him the total for his bill. It was quite expensive, so he asked the bartender if he spoke the truth. But the visitor did not hear the whispered answer so he asked a man sitting next to him about it. And the man said: The bartender said yes, but he is a big liar.

Was the bartender a Honestant?
^ No
A Yes
B No

This create this output:

Which make the csv unusable for these questions

Data out of pattern

In /categories/newest.txt some questions are not in pattern,
Some "#" needs to be deleted for cleaner data

Incorrect/Incomplete answers

I've ingested all the files and these 6 answers are unmatched. Happy to make a PR to fix them if you need

{
	"question": "What 1999 movie is this excerpt from?\nNarrator, Mewtwo: Today, Ash, Brock and Misty are taking a break from their * adventures. As always, Ash is determined to become a * master. And, as always, hes ready to endure any hardship, bear any burden... with strength, stamina and a will of steel.\nAsh Ketchum: Hungry. Need food.\nMisty: Oh, Ash! You havent done a thing all day!\nAsh Ketchum: Im too weak to work, Misty. I havent eaten since breakfast.\nBrock: This lunch is going to be just perfect for you, Ash. Its my lazy-boy no-chew stew.",
	"category": "movies",
	"answer": "Pokemo",
	"choices": ["Digimon: The Movie", "Pokemon 4Ever", "Digimon Racing", "Pokemon: The First Movie"]
}

{
	"question": "The story of Salvador Dali and his wife is the subject of this 2009 movie.",
	"category": "movies",
	"answer": "Dali",
	"choices": ["Dali  I: The Surreal Story", "Hello Dali", "Salvador and Gala:The Surreal Life", "Just Dali and Me"]
}

{
	"question": "From which of the Terminator movies is the popular catchphrase Hasta la vista, baby?",
	"category": "movies",
	"answer": "Terminator",
	"choices": ["Terminator 3: Rise of the Machines", "Terminator 2: Judgment Day", "The Terminator", "All of these"]
}

{
	"question": "Which of the Terminator movies was voted the eighth greatest science fiction movie in history by the American Film Institute in 2008?",
	"category": "movies",
	"answer": "Terminator",
	"choices": ["Terminator Salvation", "The Terminator", "Terminator 2: Judgment Day", "Terminator 3: Rise of the Machines"]
}

Needs trimming:

{
	"question": "David Bowie had a #1 hit ( U.S) in 1983 with which song?",
	"category": "music",
	"answer": " Lets Dance",
	"choices": ["Fame", "Ziggy Stardust", "China Girl", "Lets Dance"]
}

Needs trimming:

{
	"question": "Finish this slogan used against the GOP presidential candidate in 1884: Blaine, Blaine, James G. Blaine ... .",
	"category": "world",
	"answer": " ... the continental liar from the state of Maine",
	"choices": ["... what have we done to deserve such pain?", "... who first slew Abel and then murdered Cain", "... the continental liar from the state of Maine", "... who will sell all our possessions to the King of Spain"]
}

uberspot / opentriviaqa Goto Github PK

opentriviaqa's People

Contributors

Stargazers

Watchers

Forkers

opentriviaqa's Issues

Recommend Projects

Recommend Topics

Recommend Org