ewenme / geniusr Goto Github PK
View Code? Open in Web Editor NEWwork with data & lyrics from Genius
Home Page: https://ewenme.github.io/geniusr/
License: Other
work with data & lyrics from Genius
Home Page: https://ewenme.github.io/geniusr/
License: Other
In scrape_lyrics_url
(https://github.com/ewenme/geniusr/blob/master/R/lyrics.R#L90) , and subsequently (https://github.com/ewenme/geniusr/blob/master/R/lyrics.R#L31 in scrape_lyrics_id
), there is a line where you take the first element lyrics <- lyrics[1]
.
In the case below, this results in not grabbing all the lines, as they are separated by section
s.
geniusr
library(geniusr)
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
library(xml2)
library(rvest)
url = "https://genius.com/Game-of-thrones-the-red-woman-script-annotated"
res = scrape_lyrics_url(url)
dim(res)
#> [1] 52 4
We see the output has only 52 lines, but there a much more on the website.
lyrics[1]
lineBelow, we copy the same code, but comment out the lyrics <- lyrics[1]
line.
song_lyrics_url = url
session <- suppressWarnings(rvest::html(song_lyrics_url))
song <- rvest::html_nodes(session, ".header_with_cover_art-primary_info-title") %>%
rvest::html_text()
artist <- rvest::html_nodes(session, ".header_with_cover_art-primary_info-primary_artist") %>%
rvest::html_text()
lyrics <- rvest::html_nodes(session, ".lyrics p")
xml2::xml_find_all(lyrics, ".//br") %>% xml2::xml_add_sibling("p",
"\n")
xml2::xml_find_all(lyrics, ".//br") %>% xml2::xml_remove() %>%
head()
#> [[1]]
#> NULL
#>
#> [[2]]
#> NULL
#>
#> [[3]]
#> NULL
#>
#> [[4]]
#> NULL
#>
#> [[5]]
#> NULL
#>
#> [[6]]
#> NULL
lyrics <- rvest::html_text(lyrics)
length(lyrics)
#> [1] 11
We see here there are 11 elements to lyrics
, and the last one is still relevant:
cat(tail(lyrics, n = 1))
#> Several Night’s Watch brothers allied to ALLISER THORNE are pointing crossbows at the room that DAVOS and the other Night’s Watch brothers allied with JON SNOW are holed up. ALLISER THORNE approaches the door to the room with several men and knocks. The Night’s Watch brothers inside draw their sword and GHOST growls. DAVOS stands up and walks to the door. ALLISER THORNE knocks again.
#>
#>
#> ALLISER THORNE: Ser Davos, we have no cause to fight. We are both anointed knights.
#>
#>
#> DAVOS: Hear that, lads? Nothing to fear.
#>
#>
#> ALLISER THORNE: I will grant amnesty to all brothers who thrown down their arms before nightfall. And you, Ser Davos, I will allow you to travel south, a free man with a fresh horse.
#>
#>
#> DAVOS: And some mutton. I’d like some mutton.
#>
#>
#> ALLISER THORNE: What?
#>
#>
#> DAVOS: I’m not much of a hunter. I’ll need some food if I’m gonna make it south without starving.
#>
#>
#> ALLISER THORNE: We’ll give you food. You can bring the Red Woman with you if you like. Or you can leave her here with us, whichever you choose. But surrender by nightfall or this ends with blood.
#>
#>
#> DAVOS: Thank you, Ser Alliser. We’ll discuss amongst ourselves and come back to you with an answer.
#>
#> ALLISER THORNE and his men leave.
#>
#>
#> DAVOS: Boys, I’ve been running from men like that ll my life. In my learned opinion, we open that door --
#>
#>
#> NIGHT’S WATCHMAN #1: And they’ll slaughter us all.
#>
#>
#> NIGHT’S WATCHMAN #2: They want to come in, they’re gonna come in.
#>
#>
#> DAVOS: Aye, but we don’t need to make it easy for them.
#>
#>
#> NIGHT’S WATCHMAN #2: Edd is our only chance.
#>
#>
#> NIGHT’S WATCHMAN #1: It’s a sad fucking statement if Dolorous Edd is our only chance.
#>
#>
#> DAVOS: There’s always the Red Woman.
#>
#>
#> NIGHT’S WATCHMAN #1: What’s one redhead gonna do against 40 armed men?
#>
#>
#> DAVOS: You haven’t seen her do what I’ve seen her do.
#>
#> CUT TO: CASTLE BLACK - MELISANDRE’S CHAMBER
#>
#> MELISANDRE is sitting at the edge of her bed by a fire. She looks across the room at a small mirror standing on a table, then walks over to it. MELISANDRE gazes into the mirror, then disrobes. She removes her collar. The gemstone in its center glimmers. She places the collar on the table beside the mirror. The mirror’s reflection reveals that her appearance has changed to that of an en extremely elderly woman. MELISANDRE continues to stare at herself in the mirror, then walks back over to the bed and gets under the covers.
Thus, if we comment this out, we still get a tibble result, and it has the full data:
# lyrics <- lyrics[1] # removed this line
lyrics <- unlist(stringr::str_split(lyrics, pattern = "\n"))
lyrics <- lyrics[lyrics != ""]
lyrics <- lyrics[!stringr::str_detect(lyrics, pattern = "\\[|\\]")]
lyrics <- tibble::tibble(line = lyrics)
lyrics$song_lyrics_url <- song_lyrics_url
lyrics$song_name <- song
lyrics$artist_name <- artist
wanted_result = tibble::as_tibble(lyrics)
dim(res)
#> [1] 52 4
dim(wanted_result)
#> [1] 345 4
I don't know the rationale for that line, but it seems to be there are cases when it works and others that it may not work if things are separated. I'd send a PR with this, but I'm unsure as to the effects and wanted to get some feedback.
I've downloaded the latest version of the package to this day, created a user and registered a new app on the developers page for Genius. I have a client ID, password and access token. However, after loading the library, calling genius_token(T) and inputing my access token, whenever I try to pull some song lyrics I get an error saying "Error in get_song(song_id = 3039923) : Unauthorized (HTTP 401)."
I tried setting the API token directly with Sys.setenv(GENIUS_API_TOKEN = mytoken) or including the argument access_token = genius_token() in all my get requests, to no avail. I'm not really sure what to do from here within the app since there's no scopes option to add in the genius_token command. Thanks for all the help and hope this is useful to someone since I haven't found anyone with the same issue (some dude reported it and then claimed it solved spontaneously so no luck getting info there)
After installing both JosiahParry's geniusR:
devtools::install_github("josiahparry/geniusR")
library(geniusR)
and ewenme's geniusr in one session:
install.packages("geniusr")
library(geniusr)
R seems to only keep one of the two in my packages library /Library/Frameworks/R.framework/Versions/3.5/Resources/library, usually geniusr
. I noticed this because functions from geniusr seemed to work fine, but gives this error with geniusR::genius_album()
:
Error in mutate_impl(.data, dots) : Evaluation error: lazy-load database
'/Library/Frameworks/R.framework/Versions/3.5/Resources/library/geniusR/R/geniusR.rdb' is corrupt.
This is confirmed when I check the directory, when I restart the session, reinstall packages, even when I reinstall R and RStudio. Somehow, geniusr seems to be greedily removing geniusR's directory from the /library directory. Such an unexpected corner case but it was going to come up I'm afraid.
Version:
R.version
_
platform x86_64-apple-darwin15.6.0
arch x86_64
os darwin15.6.0
system x86_64, darwin15.6.0
status
major 3
minor 5.0
year 2018
month 04
day 23
svn rev 74626
language R
version.string R version 3.5.0 (2018-04-23)
nickname Joy in Playing
get lyrics for an album, given an album ID and/or an album name
Hi :)
I've begun to use the geniusr package, and I think it's great!
However I'm returned an error when I try to scrap lyrics for some artists songs.
Here it is an example:
library(geniusr)
library(tidyverse)
fortune <- search_song(search_term = "A fortune in lies",
access_token = genius_token()) %>%
filter(artist_name == "Dream Theater")
scrape_lyrics_id(song_id = fortune$song_id[1], access_token = genius_token())
This is the error returned
Error in bind_rows_(x, .id) :
Argument 5 is a list, must contain atomic vectors
This is the output for the R.version command in my computer
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 4.3
year 2017
month 11
day 30
svn rev 73796
language R
version.string R version 3.4.3 (2017-11-30)
nickname Kite-Eating Tree
Whenever I request the lyrics for a song, I only get a blank tibble instead of a tibble with the lyrics. This is also a problem with get_lyrics_search. Other functions like search_song and get_song work just fine though. The output is shown below. Thank you so much for your help.
get_lyrics_id(song_id=50158)
A tibble: 0 x 6
... with 6 variables: line , section_name , section_artist , song_name , artist_name , song_id
get_songs()
returns details of song relationships (e.g. sampled in/by), but it's a complex nested list. A function to return this list as a tidy data frame would be helpful.
Hello, when using the function get_lyrics_id, I get this error message in the R console:
Error in section_artist[is.na(section_artist)] <- artist :
replacement has length zero
I have tested it with multiple songs and always get the same result.
Thanks for your help.
Hi,
I want to collect multiple lyrics using a for loop:
N <- 8
df.ids <- paste("Runs", N, sep="")
for (i in 1:N) {
d.frame <- try(genius_lyrics(artist = A[i], song = B[i]))
assign(df.ids[i], d.frame)
}
It works fine for a single record, but when I want to collect multiple lyrics I get the following error:
Error in read_xml.response(x$response, ..., as_html = as_html) :
Not Found (HTTP 404).
In addition: Warning message:
In request_GET(session, url) : Not Found (HTTP 404).
Error : Column index must be at most 1 if positive, not 2
Error : Column index must be at most 1 if positive, not 3
Error : Column index must be at most 1 if positive, not 4
Error : Column index must be at most 1 if positive, not 5
Error : Column index must be at most 1 if positive, not 6
Error : Column index must be at most 1 if positive, not 7
Error : Column index must be at most 1 if positive, not 8
Any idea how to solve this?
Thanks!
Maurice
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.