Found Issue, Going Next Course 'name' about educative.io_scraper

[anilabhadatta]

@sumant4ssm thank you for pointing out the issue.
Currently i am not well so i will try to fix it in few days.
If possible comment the latest get file name function and uncomment the previous one present just above.

[anilabhadatta]

@paletikoushik In my testing, i am not receiving this error.
Please answer the following?
Have you created a virtual env?
If yes, then did you install the modules from requirements.txt in your virtual env?

In my tool, I am using python-slugify instead of slugify. Maybe that is causing some issues.

One more note I should add, there is a bug in file naming as well. I will fix it today.

[paletikoushik]

running the commands with
python chromedriver.py
python educative_scraper.py instead of python3 chromedriver.py..... solved my issue it asked for selenium and slugify and i used python-slugify it worked like charm thanks for the scraper and many many thanks for your support.

[sumant4ssm]

Hi @anilabhadatta , I am having the same issue, how do we fix it?

[sumant4ssm]

Alright, I am not a Python expert but I debugged this problem and found the problem was with the below code

    print("Getting File Name")
    meta_script_selector = "script[type='application/ld+json']"
    metadata = driver.find_elements( By.CSS_SELECTOR, meta_script_selector)[0].get_attribute('innerHTML')
    metadata = json.loads(metadata)
    if course_folder:
        file_name = metadata['name']

Just note this line
metadata = driver.find_elements( By.CSS_SELECTOR, meta_script_selector)[0].get_attribute('innerHTML')
It uses the 0th element but in my case, it was 1st element( Found out after looking through the sources)

Changing the previous line to the below line fixes the problem

metadata = driver.find_elements( By.CSS_SELECTOR, meta_script_selector)[1].get_attribute('innerHTML')

I am not sure it's always one, but the one solution I could think of is to look through all the array elements if there is no name found.

by the way, @anilabhadatta , Thank you for this tool.

[sumant4ssm]

thanks @anilabhadatta . It was an easy fix and was able to fix it.

[paletikoushik]

@sumant4ssm thank you for pointing out the issue. Currently i am not well so i will try to fix it in few days. If possible comment the latest get file name function and uncomment the previous one present just above.

after commenting getting below error any help is greatly appreciated

ad Webpage Function
Checking Login Function
Checking for captcha Function...
Create Course Folder Function
Getting File Name
File Name Found
Found Issue, Going Next Course slugify() got an unexpected keyword argument 'replacements'
Script Execution Complete

[anilabhadatta]

@zerohope @sumant4ssm @paletikoushik Test it now, should be fixed. Clone the repo again in a fresh location.

[paletikoushik]

cloned the repo again in fresh location still getting below error, any help is greatly appreciated sir.

Driver Loaded

                        [Selected config: 0] Starting Scraping: 0, https://www.educative.io/courses/decode-coding-interview-cpp/qABQKQqWmmk

Load Webpage Function
Checking Login Function
Checking for captcha Function...
Create Course Folder Function
Getting File Name
File Name Found
Found Issue, Going Next Course slugify() got an unexpected keyword argument 'replacements'

[paletikoushik]

Ok sir, so I made mistake

after install requirements.txt I got selenium missing.
I did pip3 install selenium and after that got slugify missing and I used pip3 install slugify, so that is creating the problem sir now how to solve this issue sir. I tried to uninstall it by pip3 uninstall slugify and installing pip3 install python-slugify still getting same error. I tested it on mac and its working fine (as that was first time i got same error like selenium missing and slugify missing and I installed pip3 install python-slugify and its working. What should I do on my windows laptop any help is greatly appreciated.

[anilabhadatta]

@paletikoushik can you share your screen? Send me a meet link and ping me up

[anilabhadatta]

Welcome.