UnicodeEncodeError about export-saved-reddit

[netfortius]

Similar issue - but with the " character (u201c) found somewhere:

UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 17: ordinal not in range(128)

Edit 5/2/2017: pulled and tested the latest version - same issue, apparently somewhere else:

UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 83: ordinal not in range(128)

[admish]

Use Python 3 and this error will clear up.

If you have multiple versions of python installed use python3 export_saved.py to run. You might also need to install praw as well with pip3 install praw.

[admish]

@rachmadaniHaryono I just installed this today so have only ever used the latest commit. Running Python 2.7.12 and Python 3.4.2 on macOS 10.12.4. Python3 with praw 4.5 ran flawlessly while Python 2.7 gave the following error:

Version 4.4.0 of praw is outdated. Version 4.5.0 was released 7 days ago.
Traceback (most recent call last):
  File "export_saved.py", line 321, in <module>
    main()
  File "export_saved.py", line 315, in main
    save_saved(reddit)
  File "export_saved.py", line 283, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 265, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 242, in write_csv
    csvwriter.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 10: ordinal not in range(128)

[admish]

Yes, I cloned the repo.

By default, OS X comes with Python 2.7. Following the instructions in the README on Macs without knowing this, the script won't work. To anyone else reading this you can change the default version by editing your ~/.bash_profile to include the line alias python='python3' then relaunch your terminal.

If this would help, here is the latest entry in the log.

:~/Workspace/export-saved-reddit$ git log
commit 9841839e273bc8675bc7bb96112650957131fb6c
Merge: 8277d77 5d2ca4b
Author: Christopher Su <[email protected]>
Date:   Wed Apr 26 19:51:43 2017 -0700

    Merge pull request #38 from rachmadaniHaryono/feature/fix-csv-type-error

    fix: usr: TypeError when writing csv field

[admish]

Version 4.4.0 of praw is outdated. Version 4.5.0 was released 7 days ago.
Traceback (most recent call last):
  File "export_saved.py", line 329, in <module>
    main()
  File "export_saved.py", line 323, in main
    save_saved(reddit)
  File "export_saved.py", line 287, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 269, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 246, in write_csv
    csvwriter.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201d' in position 68: ordinal not in range(128)

[admish]

Traceback (most recent call last):
  File "export_saved.py", line 339, in <module>
    main()
  File "export_saved.py", line 333, in main
    save_saved(reddit)
  File "export_saved.py", line 297, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 279, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 256, in write_csv
    csvwriter.writerow(row.encode('utf8', 'ignore'))
AttributeError: 'list' object has no attribute 'encode'

 $ git log
commit c8cca7e5f2734d9afd41a05b75f470aafee0f1ce
Author: rachmadaniHaryono <[email protected]>
Date:   Thu May 11 03:32:14 2017 +0800

    chg: dev: fix variable typo

[admish]

Setting up a new one now, but also about to leave work so give me a couple hours. In the meantime email me a public key that I can add into the authorized_keys file, and when I have the login info setup I'll share it so you can login.

[rachmadaniHaryono]

Thank you @admish for commenting here. Did you use the recent commit with #38?

This could mean this issue can be closed alongside with already closed issue #37 without waiting response @netfortius and @nmandal.

[rachmadaniHaryono]

Did you come the repo or just download the export_saved file? If you clone the repo can you check which commit you are now?

I will add program version flag so people can easily recognize the version later.

Also python2.7 is actually not officially supported (and we haven't wrote any python requirement in readme, sorry for that).

But I will patch this python2.7 if the latest pr can't fix it.

• add program version
• add python requirement on Readme

[rachmadaniHaryono]

@admish can you check once again on python2.7 version?

e: checkout the branch on the pr and run it once again.

e2: actually on requirement it is already recommended to use python3

Requirements

• Python 3.x

i recommend @netfortius and @nmandal can use python3

the next version i may add an error if python2 is used.

[admish]

Same error.

[rachmadaniHaryono]

on the same line? 254 or 256?

[rachmadaniHaryono]

@admish can you try it once again?

the recent code c8cca7e

if there is any error there should be only on line number 254, 256, 263, or 265

[admish]

Still the same error.

Version 4.4.0 of praw is outdated. Version 4.5.1 was released 2 days ago.
Traceback (most recent call last):
  File "export_saved.py", line 329, in <module>
    main()
  File "export_saved.py", line 323, in main
    save_saved(reddit)
  File "export_saved.py", line 287, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 269, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 246, in write_csv
    csvwriter.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201d' in position 68: ordinal not in range(128)

The git pull was successful and this was on the latest merge.

## $ git log
commit 64a37f64037c3eec623ba16de962c7a5da35f1d5
Merge: 9841839 9d4d027
Author: rachmadani haryono <[email protected]>
Date:   Mon May 8 06:51:03 2017 +0800

    Merge pull request #42 from rachmadaniHaryono/feature/add-program-version

    chg: dev: add program version.

[rachmadaniHaryono]

yeah, but that isn't the pull request branch.

add my fork and checkout the branch.

git remote add rachmadaniHaryono https://github.com/rachmadaniHaryono/export-saved-reddit
git checkout rachmadaniHaryono/feature/fix-py27-writerow

this should be the result from git show

commit c8cca7e5f2734d9afd41a05b75f470aafee0f1ce
Author: rachmadaniHaryono <[email protected]>
Date:   Thu May 11 03:32:14 2017 +0800

[rachmadaniHaryono]

@admish dbe00b0

[admish]

Traceback (most recent call last):
  File "export_saved.py", line 339, in <module>
    main()
  File "export_saved.py", line 333, in main
    save_saved(reddit)
  File "export_saved.py", line 297, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 279, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 256, in write_csv
    csvwriter.writerow([x.encode('utf8', 'ignore') for x in row])
AttributeError: 'int' object has no attribute 'encode'

[admish]

Would you like me to setup an OS X VM for you?

[rachmadaniHaryono]

yes, if possible

e:@admish fb3a4b5

[netfortius]

Issue is back, at least on OSX. Unicode error, regardless of version of python or praw used. Tried two different machines, one with python2 & praw 4.5.1, and one with python3 and praw 5.0.1. Then upgraded praw on the python2 machine, to cover that combo - consistent error across all: "can't encode character '\xe0' in position 66: ordinal not in range (128)"

[lukepayne]

It looks like this issue wasn't resolved, so I have opened a PR.
@csu could you please take a look / merge if appropriate? Thanks!

[rachmadaniHaryono]

Iirc the 3 PRs are actually for this issue, yet nobody mention if the issue is already fixed by one of these

[lukepayne]

You're right, it looks like there are 2 other PR requests that are in various states of completeness. It would be good to have a fix merged in for this issue so that others don't have to run into it.