Investigation: Errors when importing Rock The Vote posts. about chompy

[katiecrane]

@DFurnes so somehow when it goes to pass on that record, it isn't formatted correctly? Would it be helpful to add more logging to this job so we can try to pinpoint the rows that are failing?

[DFurnes]

We've had 10 failed runs of the ImportRockTheVotePosts job, likely from multiple attempts on a few CSVs. Here's the list of failed jobs:

Running php artisan queue:failed on ⬢ dosomething-chompy... up, run.9459 (Standard-1X)
+----+------------+----------+------------------------------------+---------------------+
| ID | Connection | Queue    | Class                              | Failed At           |
+----+------------+----------+------------------------------------+---------------------+
| 91 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-09 16:07:21 |
| 81 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-09 15:37:27 |
| 71 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-09 15:11:46 |
| 61 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-09 14:40:58 |
| 51 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-05 21:02:03 |
| 41 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-05 20:41:27 |
| 31 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-05 20:16:00 |
| 21 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-05 18:19:17 |
| 11 | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-05 18:10:51 |
| 1  | redis      | importer | Chompy\Jobs\ImportRockTheVotePosts | 2018-10-05 17:38:06 |
+----+------------+----------+------------------------------------+---------------------+

[DFurnes]

All of the jobs failed with the same issue, encoding a row for the CreateRockTheVotePostInRogue job:

Illuminate\Queue\InvalidPayloadException: Unable to JSON encode payload. Error code: 5 in /app/vendor/laravel/framework/src/Illuminate/Queue/Queue.php:83
Stack trace:
...
#6 /app/app/Jobs/ImportRockTheVotePosts.php(68): Chompy\Jobs\CreateRockTheVotePostInRogue::dispatch(Array)
#7 [internal function]: Chompy\Jobs\ImportRockTheVotePosts->handle(Object(Chompy\Services\Rogue))

This one is from rock-the-vote-importer2018-10-09 15:37:20.csv.

[DFurnes]

Another interesting thing is that we have some failed rows, like this one, which logged "There was an error with that user" and what looks like a validation error but did not fail the job (so we can't retry).

[DFurnes]

@katiecrane I think that's a good idea! We were trying to sleuth how far it might have gotten by looking at created user accounts, but more logging will definitely be useful here (maybe catching this InvalidPayloadException exception and logging the actual contents of the row before failing the job).

[katiecrane]

@DFurnes should we also remove the try/catch when grabbing or creating users so that it will end up in the failed table?

[DFurnes]

Oh man, here's a potential culprit! Jen found this bad-boy on row 34 of this CSV: ‚™¥_ñ 🙀

[DFurnes]

So two problems that we can fix right off the bat:

1. We should be catching any exceptions that occur when processing rows, logging details about them, and continuing to process the rest of a CSV in CreateRockTheVotePostInRogue.
2. If something goes wrong in CreateRockTheVotePostInRogue, we need to actually fail the job (not just log something and continue)! Right now we don't have a ton of insight into why individual users are failing & no straightforward way to re-try them later on if we wanted to.

[DFurnes]

@DFurnes should we also remove the try/catch when grabbing or creating users so that it will end up in the failed table?

Ah sorry, yes! Just refreshed the page and saw this. 👁 Yeah, we'd want either of these failures to actually stop the job from processing. We can also probably remove some of the extraneous isset() conditionals once we've done that (since we can trust the exception will stop the job).

[DFurnes]

Oh, Jen found another one – this time a first name of Ja_„ŽQuie. I wonder if this is an issue with Redis not handling special UTF-8 characters correctly. We ran into something similar in Bertly I think...

[DFurnes]

And D_„ŽJae! Man, who named these kids!?

[chloealee]

@DFurnes since we are switching to SQS instead of Redis, do you think these special UTF-8 characters will be handled correctly now?

[DFurnes]

@chloealee Yes, I believe SQS should handle these properly!

[chloealee]

@DFurnes yay great! PR coming soon for what you outlined here!