update process failed with exit code 110 about ttrss_ynh

[El-Gavy]

Same issue here for a lot of feeds (app version : 20230107~ynh1)

[ericgaspar]

It should be possible as multi_instance = true

[ericgaspar]

Is adding this to install and upgrade script do the job?

ynh_mysql_connect_as --user=$db_user --password="$db_pwd" --database=$db_name \
    <<< "ALTER DATABASE $db_name CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;"

[ballinger]

that also solved it for me. was not able to log into the db in the terminal, did it with myphpadmin. did not have the option for "utf8mb4" that was just the group name for all the utf8mb4's. i chose "utf8mb4_unicode_ci" which also works.

[arofarn]

Same here (Update process failed with exit code: 110) with some blogs and a lot of youtube channel (some are working but not the majority).

I tried without success :

• to lower the refresh rate to once per day from once per hour in case of "flood ban"
• reinstalling tt-rss with an import of the flux via OPML file (without tt-rss settings)

[arofarn]

In the feed debugger (LOG_EXTENDED, force refresh ON and force rehash ON), I have this king of message::

[07:36:02/913448] start
[07:36:02/913448] running HOOK_FETCH_FEED handlers...
[07:36:02/913448] feed data has not been modified by a plugin.
[07:36:02/913448] local cache will not be used for this feed
[07:36:02/913448] last unconditional update request: 
[07:36:02/913448] maximum allowed interval for conditional requests exceeded, forcing refetch
[07:36:02/913448] fetching https://www.youtube.com/feeds/videos.xml?channel_id=UCvDpfkniGG6WkX5L7DwyGaA (force_refetch: 1)...
[07:36:02/913448] [UrlHelper] fetching: https://www.youtube.com/feeds/videos.xml?channel_id=UCvDpfkniGG6WkX5L7DwyGaA
[07:36:02/913448] fetch done.
[07:36:02/913448] effective URL (after redirects): https://www.youtube.com/feeds/videos.xml?channel_id=UCvDpfkniGG6WkX5L7DwyGaA (IP: 172.217.18.206) 
[07:36:02/913448] server last modified: 
[07:36:02/913448] saving to local cache: cache/feeds/5908c6a723d24d06c96e83fd1acf4f0e0094b7f2.xml
[07:36:02/913448] running HOOK_FEED_FETCHED handlers...
[07:36:02/913448] feed data has not been modified by a plugin.
[07:36:02/913448] running HOOK_FEED_PARSED handlers...
[07:36:02/913448] language: simple
[07:36:02/913448] processing feed data...
[07:36:02/913448] site_url: https://www.youtube.com/channel/UCvDpfkniGG6WkX5L7DwyGaA
[07:36:02/913448] feed_title: Olivier Verdier
[07:36:02/913448] favicon: needs check: 1 is custom:  avg color: 
[07:36:02/913448] favicon: trying to update favicon...
[07:36:02/913448] [UrlHelper] fetching: https://www.youtube.com/channel/UCvDpfkniGG6WkX5L7DwyGaA
[07:36:02/913448] [UrlHelper] fetching: https://www.google.com/favicon.ico
[07:36:02/913448] favicon: https://www.google.com/favicon.ico looks valid, saving to feed-icons/46.ico
[07:36:02/913448] favicon: trying to calculate average color...
[07:36:02/913448] favicon: avg color: 
[07:36:02/913448] loading filters & labels...
Array
(
)
[07:36:02/913448] 0 filters loaded.
[07:36:02/913448] processing articles...
[07:36:02/913448] =================================================================================================================================
[07:36:02/913448] guid 2,yt:video:_UOM0Pz_Cgg (hash: {"ver":2,"uid":"2","hash":"SHA1:8538702c3e3ca19bdc132e689065614aaa4849be"} compat: SHA1:97fb5cb64a52095cc47c51c733c3441e60b905ce)
[07:36:02/913448] orig date: 1662320664 (2022-09-04 19:44:24)
[07:36:02/913448] title On fabrique 4 TRÉTEAUX aussi solides que pratiques 😉
[07:36:02/913448] link https://www.youtube.com/watch?v=_UOM0Pz_Cgg
[07:36:02/913448] language 
[07:36:02/913448] author Olivier Verdier
[07:36:02/913448] looking for tags...
[07:36:02/913448] tags found: 
[07:36:02/913448] done collecting data.
[07:36:02/913448] looking for enclosures...
[07:36:02/913448] article hash: 5cbaeebecfa1a6f7c6d3e57ddb01b1a4652694a1 [stored=]
[07:36:02/913448] hash differs, running HOOK_ARTICLE_FILTER handlers...
[07:36:02/913448] plugin data: 
[07:36:02/913448] matched filters: 
[07:36:02/913448] matched filter rules: 
[07:36:02/913448] filter actions: 
[07:36:02/913448] date: 1662320664 (2022/09/04 19:44:24)
[07:36:02/913448] num_comments: 0
[07:36:02/913448] article labels:
[07:36:02/913448] force catchup: 
[07:36:02/913448] base guid [2,yt:video:_UOM0Pz_Cgg or {"ver":2,"uid":"2","hash":"SHA1:8538702c3e3ca19bdc132e689065614aaa4849be"}] not found, creating...

It looks like an DB error when creating an none existing post entry ?

[Axehaft]

I have 4 feeds that are also showing the same error ... but seem to be updating normally. No clue what is causing this. If anybody needs more logs, let me know what you need and where to find it.

[Veehem]

I've many feeds with the same error

[Axehaft]

I suspect the YNH TTRSS version needs a new pull/update from the official ttrss site. It's beyond my skill/knowledge to do that though.

[ericgaspar]

@Axehaft ttrss_ynh as been updated today, we are using the latest pull in master branch...

[Thatoo]

It apparently didn't solve any problem I have been experimenting lately.

[Axehaft]

I updated as well ... still there for me too.

[Axehaft]

The Error 110 is shown here in the code: https://dev.tt-rss.org/fox/tt-rss/src/branch/master/update.php#L247

I still have no clue what it means though.

[Axehaft]

This issue continues to be unresolved. I asked the TTRSS maintainer for help:

"code 110 means there was an underlying exception during feed update, start looking in the event log.
since you’re using an unsupported setup of some nonspecific version i won’t be able to help you any further."

So I can't get help from him. It seems to be something on the YNH version of TTRSS as there is no mention of any issue on the main TTRSS forum. Interesting note: I get the same issue if I use RSS Bridge to create a feed of the sites with errors as well. So maybe something when fetching new content?

Each one that fails seems to stop here:

[13:51:38/450421] force catchup:
[13:51:38/450421] base guid [2,https://www.instagram.com/p/Cj8TxoLgFrB/ or {"ver":2,"uid":"2","hash":"SHA1:5521392c4585a5e43d61a3ec825fc66b216aa707"}] not found, creating...

[13:50:45/558284] force catchup:
[13:50:45/558284] base guid [2,https://www.youtube.com/watch?v=dafoHfQFnwQ or {"ver":2,"uid":"2","hash":"SHA1:4ae1269ef9fc7814cf6c9e1bf1ac0b95bb144f6b"}] not found, creating...

[13:53:57/450424] force catchup:
[13:53:57/450424] base guid [2,https://t.me/ChefPeteEvans/12697 or {"ver":2,"uid":"2","hash":"SHA1:e5f18f3f88bd32e0e0b16f110a13e51e2866960a"}] not found, creating...

Each log stop at this point. Does that help anyone in figuring this out?

[Veehem]

I'm also desperately waiting for a solution... and having the same problem with feeds coming from RSS Bridge

[Polonco]

I'm having the same problem with some of the feeds. Anybody found a fix or a workaround?

Following the error message displayed in /ttrss/prefs.php?tab=system when I force refetch and rehash for that feed.

`

Error	Filename	Message	User	Date
E_USER_WARNING (512)	/var/www/ttrss/classes/rssutils.php:1027	SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x98\x88" ...' for column ttrss.ttrss_entries.content at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main}		18:06
E_USER_WARNING (512)	/var/www/ttrss/classes/rssutils.php:1027	SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x8E\xA5 a...' for column ttrss.ttrss_entries.content at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main}		18:06
E_USER_WARNING (512)	/var/www/ttrss/classes/rssutils.php:1027	SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x98\x88 1...' for column ttrss.ttrss_entries.content at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main}		18:06
E_ERROR (1)	classes/rssutils.php:1027	Uncaught PDOException: SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x91\x80</...' for column ttrss.ttrss_entries.content at row 1 in /var/www/ttrss/classes/rssutils.php:1027 Stack trace: #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/classes/feeds.php(753): RSSUtils::update_rss_feed() #2 /var/www/ttrss/backend.php(136): Feeds->updatedebugger() #3 {main} thrown Remote IP: REMOVED.PUBLIC.IP.ADDRESS Request URI: /ttrss/backend.php User agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/109.0	admin	18:06
E_USER_WARNING (512)	/var/www/ttrss/classes/rssutils.php:1027	SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x92\xA6" ...' for column ttrss.ttrss_entries.content at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main}		18:06
E_USER_WARNING (512)	/var/www/ttrss/classes/rssutils.php:1027	SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x8E\x89\xE2\x9A...' for column ttrss.ttrss_entries.content at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main}		18:06
E_USER_WARNING (512)	/var/www/ttrss/classes/rssutils.php:1027	SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x8C\x88" ...' for column ttrss.ttrss_entries.content at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main}		18:06
E_USER_WARNING (512)	/var/www/ttrss/classes/rssutils.php:1027	SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x91\x80</...' for column ttrss.ttrss_entries.content at row 1 #0 /var/www/ttrss/classes/rssutils.php(1027): PDOStatement->execute() #1 /var/www/ttrss/update.php(238): RSSUtils::update_rss_feed() #2 {main}

`

[nathanael-h]

Looking at this issue, don't you think it might be caused by emojis in title, or text?
\xF0\x9F\x91\x80 is 👀 eyes emoji

Looking at sources, I see that the character set used it UTF8 https://dev.tt-rss.org/tt-rss/tt-rss/src/branch/master/sql/mysql/schema.sql#L2

I know that Nextcloud introduced a change to fix a related issue : https://docs.nextcloud.com/server/latest/admin_manual/configuration_database/mysql_4byte_support.html

I'll try to use the official docker-compose way of deploying ttrss and see if I can reproduce it. Maybe it is an upstream issue.

[Thatoo]

I don't know but the problem appear suddenly after upgrading to this version 20220626~ynh1, I never had this issue in the previous 5 years and now I have it for plenty of rss feeds, so many that ttrss became almost useless to me.

[nathanael-h]

Yeah, the same for me. I was kinda thinking "wow I catch up all the non read posts I had for months" and then realized that I didn't read any post from certain feeds for weeks 😮

So I started the docker-compose stack locally, and there is no problem retrieving feed with emoji, whereas in the ttrss_ynh app.

Here is an eg with Matrix.org blog feed :

So, i found few pages, threads and commits about this :

• https://community.tt-rss.org/t/pdo-is-coming-heres-what-you-need-to-know/689/88
• https://stackoverflow.com/questions/1049728/how-do-i-see-what-character-set-a-mysql-database-table-column-is
• https://dev.tt-rss.org/tt-rss/tt-rss/commit/102a01354b4f872a12feba3c9e875d9b0cd9dc36 special fix for MySQL db

Another thing I noticed, is that the recommended db is PostgreSQL, whereas we use MariaDB. Maybe, switching to PgSQL might help. This non yet merged, but still updated, PR in Nextcloud_ynh repo might guide us : YunoHost-Apps/nextcloud_ynh#125

Maybe a change of character set might fix the issue, but I don't know enough things about db and encoding.

What do you think?
Ping @ericgaspar

[ericgaspar]

There is already a PostgreSQL branch https://github.com/YunoHost-Apps/ttrss_ynh/tree/PostgreSQL. I am not concidering doing a migration script for migrating from Mariadb to PostgreSQL as the option to export OPML, remove TTRSS and reinstall TTRSS with PostgreSQL + reimport OPML file seems to be the easiest option.
I'm curious to hear what you think of migrating to PostgreSQL like this.

[nathanael-h]

I saw this PR changing the carset in config.php file #132 Shouldn't we have a script to change the carset in exsiting installation while upgrading?

Oh wow, super this #150 PR :) I think this is a good path, I'd suggest we also use the data migration plugin, to migrate user data (cf. https://tt-rss.org/wiki/FAQ)

1. export OPML (is it possible to export one global OPML, or is it per user? I did not find)
2. export per user data using https://dev.tt-rss.org/tt-rss/ttrss-data-migration/wiki
3. delete TTRSS
4. install TTRSS using PgSQL
5. import OPML
6. import per user data

[Veehem]

Anyway if there is a solution (I'm also waiting), please give a clear way to manage it. Everybody is not code fluent

[arofarn]

Is it possible to install both "old" ttrss with MariaDb and the new ttrss with postgreSQL on the same yunohost instance for testing without removing what works partially well ?
Or will it collide and crash the whole thing ?

[arofarn]

It didn't go as expected : the old TTRSS became unreachable (redirect to yunohost main page) and I had to uninstall both before restoring my old TTRSS (with MariaDb) from a backup...
As I didn't export the OPML before the installation of TTRSS with PostgreSQL, I stop here for today.

[loulecrivain]

Hello,
I've done some debugging and apparently this is a collation problem.
The app is configured to use UTF8MB4 collation, which is also the case for the database, right...

But ! All the tables are configured as utf8_general_ci (excerpt for ttrss_entries)

+---------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------+------------------+-----------+
| Name          | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time         | Check_time | Collation       | Checksum | Create_options | Comment | Max_index_length | Temporary |
+---------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------+------------------+-----------+
| ttrss_entries | InnoDB |      10 | Dynamic    | 1225 |           6272 |     7684096 |               0 |       327680 |  25165824 |           4794 | 2023-06-24 00:46:58 | 2023-06-30 12:08:11 | NULL       | utf8_general_ci |     NULL |                |         |                0 | N         |
+---------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------+------------------+-----------+
1 row in set (0,000 sec)

which are responsible for the PDOExceptions.

@ericgaspar do you want me to do a PR ? not sure how we should fix this.

[loulecrivain]

hello,
source of the bug is now confirmed. I ran the following command in mariadb:

ALTER TABLE ttrss_entries CONVERT TO CHARACTER SET 'utf8mb4' COLLATE 'utf8mb4_general_ci';

and this fixed all my feeds. feel free to @ me if you need more debugging info.
thank you.

[loulecrivain]

@ericgaspar that'd be quite the hack, but it could be an ok first fix given how longstanding this issue has been.

I'll fork the repo to do an install in a fresh vm and let you know

[loulecrivain]

@ericgaspar I tested with your latest release, since you made the changes before me. charset and collation change is not cascading, the tables are still in the utf8_general_ci collation and utf8 charset.

so the bug is still there. I'd like to do a PR for you but I don't have the time atm. is it okay if I come back to it in say, 2 weeks or so ?

[arofarn]

I did the same as ballinger with phpMyAdmin : I choose "utf8mb4_unicode_ci" instead of "utf8mb4_general_ci" and checked to apply to all the tables and all the columns in the tables.

That solves all the "code 110" fails.

[ericgaspar]

Closing following the transition to PostgreSQL