Pure PHP Implementation of MySQL replication protocol. This allow you to receive event like insert, update, delete with their data and raw SQL queries.
License: MIT License
Hi,
When I run php example.php,the Exception occured.
PHP Fatal error: Uncaught exception 'MySQLReplication\BinLog\Exception\BinLogException' with message 'Unknown command' in /var/www/vendor/krowinski/php-mysql-replication/src/MySQLReplication/BinLog/BinLogSocketConnect.php:190
Could support doctrine/dbal to 3.x ?
Please provide the following details.
SELECT VERSION();): 服务器版本： 5.5.11-log - Source distributionSteps required to reproduce the problem.
git clone https://github.com/huangdijia/lumen-trigger-demophp artisan trigger:startExpected Result.
=== Event heartbeat === \n
Date: 1970-01-01T00:00:00+00:00\n
Log position: 218837\n
Event size: 33\nActual Result.
Normal on other version of mysql
Greetings,
I've found too many of the properties and methods are private for me to usefully use or extend your library. When I want to make a quick change (i.e. backport ConstFieldType::DATETIME) into 1.x, I find that it takes an immense amount of work with respect to copy and pasting code just so i have access to the rowevent object that originally extended yours but slightly more logic in it.
Also, I do not want to use reflection but even if I did maintaining it is not tenable due to the sheer number of private properties.
I would argue the purpose of a library is to allow some modification but I cannot find that spirit in this library and the hopes I have for the future require me to hook into things such as getColumnData or tablemap, etc. It's a shame because this is really nice code but it's still not worth effort.
Even the config builder is too much of a burden to extend as the user that has permissions to repl data does not have access to information_schema.columns and I'd like to have it use two different connections for that but extending that requires extending all config/config* files, as well as repositories and factory file and the kicker is I need to copy and paste ALL of the code from those files because their properties are private!
Also the requirement of symfony 3.1 is tough, many libraries require 2 series and prevents an upgrade to your php-mysql-replication 2.x because of this. I understand some libraries need to lead the charge into newer symfonys, but this also prevents adoption.
Please provide the following details.
SELECT VERSION();): 5.7.27-30-log - Percona Server (GPL), Release 30, Revision 8916819--
Steps required to reproduce the problem.
Expected Result.
=== Event update ===
Date: 2019-09-06T14:38:28+08:00
Log position: 612816124
Event size: 394
Table: t8_ware_mall
Affected columns: 25
Changed rows: 1
Values: Array
(
[0] => Array
(
[before] => Array
(
[id] => 2346293598
[ware_title] => 7250綠鑽=2500元 🌟C羅🌟可提供儲值帳單★正規代儲★所有面值都可儲
[ware_price] => 2500
[original_price] => 2500
[ware_number] => 994
[ware_desc] =>
[ware_validity] => 1568711354
[action_time] => 1566997245
[post_time] => 1544003695
[refresh_time] => 1566997245
[sell_user] => 2098159
[buy_user] => 0
[ware_type] => 8
[statu] => 0
[delete_time] => 1568711354
[is_auto] => 0
[game_id] => 22784
[server_id] => 22785
[card_id] => 1
[deliveryTime] => 5
[ware_currency] =>
[currency_company] => 0
[validityTime] => 30
[top_status] => 9999999
[mobile_status] => 9999999
)
[after] => Array
(
[id] => 2346293598
[ware_title] => 7250綠鑽=2500元 🌟C羅🌟可提供儲值帳單★正規代儲★所有面值都可儲
[ware_price] => 2500
[original_price] => 2500
[ware_number] => 993
[ware_desc] =>
[ware_validity] => 1568711354
[action_time] => 1566997245
[post_time] => 1544003695
[refresh_time] => 1566997245
[sell_user] => 2098159
[buy_user] => 0
[ware_type] => 8
[statu] => 0
[delete_time] => 1568711354
[is_auto] => 0
[game_id] => 22784
[server_id] => 22785
[card_id] => 1
[deliveryTime] => 5
[ware_currency] =>
[currency_company] => 0
[validityTime] => 30
[top_status] => 9999999
[mobile_status] => 9999999
)
)
)
Memory usage 4.59 MB
Actual Result.
=== Event update ===
Date: 2019-09-06T14:41:46+08:00
Log position: 229744086
Event size: 56
Table: t8_ware_mall
Affected columns: 25
Changed rows: 1
Values: Array
(
[0] => Array
(
[before] => Array
(
[id] => 2349386924
[ware_title] =>
[ware_price] =>
[original_price] =>
[ware_number] =>
[ware_desc] =>
[ware_validity] =>
[action_time] =>
[post_time] =>
[refresh_time] =>
[sell_user] =>
[buy_user] =>
[ware_type] =>
[statu] =>
[delete_time] =>
[is_auto] =>
[game_id] =>
[server_id] =>
[card_id] =>
[deliveryTime] =>
[ware_currency] =>
[currency_company] =>
[validityTime] =>
[top_status] =>
[mobile_status] =>
)
[after] => Array
(
[id] =>
[ware_title] =>
[ware_price] =>
[original_price] =>
[ware_number] => 934
[ware_desc] =>
[ware_validity] =>
[action_time] =>
[post_time] =>
[refresh_time] =>
[sell_user] =>
[buy_user] =>
[ware_type] =>
[statu] =>
[delete_time] =>
[is_auto] =>
[game_id] =>
[server_id] =>
[card_id] =>
[deliveryTime] =>
[ware_currency] =>
[currency_company] =>
[validityTime] =>
[top_status] =>
[mobile_status] =>
)
)
)
Memory usage 4.6 MB
Please provide the following details.
SELECT VERSION();): <5.7.21-log >Steps required to reproduce the problem.
CREATE TABLE `cache` (
`key` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`value` varchar(16100) COLLATE utf8mb4_unicode_ci NOT NULL COMMENT 'memory引擎不支持text',
`expiration` int(11) NOT NULL,
UNIQUE KEY `cache_key_unique` (`key`)
) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
insert cache values ('laravel_cache5c785c036466adea360111aa28563bfd556b5fba','i:1','-1942153413')
3.
update cache set key='laravel_cache5c785c036466adea360111aa28563bfd556b5fba', value='i:2', expiration='1586248983'
Expected Result.
Date: 2020-04-07T16:42:03+08:00
Log position: 67042
Event size: 168
Table: cache
Affected columns: 3
Changed rows: 1
Values: Array
(
[0] => Array
(
[before] => Array
(
[key] => laravel_cache5c785c036466adea360111aa28563bfd556b5fba
[value] => i:1
[expiration] => -1942153413
)
[after] => Array
(
[key] => laravel_cache5c785c036466adea360111aa28563bfd556b5fba
[value] => i:2
[expiration] => 1586248983
)
)
)
Actual Result.
Date: 2020-04-07T16:42:03+08:00
Log position: 67042
Event size: 168
Table: cache
Affected columns: 3
Changed rows: 1
Values: Array
(
[0] => Array
(
[before] => Array
(
[key] => laravel_cache5c785c036466adea360111aa28563bfd556b5fba
[value] => i:1
[expiration] => -1942153413
)
[after] => Array
(
[key] => laravel_cache5c785c036466adea360111aa28563bfd556b5fbai:2;=?^?%9
[value] =>
[expiration] =>
)
)
)
is it possible to analyse binary logs without database (socket) connection?
Which classes I can use for?
Maybe with manual column name mapping.
I have set GBK，
$binLogStream = new MySQLReplicationFactory(
(new ConfigBuilder())
.............
->withCharset('gbk')
->build()
);
but error still came out at
[ErrorException]
unpack(): Type I: not enough input, need 4, have 0
but when use utf8
it's
[ErrorException]
unpack(): Type i: not enough input, need 4, have 3
php 5.6 , centos 7 64bit mysql 5.6.16
Please provide the following details.
SELECT VERSION();): 8Steps required to reproduce the problem.
Read all the events
Expected Result.
Each event should have a unique id
Actual Result.
Please provide the following details.
SELECT VERSION();): 5.6.10-logSteps required to reproduce the problem.
Sometimes, I have got the following error "No error information" from the stack trace "/src/MySQLReplication/BinLog/BinLogSocketConnect.php(63): MySQLReplication\Socket\Socket->readFromSocket()"
Expected Result.
Actual Result.
I couldn't find some information to reproduce the problem.
my table has a json field
Is there an config to parse binlog from mysql with charsets gbk?
Hello there,
We've got a running architecture of 1 primary (let's call it Bob) replicating to 10 replicas (new wording for master / slaves).
I'd like to watch for insert / update events with your package, but Bob currently replicates through the STATEMENT binlog format.
Since php-mysql-replication cannot work with this binlog format for insert / update events, I assumed that I could create an 11th replica just for this purpose (let's call it Alice), with its own binlog format set to ROW.
Problem is, when plugging php-mysql-replication on Alice:
Alice directly, insert / updates are triggered and catchable in PHP ✅Bob, nothing is triggered (tho data is properly replicated on Alice). 🚫Assuming that binlog format cannot be changed on Bob (it's prod, running thousands of queries/s, replicating to datacenters locatedx on other continents) since it involves a high regression risk, do you know if I have a mean to solve this? Maybe I just missed some MySQL configuration?
FYI I performed my tests on dev servers, where Bob is a MariaDB 10.1.
On production, Bob is a MySQL 5.5 (I know... but I doubt that changes anything to the issue).
Prod and dev run on Debian. Alice runs MySQL 5.7 on Ubuntu.
Thank you,
Ben
mysql5.7 binlog_format = mixed :
when I execute "UPDATE hj_access SET access_type = '/appver1' WHERE hj_access.id = 1"
dump_event.php output :
"UPDATE hj_access SET access_type = '/appver1' WHERE hj_access.id = 1�0�w"
I think the length pass to BinaryDataReader->read() has something wrong.
but binlog_format = row seems to be right.
Please provide the following details.
SELECT VERSION();): <10 Maria >Steps required to reproduce the problem.
Expected Result.
Actual Result.
I'm trying to figure out how to save the current binlog filename when a script exits, so it can restart where it left off using the filename and position.
I see the binlog position comes in every event, and there is a Rotate event where you get the new binlog filename when the filename changes, but what if Rotate is never called and your script exits? Is there a way to pull the current binlog filename on connect or in every event?
Thanks for your help.
Hi,
I tested the code on my ubuntu 14.04,and I sometimes got the
Event Query: COMMIT ,sometimes got Event xid;
would you mind explain the difference between the two events;
are they meaning of the end of a Transaction( for example : SQL statement: begin; insert .. update..or some other SQLs ; commit;)???
THANKS!!!
As 5.6 is deprecated new code will be only for 7.1 >= so next version will be rewritten with strict + strong hinst
Hi, again.
I think I spotted a bug when looking at the class.
Since mysql-binlog-connector-java hasn't been supported for a while I was looking for how different libraries handle empty holes made by partial updates in JSON(some complain it occurs in MySQL 8.0.16+).
Please provide the following details.
Steps required to reproduce the problem.
Expected Result.
Actual Result.
I've tested this same solution on my local system using a regular install of MySQL. When I moved to test on my production system, the events are not propagating as expected. My concern is that the version of Percona Cluster we're using is at fault, and I'm curious if there are any ideas on how to address.
We have a 3 node cluster. I'm attempting to run the php-mysql-replication application on our 3rd node.
I've added the server config (of the node that is not working) below for reference:
mysql> SHOW variables WHERE variable_name LIKE "%binlog%";
+--------------------------------------------+----------------------+
| Variable_name | Value |
+--------------------------------------------+----------------------+
| binlog_cache_size | 32768 |
| binlog_checksum | CRC32 |
| binlog_direct_non_transactional_updates | OFF |
| binlog_error_action | ABORT_SERVER |
| binlog_format | ROW |
| binlog_group_commit_sync_delay | 0 |
| binlog_group_commit_sync_no_delay_count | 0 |
| binlog_gtid_simple_recovery | ON |
| binlog_max_flush_queue_time | 0 |
| binlog_order_commits | ON |
| binlog_row_image | FULL |
| binlog_rows_query_log_events | OFF |
| binlog_skip_flush_commands | OFF |
| binlog_space_limit | 0 |
| binlog_stmt_cache_size | 32768 |
| binlog_transaction_dependency_history_size | 25000 |
| binlog_transaction_dependency_tracking | COMMIT_ORDER |
| encrypt_binlog | OFF |
| have_backup_safe_binlog_info | YES |
| innodb_api_enable_binlog | OFF |
| innodb_locks_unsafe_for_binlog | OFF |
| log_statements_unsafe_for_binlog | ON |
| max_binlog_cache_size | 18446744073709547520 |
| max_binlog_files | 0 |
| max_binlog_size | 1073741824 |
| max_binlog_stmt_cache_size | 18446744073709547520 |
| sync_binlog | 1 |
| wsrep_forced_binlog_format | NONE |
+--------------------------------------------+----------------------+
28 rows in set (0.00 sec)
mysql> SHOW variables WHERE variable_name LIKE "server_%";
+----------------+--------------------------------------+
| Variable_name | Value |
+----------------+--------------------------------------+
| server_id | 3 |
| server_id_bits | 32 |
| server_uuid | 4fe64515-be6b-11e9-8e7f-026f00293d26 |
+----------------+--------------------------------------+
3 rows in set (0.00 sec)
When the application is run, I only receive heartbeat events, even though I am 100% other activity is happening across the cluster (I do occasionally get "format description" events, but I'm not terribly sure what these are for). I have yet to see any of the important events like updates, writes, or deletes.
{
"type": "format description",
"eventInfo": {
"timestamp": 1598513185,
"type": 15,
"id": 3,
"size": 119,
"pos": 0,
"flag": 0,
"checkSum": true,
"sizeNoHeader": null,
"dateTime": null,
"binLogCurrent": {
"binLogPosition": 11890,
"binFileName": "mysql-bin.000016",
"gtid": null,
"mariaDbGtid": null
}
},
"subject": null,
"arguments": null
}{
"type": "heartbeat",
"eventInfo": {
"timestamp": 0,
"type": 27,
"id": 3,
"size": 39,
"pos": 11890,
"flag": 0,
"checkSum": true,
"sizeNoHeader": null,
"dateTime": null,
"binLogCurrent": {
"binLogPosition": 11890,
"binFileName": "mysql-bin.000016",
"gtid": null,
"mariaDbGtid": null
}
},
"subject": null,
"arguments": null
}{
"type": "heartbeat",
"eventInfo": {
"timestamp": 0,
"type": 27,
"id": 3,
"size": 39,
"pos": 11890,
"flag": 0,
"checkSum": true,
"sizeNoHeader": null,
"dateTime": null,
"binLogCurrent": {
"binLogPosition": 11890,
"binFileName": "mysql-bin.000016",
"gtid": null,
"mariaDbGtid": null
}
},
"subject": null,
"arguments": null
}EDIT: additional note; it looks like that each heartbeat event is identical while the application is running. The binlog position and file never changes. Not sure if this is helpful information or not, but thought I should include it.
in src/MySQLReplication/BinLog/BinLogConnect.php, line 162
This loop uses a function. To avoid the overhead of executing the function n times, you should precalculate it before the loop.
}
else
{
$error_code = unpack('v', $packet[1] . $packet[2])[1];
$error_msg = '';
for ($i = 9; $i < strlen($packet); $i++)
{
$error_msg .= $packet[$i];
}
throw new BinLogException($error_msg, $error_code);Posted from SensioLabsInsight
Hi
I am having this error:
Symfony\Component\Debug\Exception\ContextErrorException: Notice: Undefined offset: -1 (uncaught exception) at /var/www/html/salesforce-bridge/shared/vendor/krowinski/php-mysql-replication/src/MySQLReplication/Event/RowEvent/RowEvent.php line 372 while running console commandmy:task{"exception":"[object] (Symfony\\Component\\Debug\\Exception\\ContextErrorException(code: 0): Notice: Undefined offset: -1 at /var/www/html/salesforce-bridge/shared/vendor/krowinski/php-mysql-replication/src/MySQLReplication/Event/RowEvent/RowEvent.php:372)"}
Do you know why is this error raised? It looks like it's related with the ENUM fields, which I am using in my table, but apparently i don't get the error always but just some times.
Thanks,
Jury
Please provide the following details.
SELECT VERSION();): 5.7.26-logSteps required to reproduce the problem.
Expected Result.
Actual Result.
Steps required to reproduce the problem.
Expected Result.
Actual Result.
Cause
On restart the replication server sends an EOF packet is sent. The Event::consume() method simply skips the first byte, assuming it is an OK byte and tries to unpack a full OK header - this fails, since the EOF packet is much shorter.
First of all, thanks for this library.
The issue is not that severe, since the script exits anyhow, but since the unpack errors came first in our logs, we thought that the replication server sent an illegal packet, and spent a few hours investigating in that direction.
We have a binlog event that is larger than 1 GB and therefore receive the error message log event entry exceeded max_allowed_packet; Increase max_allowed_packet on master; the first event 'mysql-bin.021114' at 2147481564, the last event read from '/var/log/mysql/mysql-bin.021114' at 123, the last byte read from '/var/log/mysql/mysql-bin.021114' at 2147481583..
Unfortunately, the values for max_allowed_packet or slave_max_allowed_packet can not be set larger than 1 GB.
How can we solve the problem or skip the event?
Sorry, my English is not good, I hope you can understand my description
This is the description of machine translation
php-mysql-replication Version*: ^6.2.0
file: MySQLReplication\Event\dispatch(EventDTO $eventDTO = null)
6.1.0 Version
$this->eventDispatcher->dispatch($eventDTO->getType(), $eventDTO);
^6.2.0 Version
$this->eventDispatcher->dispatch($eventDTO, $eventDTO->getType());
^6.2.0 Version composer.json require "symfony/event-dispatcher": "^3.1|^4.0|^5.0"
but ^3.1|^4.0 Method parameters
dispatch($eventName, Event $event = null)
It is different from the call mode
error info :Argument 2 passed to Symfony\Component\EventDispatcher\EventDispatcher::dispatch() must be an instance of Symfony\Component\EventDispatcher\Event or null, string given, called in /WMS/vendor/krowinski/php-mysql-replication/src/MySQLReplication/Event/Event.php on line 132[/WMS/vendor/symfony/event-dispatcher/EventDispatcher.php:36]
SELECT VERSION();): 5.7.17 PerconaSteps required to reproduce the problem.
Expected Result.
RowEvent which contains the JSON-field with value ["\"test"].Actual Result.
RowEvent which contains the JSON-field with invalid value [""test"].I'll add a patch that fixes this issue.
I've also added a Question on Stackoverflow, link here:
I'm trying to use withBinLogPosition(..) when constructing the MYSQLReplicationFactory, as follows:
$dbStream = new MySQLReplicationFactory(
(new ConfigBuilder())
->withUser($userName)
->withPassword($password)
->withHost($hostName)
->withEventsOnly([
ConstEventType::UPDATE_ROWS_EVENT_V1, ConstEventType::UPDATE_ROWS_EVENT_V2,
ConstEventType::WRITE_ROWS_EVENT_V1, ConstEventType::WRITE_ROWS_EVENT_V2,
ConstEventType::DELETE_ROWS_EVENT_V1, ConstEventType::DELETE_ROWS_EVENT_V2,
])
->withSlaveId(9999)
->withBinLogPosition($binLogPosition)
->build()
);
// register benchmark subscriber handler
$dbStream->registerSubscriber(new MysqlStreamEventSubscribers($logger));
In above piece of code, I've not specified withBinLogFileName(..). I assume specifying log file name to start with is not needed (correct if wrong?). When the value of $binLogPosition is something which has already been received, the above code does not replay from that log position onwards. I don't get any old records, just get any newer DB events. Any pointer to where I'm going wrong would be helpful.
\MySQLReplication\MySQLReplicationFactory::registerSubscriber() and \MySQLReplication\MySQLReplicationFactory::unregisterSubscriber() should require the interface \Symfony\Component\EventDispatcher\EventSubscriberInterface instead of \MySQLReplication\Event\EventSubscribers as the underlying event dispatcher methods \Symfony\Component\EventDispatcher\EventDispatcher::addSubscriber / \Symfony\Component\EventDispatcher\EventDispatcher::removeSubscriber do.
Maria db have diffrent gtid marker https://mariadb.com/kb/en/mariadb/gtid/
Please provide the following details.
Steps required to reproduce the problem.
CREATE TABLE test(field DECIMAL(11,4));
INSERT INTO TEST(-57.1234);
Actual Result.
In RowEvent.php line 665:
bcmul(): Argument #1 ($num1) is not well-formed
I have taken a look at the "RowEvent" where the fault occurs. Printing out the offending argument shows that it is indeed malformed : -57.-1234. The negative is applied to both integral and fractional parts, hence the problem. Removing the XOR of the value didn't solve the issue completely, so I "abs" the values and this worked.
I don't know the intricacies of binlogs good enough to be sure that my fix is good —tbh the ABS() seems like a hack to me...— but if it is I can make a pull request for it to be integrated into the project.
do you plan to support Symfony 5?
"require": { "php": ">=7.1", "ext-sockets": "*", "ext-json": "*", "ext-bcmath": "*", "doctrine/dbal": "^2.5", "doctrine/collections": "^1.3", "symfony/event-dispatcher": "^2.8|^3.1|^4.0", "symfony/dependency-injection": "^2.8|^3.1|^4.0", "psr/simple-cache": "^1.0" },
$binLogStream = new MySQLReplicationFactory(
(new ConfigBuilder())
->withUser('remote_user')
->withHost('101.1**.2**.1**')
->withPassword('password')
->withDatabasesOnly([
'db_1'
])
->withTablesOnly([
'tb_1'
])
->build()
);
Expected Result.
Actual Result.
Please see the image below
If use the truncate, the event will not be recorded.
Hi, i am running mysql 5.7.x the call $this->binaryDataReader->readIntBeBySize(5) in getDatetime2 return a negative. (datetime column)
PHP Fatal error: Uncaught exception 'Exception' with message 'DateTime::__construct(): Failed to parse time string (10069-12-11 12:60:15) at position 12 (1): Double time specification' in C:\php-mysql-replication\src\MySQLReplication\Event\RowEvent\RowEvent.php:646
Stack trace:
#0 C:\php-mysql-replication\src\MySQLReplication\Event\RowEvent\RowEvent.php(646): DateTime->__construct('10069-12-11 12:...')
#1 C:\php-mysql-replication\src\MySQLReplication\Event\RowEvent\RowEvent.php(402): MySQLReplication\Event\RowEvent\RowEvent->getDatetime2(Array)
#2 C:\php-mysql-replication\src\MySQLReplication\Event\RowEvent\RowEvent.php(898): MySQLReplication\Event\RowEvent\RowEvent->getColumnData('\xFF')
#3 C:\php-mysql-replication\src\MySQLReplication\Event\Event.php(118): MySQLReplication\Event\RowEvent\RowEvent->makeUpdateRowsDTO()
#4 C:\php-mysql-replication\src\MySQLReplication\MySQLReplicationFactory.php(142): MySQLReplication\Event\Event->consume()
#5 C:\php-mysql-replication\example\dump_events.php(57): MySQLReplication\MySQLReplicat in C:\php-mysql-replication\src\MySQLReplication\Event\RowEvent\RowEvent.php on line 646
i was able to run https://github.com/noplay/python-mysql-replication
does it support MySQL 5.7.x ? thank you very much
I came across a fetal error:
[ErrorException]
unpack(): Type i: not enough input, need 4, have 2
only in src/MySQLReplication/BinLog/BinLogServerInfo.php there are same function call,
self::$serverInfo['connection_id'] = unpack('I', $data[$i] . $data[++$i] . $data[++$i] . $data[++$i])[1];
Run php-mysql-replication in swoole, I want consume events with concurrent, Can I do it like this?
// src/MySQLReplication/MySQLReplicationFactory.php:113
use Swoole\Coroutine;
public function run(): void
{
while (1) {
Coroutine::create(function () {
$this->consume();
});
}
}Right now if you have comment in your table, results will be incorrect.
Following code https://github.com/krowinski/php-mysql-replication/blob/master/src/MySQLReplication/BinaryDataReader/BinaryDataReader.php#L72 works by accident because s uses native architecture. And every Intel/AMD CPU is in little endian architecture which is the same as binlog bytes layout.
Architecture should be forced in unpack pattern ( s< - like in Perl, I'm not sure if PHP supports it).
Or native architecture should be checked and bytes reversed before unpacking in case of mismatch.
Please provide the following details.
SELECT VERSION();): MySQL 8.0Steps required to reproduce the problem.
Expected Result.
The replication application should work continuously, it works for sometimes and throws error.
Actual Result.
After running for few seconds, sometimes minutes it throws the following error;
PHP Fatal error: Uncaught MySQLReplication\Socket\SocketException: Success in /var/apps/nxo-fetcher-new/src/MySQLReplication/Socket/Socket.php:65
Stack trace:
#0 /var/apps/nxo-fetcher-new/src/MySQLReplication/BinLog/BinLogSocketConnect.php(63): MySQLReplication\Socket\Socket->readFromSocket(1273)
#1 /var/apps/nxo-fetcher-new/src/MySQLReplication/Event/Event.php(57): MySQLReplication\BinLog\BinLogSocketConnect->getResponse()
#2 /var/apps/nxo-fetcher-new/src/MySQLReplication/MySQLReplicationFactory.php(130): MySQLReplication\Event\Event->consume()
#3 /var/apps/nxo-fetcher-new/src/MySQLReplication/MySQLReplicationFactory.php(116): MySQLReplication\MySQLReplicationFactory->consume()
#4 /var/apps/nxo-fetcher-new/app/stream.php(78): MySQLReplication\MySQLReplicationFactory->run()
#5 {main}
thrown in /var/apps/nxo-fetcher-new/src/MySQLReplication/Socket/Socket.php on line 65
Please provide the following details.
Unknown row type: 11
Please provide the following details.
SELECT VERSION();): 10.3.21-MariaDB-1:10.3.21+maria~stretch-logSteps required to reproduce the problem.
Expected Result.
Actual Result.
Please provide the following details.
Steps required to reproduce the problem (take from the unit tests)
CREATE TABLE test (test DECIMAL(20,10));
INSERT INTO test VALUES(-42000.123456);
Expected Result.
'-42000.123456'
Actual Result.
'-42000.1234560000'
Why this is a problem
Mathematically speaking, both results are valid. However decimals are represented using strings in the lib (bcmath). Hence, when tested for equality, '-42000.123456' is not the same as '-42000.1234560000'. Greater/Lower comparison do work, but not strict equality. The unit tests enforce the presence of these trailing zeroes, which to me should not be.
I can do a pull request to fix this if you agree with me that this is an issue. Not a big one, but still :)
SELECT VERSION();): <5.7.35-38-log PERCONA>$eventInfo->getBinLogCurrent()->getBinFileName();
falls with
Exception 'TypeError' with message 'Return value of MySQLReplication\BinLog\BinLogCurrent::getBinFileName() must be of the type string, null returned'
Starting conditions:
CREATE DATABASE test;
CREATE TABLE `test`.`test_table`
(
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`is_clicked` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE = InnoDB
AUTO_INCREMENT = 1
DEFAULT CHARSET = utf8;
Insert 1 string:
insert into test.test_table (id, is_clicked) values (1,0);
And run replication with GTID = ''
Reproducing error:
show master status ;
binlog.000005,21529,,,2b172ed7-1ad8-11ec-8570-0242ac12000e:1-100
var_dump($eventInfo->getBinlogCurrent);
class MySQLReplication\BinLog\BinLogCurrent#85 (4) {
private $binLogPosition =>
int(21498)
private $binFileName =>
string(13) "binlog.000005"
private $gtid =>
string(40) "2b172ed7-1ad8-11ec-8570-0242ac12000e:100"
private $mariaDbGtid =>
NULL
}
class MySQLReplication\BinLog\BinLogCurrent#85 (4) {
private $binLogPosition =>
int(21498)
private $binFileName =>
NULL
private $gtid =>
string(40) "2b172ed7-1ad8-11ec-8570-0242ac12000e:100"
private $mariaDbGtid =>
NULL
}
Expected Result.
Actual Result.
Is it a bug?
I'm not sure if this is intended, but I noticed that the examples are passing a Config object when creating a MySQLReplicationFactory object, but it doesn't look like the constructor for MySQLReplicationFactory is using it, as the constructor doesn't have any arguments. It looks like the constructor is always referencing a static instance of the Config object, which seems to be the last Config object created.
Steps required to reproduce the problem.
Expected Result.
Actual Result.
The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID sets and the missing purged transactions are too long to print in this message. For more information, please see the master's error log or the manual for GTID_SUBTRACT.
I've checked master's binlog, it contains required gtid_set.
Please provide the following details.
SELECT VERSION();): MySQL 8Steps required to reproduce the problem.
Happening all the time
Expected Result.
Tried using row, statement, mixed based events but cannot see rows inserted, updated, deleted by triggers
Actual Result.
No Events
in src/MySQLReplication/Event/Event.php, line 29
This
mySQLRepositoryattribute is declared but never used. You should remove it.
class Event
{
/**
* @var MySQLRepository
*/
private $mySQLRepository;
/**
* @var Config
*/
private $config;
/**Posted from SensioLabsInsight
Great job, this is a very useful piece of code.
Small issue here with PHP 8.1:
Operating System: <Linux | Windows | OS X>
Linux
PHP Version: <5.6 | 7.0 | ...>
PHP 8.1
php-mysql-replication Version: <1.0.0>
1.0.0
*mysql version
Oracle-MYSQL 8.0.27
Steps required to reproduce the problem.
1. Run the dump_events.php
2. Get spammed by deprecated warnings
Expected Result.
Not get spammed by warnings
Actual Result.
Return type of MySQLReplication\Event\EventInfo::jsonSerialize() should either be compatible with JsonSerializable::jsonSerialize(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /<path>/vendor/krowinski/php-mysql-replication/src/MySQLReplication/Event/EventInfo.php on line 100
Hello there,
We have a running application which:
MySQLReplication\Event\DTO\RowsDTO events (inserts, updates, deletes)RowsDTO::getValues() to process themforeach ($rows->getValues()), stores the binlog position of the event: $event->getEventInfo()->getBinLogCurrent()->getBinFileName() / $event->getEventInfo()->getBinLogCurrent()->getBinLogPosition().When the app starts, it starts from the previously stored filename / position.
In most of the times, this works well. However, we found a typical case when it doesn't.
Consider you have an INSERT containing several thousands rows (i.e. INSERT INTO [table] SELECT * FROM [some_other_table]). As opposed to what we expected, this doesn't fire a single RowsDTO event with a huge getValues() array, but several RowsDTO events with just a few dozens items in getValues() in each.
We didn't understood why, until we discovered the binlog_row_event_max_size option on the replica server, but that's not the point.
To reproduce:
RowsDTO::getValues() eventRowsDTO events.RowsDTO events of that INSERT query are simply dismissed and apps resumes to the next query in the binlog.The only solution we have right now is to pray that the application is not shut down (or doesn't crash) while processing that kind of big statements. We could increase binlog_row_event_max_size on the replica server but this won't guarantee RowsDTO won't be split and OTOH having a huge array in RowsDTO::getValues() might lead to memory issues.
As a quick fix, we now temporarily store filename / position on QueryDTO that are fired before and after the RowsDTO events, so that if not all RowsDTO can be processed, they can be replayed with the former position (00001 in our example). This leads to duplicate processing, but we prefer this instead of missing events.
Thank you,
Ben
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.