Procedure.unit, Customer.unit are null in DB sometimes about smevente

[veny]

it shouldn't be possible, because both of the references are mandatory and notNull

found such entries:

• select from procedure where unit is null
• select from customer where unit is null

[veny]

DB fix:

• update customer set unit=#13:0 where unit is null
• update procedure set unit=#13:0 where unit is null

[veny]

probably solved by corresponding commit

[veny]

no deterministic scenario how to reproduce this, but sometimes:

• create event
• change event, primarily procedure
• go to next week
• go back to this week
• delete the event
  ->> bingo

[veny]

in logs found the problem even by simple creation of event

[veny]

suspicion: the loaded links are detached with first level of associations, so by storage the second level (event.procedur.unit, event.customer.unit) is set to null

[veny]

solution has to be newly implemented -> reopen

[veny]

the cause found:
the was detaching until first level of associations in AbstractDao.
by obtaining list of events in a time frame, following code is invoked: event.getCustomer().setUnit(null);
=> this sets 'unit' to null on object which is not detached -> it is mirrored into DB

[veny]

• problem occured again

orientdb {smevente}> select from procedure where unit is null

----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
#   |@RID  |unit|time|color |revision|name    |type     |messageText                                                                     
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
0   |#16:11|null|30  |FC000D|0       |Extrakce|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
1   |#16:16|null|30  |c1f989|0       |Prevence|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------


orientdb {smevente}> select from customer where unit is null;

----+--------+----+------+-------+-------+-------+--------+--------+--------+---------+---------+---------+------------+------------+--------------
#   |@RID    |unit|degree|surname|zipCode|careers|revision|city    |employer|firstname|phoneNumb|birthNumb|asciiFullnam|sendingChann|street        
----+--------+----+------+-------+-------+-------+--------+--------+--------+---------+---------+---------+------------+------------+--------------
0   |#15:912 |null|ing.  |Špůr   |33007  |null   |0       |Druztová|null    |Josef    |604936168|371024004|JOSEF SPŮR  |1           |Druztová 253  
1   |#15:1235|null|null  |Marcel |30100  |null   |0       |Plzeň   |null    |Falcman  |722273151|800829...|FALCMAN M...|1           |Pod záhorsk...
----+--------+----+------+-------+-------+-------+--------+--------+--------+---------+---------+---------+------------+------------+--------------

[veny]

analysis:
both customers had one deleted event with corresponding procedure (16:11, 16:16) -> problem caused probably by deleting an event -> changed AbstractDaoOrientdb#remove: db.load(rid, "*:0") -> db.load(rid)

[veny]

again:

orientdb {smevente}> select from procedure where unit is null

----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
#   |@RID  |unit|time|color |revision|name    |type     |messageText                                                                     
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
0   |#16:16|null|30  |c1f989|0       |Prevence|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------

1 item(s) found. Query executed in 0.004 sec(s).
orientdb {smevente}> select from customer where unit is null 

----+--------+----+------+-------+-------+--------+--------+---------+---------+---------+---------+---------+----------+----------+---------------
#   |@RID    |unit|degree|zipCode|careers|revision|employer|firstname|surname  |phoneNumb|birthNumb|city     |street    |sendingCha|asciiFullname  
----+--------+----+------+-------+-------+--------+--------+---------+---------+---------+---------+---------+----------+----------+---------------
0   |#15:1058|null|null  |33012  |null   |0       |null    |Miroslava|Kožíšková|721902259|685905...|Horní ...|Družste...|1         |MIROSLAVA KO...
----+--------+----+------+-------+-------+--------+--------+---------+---------+---------+---------+---------+----------+----------+---------------

[veny]

again (only procedure):

orientdb {smevente}> select from procedure where unit is null

----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
#   |@RID  |unit|time|color |revision|name    |type     |messageText                                                                     
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
0   |#16:16|null|30  |c1f989|0       |Prevence|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
1   |#16:17|null|10  |6763C7|0       |Kontrola|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------

[veny]

and again:

orientdb {smevente}> select from customer where unit is null 

----+--------+----+------+-------+-------+-------+--------+--------+---------+----------+----------+----------+-----------+-----------+------------
#   |@RID    |unit|degree|surname|zipCode|careers|revision|employer|firstname|phoneNumbe|birthNumbe|street    |city       |asciiFullna|sendingChann
----+--------+----+------+-------+-------+-------+--------+--------+---------+----------+----------+----------+-----------+-----------+------------
0   |#15:1477|null|null  |Vrba   |34004  |null   |0       |null    |Jiří     |732884899 |5809060631|U Řezné...|Železná ...|JIRI VRBA  |1           
----+--------+----+------+-------+-------+-------+--------+--------+---------+----------+----------+----------+-----------+-----------+------------

1 item(s) found. Query executed in 0.057 sec(s).
orientdb {smevente}> select from procedure where unit is null

----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
#   |@RID  |unit|time|color |revision|name    |type     |messageText                                                                     
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
0   |#16:16|null|30  |c1f989|0       |Prevence|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------

1 item(s) found. Query executed in 0.004 sec(s).

[veny]

and again:

orientdb {smevente}> select from procedure where unit is null          

----+-----+----+----+------+--------+---------+---------+--------------------------------------------------------------------------------
#   |@RID |unit|time|color |revision|name     |type     |messageText                                                                     
----+-----+----+----+------+--------+---------+---------+--------------------------------------------------------------------------------
0   |#16:5|null|30  |DB0CF2|0       |Konzer...|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+-----+----+----+------+--------+---------+---------+--------------------------------------------------------------------------------

1 item(s) found. Query executed in 0.005 sec(s).
orientdb {smevente}> select from customer where unit is null 

----+------+----+-----------+--------+---------+-------+-----------+------+----------+-----+-------+--------+-------+-------------+--------------
#   |@RID  |unit|phoneNumber|revision|firstname|surname|birthNumber|degree|street    |city |zipCode|employer|careers|asciiFullname|sendingChannel
----+------+----+-----------+--------+---------+-------+-----------+------+----------+-----+-------+--------+-------+-------------+--------------
0   |#15:83|null|731721945  |0       |Tomáš    |Gabriel|9104272111 |null  |Malická 17|Plzeň|30164  |null    |null   |TOMAS GABRIEL|1             
----+------+----+-----------+--------+---------+-------+-----------+------+----------+-----+-------+--------+-------+-------------+--------------

1 item(s) found. Query executed in 0.089 sec(s).

[veny]

added cron monitor (every minute):

* * * * * JAVA_HOME=/opt/java; export PATH=$PATH:$JAVA_HOME/bin; cnt=$(/opt/orientdb/bin/console.sh "connect remote:/smevente admin admin; select from procedure where unit is null;" | grep 'item(s) found' | awk '{ print $1 }'); if [ ! "$cnt" -eq 0 ] ; then echo -e 'found bad procedure' | mail -s "Smevente: BAD PROCEDURE $(date)" -r [email protected] [email protected]; fi

[veny]

switch to OrientDB 1.7.9

[veny]

again:

orientdb {db=smevente}> select from customer where unit is null 

----+--------+----+------+-------+-------+--------+--------+--------+---------+---------+---------+---------+-----------+------------+-------------
#   |@RID    |unit|degree|zipCode|careers|revision|city    |employer|firstname|surname  |phoneNumb|birthNumb|sendingChan|street      |asciiFullname
----+--------+----+------+-------+-------+--------+--------+--------+---------+---------+---------+---------+-----------+------------+-------------
0   |#15:1721|null|null  |33011  |null   |0       |Třemošná|null    |Markéta  |Štruncová|739859303|995402...|1          |Budovatel...|MARKETA ST...
----+--------+----+------+-------+-------+--------+--------+--------+---------+---------+---------+---------+-----------+------------+-------------

1 item(s) found. Query executed in 0.097 sec(s).
orientdb {db=smevente}> select from procedure where unit is null

----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
#   |@RID  |unit|time|color |revision|name    |type     |messageText                                                                     
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
0   |#16:16|null|30  |c1f989|0       |Prevence|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------

1 item(s) found. Query executed in 0.003 sec(s).

---

seems to be a caching problem by delete of an event:

• firts email: 9:44:03
• no special entries in https access log
• log snapshot:

2014-10-14 09:43:26,310  INFO EventServiceImpl [recepce] - found events, authorId=#12:3, from=Sun Oct 19 22:00:00 UTC 2014, to=Sun Oct 26 22:59:59 UTC 2014, size=60
2014-10-14 09:43:31,163  INFO EventServiceImpl [recepce] - deleted event, id=#17:24483
2014-10-14 09:43:45,500  INFO EventServiceImpl [recepce] - found events, authorId=#12:3, from=Sun Oct 12 22:00:00 UTC 2014, to=Sun Oct 19 21:59:59 UTC 2014, size=53
2014-10-14 09:44:09,177  INFO PingController [] - monitor, ip=127.0.0.1

• the deleted event has damaged customer and procedure

orientdb {db=smevente}> select from event where @rid = #17:24483

----+---------+----+----+------+------+------+--------+--------+---------+---------+---------+---------+---------+---------+---------+--------------
#   |@RID     |sent|type|author|length|notice|customer|revision|procedure|updatedAt|updatedBy|deletedBy|sendAttem|startTime|deletedAt|text          
----+---------+----+----+------+------+------+--------+--------+---------+---------+---------+---------+---------+---------+---------+--------------
0   |#17:24483|null|null|#12:3 |30    |      |#15:1721|null    |#16:16   |null     |null     |#12:2    |0        |2014-1...|2014-1...|Dobry den,p...
----+---------+----+----+------+------+------+--------+--------+---------+---------+---------+---------+---------+---------+---------+--------------

• added logging of cache into AbstractDaoOrientdb#remove

[veny]

again:

orientdb {db=smevente}> select from procedure where unit is null          

----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
#   |@RID  |unit|time|color |revision|name    |type     |messageText                                                                     
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
0   |#16:16|null|30  |c1f989|0       |Prevence|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------

1 item(s) found. Query executed in 0.006 sec(s).

again:

orientdb {db=smevente}> select from procedure where unit is null

----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
#   |@RID  |unit|time|color |revision|name    |type     |messageText                                                                     
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------
0   |#16:17|null|10  |6763C7|0       |Kontrola|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+------+----+----+------+--------+--------+---------+--------------------------------------------------------------------------------

1 item(s) found. Query executed in 0.005 sec(s).

[veny]

----+------+----+----+------+--------+---------+---------+--------------------------------------------------------------------------------
#   |@RID  |unit|time|color |revision|name     |type     |messageText                                                                     
----+------+----+----+------+--------+---------+---------+--------------------------------------------------------------------------------
0   |#16:2 |null|50  |FCAB42|0       |Estetika |IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
1   |#16:3 |null|60  |C2C1B8|0       |Poznámky |IN_CAL...|                                                                                
2   |#16:5 |null|30  |DB0CF2|0       |Konzer...|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
3   |#16:6 |null|60  |F7FAF8|0       |Implan...|IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
4   |#16:7 |null|120 |B89BF2|0       |Cerek    |IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
5   |#16:16|null|30  |c1f989|0       |Prevence |IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
6   |#16:17|null|10  |6763C7|0       |Kontrola |IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
7   |#16:19|null|40  |f9e427|null    |Endo     |IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
8   |#16:52|null|40  |00f0f4|0       |Kyretáž  |IN_CAL...|Dobry den,pripominame Vam termin osetreni dne #{date} v #{time}.V pripade zme...
----+------+----+----+------+--------+---------+---------+--------------------------------------------------------------------------------

[veny]

seems to be solved by no detasching in bulkSend