我有一个很大的表,其中我必须更新几行。我尝试使用多个线程同时更新不同的行,但是mysql似乎锁定了表中的所有行,而不是只锁定那些与where子句匹配的行。因此,更新不是并发的,有时会导致锁超时错误。
有没有办法同时更新不同的行?
我在后台使用django来执行更新操作。我可以换成postgres,但它会有帮助还是仍然有锁的问题?
更新:添加了代码段:
def process_calculate_training(base_dept, exch_dept):
# First initialize training to None, this is not happening concurrently.
Emp.objects.filter(exch_dept=exch_dept, base_dept=base_dept).update(training=None)
# Compute training and store result
pass
def start_calculating_training():
thread_func_args = [['MECH', 'COE'], ['MECH', 'ECE'], ['MECH', 'ICE'], ['MECH', 'IT']]
with ThreadPool(4) as p:
p.starmap(process_calculate_training, thread_func_args)
更新:执行'show engine innodb status;'
发现django自动将隔离级别设置为“read committed”。因此mysql只锁定那些需要更新的行。但是更新并不是同时进行的,因为两个线程似乎在等待锁,即使它们必须更新不同的行。
是的 show engine innodb status;
在并发更新操作期间。结果如下:
TRANSACTIONS
------------
Trx id counter 2446659
Purge done for trx's n:o < 2446655 undo n:o < 0 state: running but idle
History list length 32
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421137001004688, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421137001002848, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421137001003768, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 2446658, ACTIVE 11 sec fetching rows
mysql tables in use 1, locked 1
LOCK WAIT 4601 lock struct(s), heap size 1024208, 7281 row lock(s), undo log entries 2165
MySQL thread id 427, OS thread handle 139661736408832, query id 651475 localhost root updating
UPDATE `sdk_emp` SET `training` = NULL WHERE (`sdk_emp`.`base_dept` = 'MECH' AND `sdk_emp`.`exch_dept` = 'COE')
------- TRX HAS BEEN WAITING 9 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 345 page no 5238 n bits 240 index PRIMARY of table `empdb`.`sdk_emp` trx id 2446658 lock_mode X locks rec but not gap waiting
Record lock, heap no 135
------------------
---TRANSACTION 2446657, ACTIVE 11 sec fetching rows
mysql tables in use 1, locked 1
13971 lock struct(s), heap size 2351312, 15525 row lock(s), undo log entries 6307
MySQL thread id 424, OS thread handle 139661737813760, query id 651473 localhost root updating
UPDATE `sdk_emp` SET `training` = NULL WHERE (`sdk_emp`.`base_dept` = 'MECH' AND `sdk_emp`.`exch_dept` = 'ECE')
---TRANSACTION 2446656, ACTIVE 11 sec fetching rows
mysql tables in use 1, locked 1
LOCK WAIT 879 lock struct(s), heap size 270544, 1985 row lock(s), undo log entries 411
MySQL thread id 426, OS thread handle 139661736810240, query id 651471 localhost root updating
UPDATE `sdk_emp` SET `training` = NULL WHERE (`sdk_emp`.`base_dept` = 'MECH' AND `sdk_emp`.`exch_dept` = 'ICE')
------- TRX HAS BEEN WAITING 10 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 345 page no 1078 n bits 240 index PRIMARY of table `empdb`.`sdk_emp` trx id 2446656 lock_mode X locks rec but not gap waiting
Record lock, heap no 125
------------------
---TRANSACTION 2446655, ACTIVE 11 sec fetching rows
mysql tables in use 1, locked 1
13984 lock struct(s), heap size 2318544, 15263 row lock(s), undo log entries 6308
MySQL thread id 425, OS thread handle 139661737010944, query id 651469 localhost root updating
UPDATE `sdk_emp` SET `training` = NULL WHERE (`sdk_emp`.`base_dept` = 'MECH' AND `sdk_emp`.`exch_dept` = 'IT')
一段时间后再次运行命令:
TRANSACTIONS
------------
Trx id counter 2446659
Purge done for trx's n:o < 2446655 undo n:o < 0 state: running but idle
History list length 32
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421137001004688, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421137001002848, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421137001003768, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 2446658, ACTIVE 31 sec fetching rows
mysql tables in use 1, locked 1
LOCK WAIT 4601 lock struct(s), heap size 1024208, 7281 row lock(s), undo log entries 2165
MySQL thread id 427, OS thread handle 139661736408832, query id 651475 localhost root updating
UPDATE `sdk_emp` SET `training` = NULL WHERE (`sdk_emp`.`base_dept` = 'MECH' AND `sdk_emp`.`exch_dept` = 'COE')
------- TRX HAS BEEN WAITING 29 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 345 page no 5238 n bits 240 index PRIMARY of table `empdb`.`sdk_emp` trx id 2446658 lock_mode X locks rec but not gap waiting
Record lock, heap no 135
------------------
---TRANSACTION 2446657, ACTIVE 31 sec updating or deleting
mysql tables in use 1, locked 1
27466 lock struct(s), heap size 4120784, 25388 row lock(s), undo log entries 12691
MySQL thread id 424, OS thread handle 139661737813760, query id 651473 localhost root updating
UPDATE `sdk_emp` SET `training` = NULL WHERE (`sdk_emp`.`base_dept` = 'MECH' AND `sdk_emp`.`exch_dept` = 'ECE')
---TRANSACTION 2446656, ACTIVE 31 sec fetching rows
mysql tables in use 1, locked 1
LOCK WAIT 879 lock struct(s), heap size 270544, 1985 row lock(s), undo log entries 411
MySQL thread id 426, OS thread handle 139661736810240, query id 651471 localhost root updating
UPDATE `sdk_emp` SET `training` = NULL WHERE (`sdk_emp`.`base_dept` = 'MECH' AND `sdk_emp`.`exch_dept` = 'ICE')
------- TRX HAS BEEN WAITING 30 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 345 page no 1078 n bits 240 index PRIMARY of table `empdb`.`sdk_emp` trx id 2446656 lock_mode X locks rec but not gap waiting
Record lock, heap no 125
------------------
---TRANSACTION 2446655, ACTIVE 31 sec fetching rows
mysql tables in use 1, locked 1
27489 lock struct(s), heap size 4169936, 25817 row lock(s), undo log entries 12692
MySQL thread id 425, OS thread handle 139661737010944, query id 651469 localhost root updating
UPDATE `sdk_emp` SET `training` = NULL WHERE (`sdk_emp`.`base_dept` = 'MECH' AND `sdk_emp`.`exch_dept` = 'IT')
我不明白为什么两个线程不能得到锁,因为他们将更新不同的行?解决办法是什么?
1条答案
按热度按时间p4rjhz4m1#
确保表索引良好,并且更新查询没有执行完全表扫描或完全索引扫描。这样,您的语句可能需要更少的锁。如果可能,按主键或唯一键更新。
对语句进行解释将帮助您了解查询的效率。
mysql的默认隔离级别是repeatable read,如果您使用的是transaction,它将为它所接触的行保留行锁,直到事务结束,即使行不匹配。
将隔离级别更改为readcommitted可以解决这个问题,即如果不匹配,行锁将立即释放。