锁定表会导致django出现保存点问题

chy5wohz  于 2021-06-17  发布在  Mysql
关注(0)|答案(1)|浏览(386)

我试图在django应用程序的mariadb数据库中构建一个有向无环图(dag)。因为这是非循环的,所以我需要验证任何添加的元素(顶点/边)都不会在图中创建循环。
许多客户机将尝试在一天中同时添加元素,但是这些循环检查需要是原子的,因此我推断在添加/更新元素时需要使用一些锁。django似乎没有提供这样的东西,所以我尝试使用一个原始的 LOCK TABLES / UNLOCK TABLES 查询。这是我用来做的代码。。。

  1. def lock_tables():
  2. cursor = get_connection(DEFAULT_DB_ALIAS).cursor()
  3. tables = [
  4. 'vertex',
  5. 'edge'
  6. ]
  7. lock_query = ', '.join(
  8. "{} {}".format(table, 'WRITE') for table in tables
  9. )
  10. query = 'LOCK TABLES {}'.format(lock_query)
  11. cursor.execute(query)
  12. def unlock_tables():
  13. cursor = get_connection(DEFAULT_DB_ALIAS).cursor()
  14. cursor.execute('UNLOCK TABLES')

然后在我的模式下 save 方法。。。

  1. @transaction.atomic()
  2. def save(self, *args,**kwargs):
  3. print("---INSIDE MODEL SAVE")
  4. try:
  5. print("---LOCKING TABLES")
  6. lock_tables()
  7. print("---LOCKED TABLES")
  8. super().save(*args,**kwargs)
  9. # TODO: Add Cycle check here
  10. except Exception as ex:
  11. print("---EXCEPTION THROWN INSIDE SAVE: {}".format(ex))
  12. raise
  13. finally:
  14. print("---UNLOCKING TABLES")
  15. unlock_tables()
  16. print("---UNLOCKED TABLES")

但是,有关锁定和解锁这些表的某些内容会影响使用创建的保存点 django.db.transaction.atomic ... 在django试图退出 atomic 上下文中,它尝试回滚到已发布的保存点。
下面是一些日志,我试图捕捉问题, Executing Query 线路来自 django.db.backends.mysql.base , STARTING/EXITING ATOMIC 线路来自
django.db.transactions.atomic __enter__ / __exit__ 方法及注意事项 #### 是我在事后加上的评论,试图解释我的想法。

  1. ---STARTING ATOMIC #### Atomic context wrapping my serializer's create method
  2. Executing query: 'SAVEPOINT `s139667621889792_x1`' - args: None
  3. ---STARTING ATOMIC #### Atomic context wrapping my model's save method
  4. Executing query: 'SAVEPOINT `s139667621889792_x2`' - args: None
  5. ---INSIDE MODEL SAVE
  6. ---LOCKING TABLES
  7. Executing query: 'LOCK TABLES vertex WRITE, edge WRITE
  8. ---LOCKED TABLES
  9. ---STARTING ATOMIC #### I think Django must wrap some queries in an atomic block, but this doesnt even create a savepoint
  10. Executing query: 'INSERT INTO `edge`...
  11. ---EXITING ATOMIC
  12. #### WHERE MY CYCLE CHECK CODE WOULD RUN - not implemented yet
  13. ---UNLOCKING TABLES
  14. Executing query: 'UNLOCK TABLES' - args: None
  15. ---UNLOCKED TABLES
  16. ---EXITING ATOMIC
  17. Executing query: 'RELEASE SAVEPOINT `s139667621889792_x2`' - args: None
  18. Executing query: 'ROLLBACK TO SAVEPOINT `s139667621889792_x2`' - args: None ### WHAT I BELIEVE TO BE THE OFFENDING QUERY
  19. ---EXITING ATOMIC
  20. Executing query: 'ROLLBACK TO SAVEPOINT `s139667621889792_x1`' - args: None
  21. Traceback (most recent call last):
  22. File ".../site-packages/django/db/backends/utils.py", line 83, in _execute
  23. return self.cursor.execute(sql)
  24. File ".../site-packages/django/db/backends/mysql/base.py", line 72, in execute
  25. return self.cursor.execute(query, args)
  26. File ".../site-packages/pymysql/cursors.py", line 170, in execute
  27. result = self._query(query)
  28. File ".../site-packages/pymysql/cursors.py", line 328, in _query
  29. conn.query(q)
  30. File ".../site-packages/pymysql/connections.py", line 516, in query
  31. self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  32. File ".../site-packages/pymysql/connections.py", line 727, in _read_query_result
  33. result.read()
  34. File ".../site-packages/pymysql/connections.py", line 1066, in read
  35. first_packet = self.connection._read_packet()
  36. File ".../site-packages/pymysql/connections.py", line 683, in _read_packet
  37. packet.check_error()
  38. File ".../site-packages/pymysql/protocol.py", line 220, in check_error
  39. err.raise_mysql_exception(self._data)
  40. File ".../site-packages/pymysql/err.py", line 109, in raise_mysql_exception
  41. raise errorclass(errno, errval)
  42. pymysql.err.InternalError: (1305, 'SAVEPOINT s139667621889792_x2 does not exist')

如上所示,django试图回滚到它已经发布的保存点。如果我删除了对lock/unlock表的调用,那么这段代码可以完美地工作,但是我不能再保证我的周期检查是原子的。
以前有没有人遇到过这个问题,或者有没有什么技巧可以让你更深入地了解原因?
编辑:我读得越多,我越觉得我想要的行为是不可能的。根据mysql关于锁的文档,当您在表上获得锁时,事务似乎被提交。这打破了我的用例,因为我希望在循环检查失败时回滚事务。

rqmkfv5c

rqmkfv5c1#

任何反循环算法都取决于在执行检查时表没有改变。对的?执行周期检查需要多长时间?你每天需要多少支票?
假设你有足够的时间做所有这些工作,那么考虑一下:

  1. SELECT GET_LOCK('cycle_check'); -- (you may want timeout)
  2. BEGIN;
  3. INSERT new item in graph
  4. perform cycle check
  5. if ... COMMIT else ROLLBACK
  6. SELECT RELEASE_LOCK('cycle_check');

请注意,此锁定机制与 LOCK TABLES 毫无用处。
要防止在循环检查期间读取,还需要:

  1. SELECT GET_LOCK('cycle_check');
  2. SELECT ...;
  3. SELECT RELEASE_LOCK('cycle_check');

(旁注:这是极为罕见的 GET_LOCK 以“正确”的方式进行锁定。请不要将此扩展到任意其他情况。)

展开查看全部

相关问题