python 使用spellfix1表反映sqlite数据库

pkln4tw6  于 2024-01-05  发布在  Python
关注(0)|答案(2)|浏览(182)

我试图反映一个数据库(更具体地说是plex media server database)是基于sqlite的。
我可以连接到DB:

  1. import sqlalchemy as sa
  2. from sqlalchemy.orm import create_session
  3. from sqlalchemy.ext.automap import automap_base
  4. from sqlalchemy.engine.reflection import Inspector
  5. conn = 'sqlite:////..../com.plexapp.plugins.library.db-2016-07-17'
  6. eng = sa.create_engine(conn)
  7. plex = eng.connect()

字符串
我可以查询DB:

  1. s = create_session(plex)
  2. r = s.execute('SELECT COUNT(*) FROM METADATA_ITEMS')
  3. r.fetchall()
  4. >>> [(55368,)]


我甚至可以用SQLAlchemy的Inspector检查所有的表

  1. i = Inspector.from_engine(plex)
  2. i.get_sorted_table_and_fkc_names()
  3. >>>> [('metadata_item_views', set()),
  4. ('cloudsync_files', set()),
  5. ('library_timeline_entries', set()),
  6. ('synced_ancestor_items', set()),
  7. ('schema_migrations', set()),
  8. ('fts4_tag_titles_stat', set()),
  9. ('spellfix_tag_titles', set()),
  10. ('metadata_items', set()),
  11. ('media_streams', set()),
  12. ('metadata_relations', set()),
  13. ('play_queue_generators', set()),
  14. ...


然而,我似乎不能反映整个DB。我得到了这个丑陋的异常:

  1. m = sa.MetaData(bind=plex)
  2. m.reflect()
  3. >>>>
  4. ---------------------------------------------------------------------------
  5. OperationalError Traceback (most recent call last)
  6. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, *args)
  7. 1138 parameters,
  8. -> 1139 context)
  9. 1140 except Exception as e:
  10. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/default.py in do_execute(self, cursor, statement, parameters, context)
  11. 449 def do_execute(self, cursor, statement, parameters, context=None):
  12. --> 450 cursor.execute(statement, parameters)
  13. 451
  14. OperationalError: no such module: spellfix1
  15. The above exception was the direct cause of the following exception:
  16. OperationalError Traceback (most recent call last)
  17. <ipython-input-58-8cb90bbdb0ad> in <module>()
  18. 1 m = sa.MetaData(bind=plex)
  19. 2
  20. ----> 3 m.reflect()
  21. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in reflect(self, bind, schema, views, only, extend_existing, autoload_replace, **dialect_kwargs)
  22. 3653
  23. 3654 for name in load:
  24. -> 3655 Table(name, self, **reflect_opts)
  25. 3656
  26. 3657 def append_ddl_listener(self, event_name, listener):
  27. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in __new__(cls, *args, **kw)
  28. 414 except:
  29. 415 with util.safe_reraise():
  30. --> 416 metadata._remove_table(name, schema)
  31. 417
  32. 418 @property
  33. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py in __exit__(self, type_, value, traceback)
  34. 58 exc_type, exc_value, exc_tb = self._exc_info
  35. 59 self._exc_info = None # remove potential circular references
  36. ---> 60 compat.reraise(exc_type, exc_value, exc_tb)
  37. 61 else:
  38. 62 if not compat.py3k and self._exc_info and self._exc_info[1]:
  39. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/util/compat.py in reraise(tp, value, tb, cause)
  40. 184 if value.__traceback__ is not tb:
  41. 185 raise value.with_traceback(tb)
  42. --> 186 raise value
  43. 187
  44. 188 else:
  45. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in __new__(cls, *args, **kw)
  46. 409 metadata._add_table(name, schema, table)
  47. 410 try:
  48. --> 411 table._init(name, metadata, *args, **kw)
  49. 412 table.dispatch.after_parent_attach(table, metadata)
  50. 413 return table
  51. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in _init(self, name, metadata, *args, **kwargs)
  52. 482 # circular foreign keys
  53. 483 if autoload:
  54. --> 484 self._autoload(metadata, autoload_with, include_columns)
  55. 485
  56. 486 # initialize all the column, etc. objects. done after reflection to
  57. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in _autoload(self, metadata, autoload_with, include_columns, exclude_columns)
  58. 494 autoload_with.run_callable(
  59. 495 autoload_with.dialect.reflecttable,
  60. --> 496 self, include_columns, exclude_columns
  61. 497 )
  62. 498 else:
  63. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in run_callable(self, callable_, *args, **kwargs)
  64. 1475
  65. 1476 """
  66. -> 1477 return callable_(self, *args, **kwargs)
  67. 1478
  68. 1479 def _run_visitor(self, visitorcallable, element, **kwargs):
  69. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/default.py in reflecttable(self, connection, table, include_columns, exclude_columns)
  70. 362 self, connection, table, include_columns, exclude_columns):
  71. 363 insp = reflection.Inspector.from_engine(connection)
  72. --> 364 return insp.reflecttable(table, include_columns, exclude_columns)
  73. 365
  74. 366 def get_pk_constraint(self, conn, table_name, schema=None, **kw):
  75. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/reflection.py in reflecttable(self, table, include_columns, exclude_columns)
  76. 561
  77. 562 for col_d in self.get_columns(
  78. --> 563 table_name, schema, **table.dialect_kwargs):
  79. 564 found_table = True
  80. 565
  81. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/reflection.py in get_columns(self, table_name, schema, **kw)
  82. 367 col_defs = self.dialect.get_columns(self.bind, table_name, schema,
  83. 368 info_cache=self.info_cache,
  84. --> 369 **kw)
  85. 370 for col_def in col_defs:
  86. 371 # make this easy and only return instances for coltype
  87. <string> in get_columns(self, connection, table_name, schema, **kw)
  88. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/reflection.py in cache(fn, self, con, *args, **kw)
  89. 52 ret = info_cache.get(key)
  90. 53 if ret is None:
  91. ---> 54 ret = fn(self, con, *args, **kw)
  92. 55 info_cache[key] = ret
  93. 56 return ret
  94. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/dialects/sqlite/base.py in get_columns(self, connection, table_name, schema, **kw)
  95. 1181 def get_columns(self, connection, table_name, schema=None, **kw):
  96. 1182 info = self._get_table_pragma(
  97. -> 1183 connection, "table_info", table_name, schema=schema)
  98. 1184
  99. 1185 columns = []
  100. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/dialects/sqlite/base.py in _get_table_pragma(self, connection, pragma, table_name, schema)
  101. 1478 qtable = quote(table_name)
  102. 1479 statement = "%s%s(%s)" % (statement, pragma, qtable)
  103. -> 1480 cursor = connection.execute(statement)
  104. 1481 if not cursor._soft_closed:
  105. 1482 # work around SQLite issue whereby cursor.description
  106. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in execute(self, object, *multiparams, **params)
  107. 904 """
  108. 905 if isinstance(object, util.string_types[0]):
  109. --> 906 return self._execute_text(object, multiparams, params)
  110. 907 try:
  111. 908 meth = object._execute_on_connection
  112. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _execute_text(self, statement, multiparams, params)
  113. 1052 statement,
  114. 1053 parameters,
  115. -> 1054 statement, parameters
  116. 1055 )
  117. 1056 if self._has_events or self.engine._has_events:
  118. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, *args)
  119. 1144 parameters,
  120. 1145 cursor,
  121. -> 1146 context)
  122. 1147
  123. 1148 if self._has_events or self.engine._has_events:
  124. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _handle_dbapi_exception(self, e, statement, parameters, cursor, context)
  125. 1339 util.raise_from_cause(
  126. 1340 sqlalchemy_exception,
  127. -> 1341 exc_info
  128. 1342 )
  129. 1343 else:
  130. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/util/compat.py in raise_from_cause(exception, exc_info)
  131. 200 exc_type, exc_value, exc_tb = exc_info
  132. 201 cause = exc_value if exc_value is not exception else None
  133. --> 202 reraise(type(exception), exception, tb=exc_tb, cause=cause)
  134. 203
  135. 204 if py3k:
  136. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/util/compat.py in reraise(tp, value, tb, cause)
  137. 183 value.__cause__ = cause
  138. 184 if value.__traceback__ is not tb:
  139. --> 185 raise value.with_traceback(tb)
  140. 186 raise value
  141. 187
  142. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, *args)
  143. 1137 statement,
  144. 1138 parameters,
  145. -> 1139 context)
  146. 1140 except Exception as e:
  147. 1141 self._handle_dbapi_exception(
  148. /opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/default.py in do_execute(self, cursor, statement, parameters, context)
  149. 448
  150. 449 def do_execute(self, cursor, statement, parameters, context=None):
  151. --> 450 cursor.execute(statement, parameters)
  152. 451
  153. 452 def do_execute_no_params(self, cursor, statement, context=None):
  154. OperationalError: (sqlite3.OperationalError) no such module: spellfix1 [SQL: 'PRAGMA table_info("spellfix_metadata_titles")']


我知道spellfix1是sqlite的一个特性,但我找不到任何python支持/插件来正确地反映它,或者我不完全理解所引发的异常。
1.我想知道为什么会发生这种情况(为什么这些表很特殊,不能正常反映?)
1.有没有任何库/扩展可以让sqlalchemy支持这个?
谢谢你,谢谢

gtlvzcf8

gtlvzcf81#

SQLite是一个嵌入式数据库,并且可以自定义。XNUMX Media Server添加了spellfix 1扩展; documentation说:
spellfix 1虚拟表不包含在SQLite合并中,也不是任何标准SQLite构建的一部分。它是一个loadable extension
为了能够访问这个表,您必须自己编译这个扩展,并将其加载到Python数据库连接中。
如果你实际上不需要做拼写检查,你可以创建一个没有虚拟表的数据库副本:
1.使用sqlite3命令行shell将数据库转换为文本:

  1. sqlite3 some/where/...library.db ".dump" > library.sql

字符串
(this没有安装模块的情况下工作);
1.从SQL脚本中删除虚拟表的创建;这是一行代码,如下所示:

  1. INSERT INTO sqlite_master(type,name,tbl_name,rootpage,sql)VALUES('table','xxx','xxx',0,'CREATE VIRTUAL TABLE xxx using spellfix1(...)');


1.从该脚本创建一个新数据库:

  1. sqlite3 new.db < library.sql

展开查看全部
qhhrdooz

qhhrdooz2#

我发现了一个不错的变通办法。
SQLAlchemy允许元数据反射为“only”标志提供可调用。这将忽略需要插件的不受支持的表。(感谢@CL.提供忽略这些表的提示!)

  1. m = sa.MetaData(plex)
  2. m.reflect(only=(lambda x, y: 'spellfix' not in x))
  3. Base = automap_base(metadata=m)
  4. Base.prepare(plex)

字符串
工作起来很有魅力!

相关问题