apache Django -大文件上传-将文件从临时文件夹移动到最终目标失败

vhipe2zx  于 2023-10-23  发布在  Apache
关注(0)|答案(1)|浏览(148)

在升级Python、Django和Alpine到最新版本时遇到了这个错误。而且很奇怪当我上传的文件大于FILE_UPLOAD_MAX_MEMORY_SIZE（设置为5 MB）时，会出现错误。
我有两个开发环境。

  • 一个是本地开发服务器（只是普通的py manage.py runserver），
  • 另一个是运行在Docker（Windows）+ Alpine Linux + Apache 2 + mod_wsgi下的，这也是一个“开发”环境。

此错误仅在Docker环境下发生，并且仅在文件超过最大内存大小时发生。在所有其他情况下，上传工作-即使文件超出本地开发服务器的内存限制。
错误：

  1. Saving new order file to: customer_name/_web_portal/PendingOrders/7935/zip_03.zip 
  3. Internal Server Error: /b2b_api/orders/
  4. Traceback (most recent call last):
  5.   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
  6.     response = get_response(request)
  7.                ^^^^^^^^^^^^^^^^^^^^^
  8.   File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
  9.     response = wrapped_callback(request, *callback_args, **callback_kwargs)
  10.                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  11.   File "/usr/local/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
  12.     return view_func(*args, **kwargs)
  13.            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  14.   File "/usr/local/lib/python3.11/site-packages/rest_framework/viewsets.py", line 125, in view
  15.     return self.dispatch(request, *args, **kwargs)
  16.            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  17.   File "/code/api/views.py", line 352, in dispatch
  18.     return super().dispatch(request, *args, **kwargs)
  19.            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  20.   File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch
  21.     response = self.handle_exception(exc)
  22.                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  23.   File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
  24.     self.raise_uncaught_exception(exc)
  25.   File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
  26.     raise exc
  27.   File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
  28.     response = handler(request, *args, **kwargs)
  29.                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  30.   File "/usr/local/lib/python3.11/site-packages/rest_framework/mixins.py", line 19, in create
  31.     self.perform_create(serializer)
  32.   File "/code/api/views.py", line 375, in perform_create
  33.     serializer.save()
  34.   File "/usr/local/lib/python3.11/site-packages/rest_framework/serializers.py", line 212, in save
  35.     self.instance = self.create(validated_data)
  36.                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  37.   File "/code/api/serializers.py", line 286, in create
  38.     instance = super().create(validated_data)
  39.                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  40.   File "/code/api/serializers.py", line 125, in create
  41.     db_file.save()
  42.   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 814, in save
  43.     self.save_base(
  44.   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 877, in save_base
  45.     updated = self._save_table(
  46.               ^^^^^^^^^^^^^^^^^
  47.   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1020, in _save_table
  48.     results = self._do_insert(
  49.               ^^^^^^^^^^^^^^^^
  50.   File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1061, in _do_insert
  51.     return manager._insert(
  52.            ^^^^^^^^^^^^^^^^
  53.   File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
  54.     return getattr(self.get_queryset(), name)(*args, **kwargs)
  55.            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  56.   File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1805, in _insert
  57.     return query.get_compiler(using=using).execute_sql(returning_fields)
  58.            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  59.   File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1821, in execute_sql
  60.     for sql, params in self.as_sql():
  61.                        ^^^^^^^^^^^^^
  62.   File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1745, in as_sql
  63.     value_rows = [
  64.                  ^
  65.   File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1746, in <listcomp>
  66.     [
  67.   File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1747, in <listcomp>
  68.     self.prepare_value(field, self.pre_save_val(field, obj))
  69.                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  70.   File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1695, in pre_save_val
  71.     return field.pre_save(obj, add=True)
  72.            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  73.   File "/usr/local/lib/python3.11/site-packages/django/db/models/fields/files.py", line 317, in pre_save
  74.     file.save(file.name, file.file, save=False)
  75.   File "/usr/local/lib/python3.11/site-packages/django/db/models/fields/files.py", line 93, in save
  76.     self.name = self.storage.save(name, content, max_length=self.field.max_length)
  77.                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  78.   File "/usr/local/lib/python3.11/site-packages/django/core/files/storage/base.py", line 38, in save
  79.     name = self._save(name, content)
  80.            ^^^^^^^^^^^^^^^^^^^^^^^^^
  81.   File "/usr/local/lib/python3.11/site-packages/django/core/files/storage/filesystem.py", line 130, in _save
  82.     os.chmod(full_path, self.file_permissions_mode)
  83. FileNotFoundError: [Errno 2] No such file or directory: '/sftp/customer_name/_web_portal/PendingOrders/7935/zip_03.zip'

奇怪的是，当我查看文件夹时，文件就在那里。Apache运行的用户可以访问它。
尝试检查inotifywait发生了什么，这是上传目录的输出：

  1. Setting up watches.  Beware: since -r was given, this may take a while!
  2. Watches established.
  3. /sftp/customer_name/_web_portal/PendingOrders/ CREATE,ISDIR 7982
  4. /sftp/customer_name_web_portal/PendingOrders/ OPEN,ISDIR 7982
  5. /sftp/customer_name/_web_portal/PendingOrders/ ACCESS,ISDIR 7982
  6. /sftp/customer_name/_web_portal/PendingOrders/ CLOSE_NOWRITE,CLOSE,ISDIR 7982
  7. # --> between those two records the error is thrown
  8. /sftp/customer_name/_web_portal/PendingOrders/7982/ CLOSE_WRITE,CLOSE zip_02.zip

还尝试在错误之前编辑django文件（就在os.chmod.

  1. os.listdir(directory)=['zip_10MB.zip']
  2. os.path.exists(full_path)=False
  3. os.path.isfile(full_path)=False
  4. repr(full_path)="'/sftp/customer_name/_web_portal/PendingOrders/8013/zip_10MB.zip'" repr(directory)="'/sftp/customer_name/_web_portal/PendingOrders/8013'"

这对我来说是不真实的，因为listdir输出文件夹中有一个文件，但os.path.exists输出False？！我发现os.path.exists()和isfile在下面调用os.stat()来查找必要的信息，.所以我尝试了os.stat(full_path)，它失败了，出现了FileNotFoundError。
在抛出错误后，我用运行Apache的同一个用户登录到Docker，并尝试在所述文件路径上运行os.stat()，它返回OK（fileinfo）。
我还检查了Django在将文件从临时目录移动到最终目的地时使用的操作-它是django/core/files/move.py file_move_safe函数-它是一个普通的os.rename()
我尝试的下一件事是把time.sleep()放在实际错误之前，cd-ing在上传目录中，并对每个用户执行ls命令，输出如下：root用户：

  1. total 10432
  2. drwxrwx---    1 web_port web_port       512 Aug 17 09:48 .
  3. drwxrwx---    1 web_port web_port       512 Aug 17 09:48 ..
  4. -rw-------    1 web_port web_port  10679630 Aug 17 09:48 zip_10MB.zip

web_portal（apache / application）用户：

  1. ls: ./zip_10MB.zip: No such file or directory
  2. total 0
  3. drwxrwx---    1 web_port web_port       512 Aug 17 09:48 .
  4. drwxrwx---    1 web_port web_port       512 Aug 17 09:48 ..

PS：我使用的是python:3.11-alpine Docker镜像（Docker Desktop），有Django==4.2.4和djangorestframework==3.14.0。
更新：尝试将我的Docker镜像/容器放在Linux操作系统上-文件上传工作没有问题。所以我猜Docker for Windows有问题，或者在Windows上如何处理“绑定”挂载。也许吧
更新#2：得到了一个想法，如果也许该文件被Windows Defender或其他东西锁定。idk我快没主意了。所以我已经禁用了Windows Defender，在文件夹上添加了例外，..错误仍然存在。接下来，我尝试在文件夹上启用Windows审核日志记录，但我不知道如何从安全日志的输出中获得意义。
有没有人知道问题可能是什么，或者我下一步可以尝试什么来进一步限制这个问题的原因？

apache

1条答案

按热度按时间
j7dteeu8

j7dteeu81#

尝试在你的settings.py文件中设置MAX_UPLOADING_FILE_SIZE，这个解决方案可以帮助我im我的项目：

  1. MAX_UPLOADING_FILE_SIZE = 31457280  # 30 MB
赞(0）分享  回复(0）举报  2023-10-23
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com