python 在ctypes加载的共享库中卸载共享库

oknwwptz  于 2023-01-12  发布在  Python
关注(0)|答案(2)|浏览(118)

我正在从我的python脚本中调用一个so文件。据我所知,我并不真的需要释放在python中使用ctypes打开的共享库。然而,在我的so文件代码中,它dlopen了另一个so文件,并且没有dlclose()。在这种情况下,从python端使用它安全吗?我不需要释放在ctypes loade so文件中打开的共享库吗?

mf98qq94

mf98qq941#

规则***清理后,你自己***总是适用(虽然现代技术照顾清洁方面为您)。
【Python文档】:ctypes -Python的一个外来函数库包含了很多有用的信息,应该是你的朋友。

  • CTypes* 在加载 .dll.so)时使用 DlOpen。正如我所注意到的,它调用相应的 DlClose,这意味着 .dll(以及加载它时加载的所有依赖项)将保留在内存中,直到进程终止(或显式卸载)。

[Man7]: DLOPEN(3)开始:
如果由 filename 指定的对象依赖于其他共享对象,则动态链接器也会使用相同的规则自动加载这些共享对象(如果这些对象反过来又具有依赖性,则此过程可能递归发生,依此类推)。
...
如果用**dlopen()再次加载同一个共享对象,则返回同一个对象句柄。动态链接器维护对象句柄的引用计数,因此动态加载的共享对象不会被释放,直到dlclose()被调用的次数与dlopen()**成功调用的次数相同。任何初始化返回(见下文)只被调用一次。
所以,我不认为你会有问题(当然,一切都取决于上下文).正如你所注意到的,多次加载一个库并不是每次都加载它,所以耗尽内存的可能性很小(除非你加载大量不同的 .dll,每个都有很多不同的依赖项).
我能想到的一种情况是加载使用另一个 .dll 中的符号的 .dll。如果该符号也在之前加载的另一个(3rd.dll 中定义,则代码的行为将与预期不同。
不管怎样,你可以手动卸载(或者更好:减少它的 RefCount)a .dll(我不确定这是否符合 * 推荐方式 * 或 * 最佳实践 *),如下例所示。

  • dll00.c*:
#include <stdio.h>

int test()
{
    printf("[%s] (%d) - [%s]\n", __FILE__, __LINE__, __FUNCTION__);
    return 0;
}
  • 代码00.py *:
#!/usr/bin/env python

import ctypes as cts
import sys

DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")

def _load_dll(name):
    dll = cts.CDLL(name)
    print(dll)
    return dll

def _load_test_func(dll):
    testf = dll.test
    testf.restype = cts.c_int
    return testf

def main(*argv):
    dlclose = cts.CDLL(None).dlclose  # This WON'T work on Win
    dlclose.argtypes = (cts.c_void_p,)

    print("Loading a dll via `ctypes`, then delete the object. The dll is not unloaded. Call `dlclose` to unload. A 2nd call will fail.")
    dll = _load_dll(DLL_NAME)
    handle = dll._handle
    del dll
    print("{:} returned {:d}".format(dlclose.__name__, dlclose(handle)))  # Even if the ctypes dll object was destroyed, the dll wasn't unloaded
    print("{:} returned {:d}".format(dlclose.__name__, dlclose(handle)))  # A new dlclose call will fail

    print("\nUse `ctypes` to load the dll twice. The dll is not actually loaded only the 1st time (both have the same handle), but its ref count is increased. `dlclose` must be also called twice.")
    dll0 = _load_dll(DLL_NAME)
    dll1 = _load_dll(DLL_NAME)
    print("{:} returned {:d}".format(dlclose.__name__, dlclose(dll0._handle)))
    print("{:} returned {:d}".format(dlclose.__name__, dlclose(dll1._handle)))
    print("{:} returned {:d}".format(dlclose.__name__, dlclose(dll1._handle)))

    print("\nLoad a dll via `ctypes`, and load one of its funcs. Try calling it before and after unloading the dll.")
    dll = _load_dll(DLL_NAME)
    test = _load_test_func(dll)
    print("{:} returned {:d}".format(test.__name__, test()))
    print("{:} returned {:d}".format(dlclose.__name__, dlclose(dll._handle)))
    print("{:} returned {:d}".format(test.__name__, test()))  # Comment this line as it would segfault !!!

if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.\n")
    sys.exit(rc)

输出

(qaic-env) [cfati@cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q052179325]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[064bit prompt]> ls
code00.py  dll00.c
[064bit prompt]>
[064bit prompt]> gcc -fPIC -shared -o dll00.so dll00.c
[064bit prompt]>
[064bit prompt]> python ./code00.py
Python 3.8.10 (default, Nov 14 2022, 12:59:47) [GCC 9.4.0] 064bit on linux

Loading a dll via `ctypes`, then delete the object. The dll is not unloaded. Call `dlclose` to unload. A 2nd call will fail.
<CDLL './dll00.so', handle 137b280 at 0x7f75c7863c10>
dlclose returned 0
dlclose returned -1

Use `ctypes` to load the dll twice. The dll is not actually loaded only the 1st time (both have the same handle), but its ref count is increased. `dlclose` must be also called twice.
<CDLL './dll00.so', handle 137b630 at 0x7f75c7863c10>
<CDLL './dll00.so', handle 137b630 at 0x7f75c777e340>
dlclose returned 0
dlclose returned 0
dlclose returned -1

Load a dll via `ctypes`, and load one of its funcs. Try calling it before and after unloading the dll.
<CDLL './dll00.so', handle 13edea0 at 0x7f75c777e220>
[dll00.c] (6) - [test]
test returned 0
dlclose returned 0
Segmentation fault

相关(或多或少):

h7appiyu

h7appiyu2#

卸载依赖项的示例

在Linux Fedora 32、Python 3.7.6(anaconda)、ctypes 1.1.0、g++ 10.2.1上进行了测试。依赖项为OpenCv(版本4.2)。更多详细信息请访问:How can I unload a DLL using ctypes in Python?

代码.cpp

#include <opencv2/core/core.hpp>
#include <iostream> 

extern "C" int my_fct(int n)
{
    cv::Mat1b mat = cv::Mat1b(10,8,(unsigned char) 1 );  // change 1 to test unloading
    
    return mat(0,1) * n;
}

使用g++ code.cpp -shared -fPIC -Wall -std=c++17 -I/usr/include/opencv4 -lopencv_core -o so_opencv.so编译
Python代码

from sys import platform
import ctypes

class CtypesLib:

    def __init__(self, fp_lib, dependencies=[]):
        self._dependencies = [CtypesLib(fp_dep) for fp_dep in dependencies]

        if platform == "linux" or platform == "linux2":  # Linux
            self._dlclose_func = ctypes.cdll.LoadLibrary('').dlclose
            self._dlclose_func.argtypes = [ctypes.c_void_p]
            self._ctypes_lib = ctypes.cdll.LoadLibrary(fp_lib)
        elif platform == "win32":  # Windows
            self._ctypes_lib = ctypes.WinDLL(fp_lib)

        self._handle = self._ctypes_lib._handle

    def __getattr__(self, attr):
        return self._ctypes_lib.__getattr__(attr)

    def __del__(self):
        for dep in self._dependencies:
            del dep

        del self._ctypes_lib

        if platform == "linux" or platform == "linux2":  # Linux
            self._dlclose_func(self._handle)
        elif platform == "win32":  # Windows
            ctypes.windll.kernel32.FreeLibrary(self._handle)

fp_lib = './so_opencv.so'

ctypes_lib = CtypesLib(fp_lib, ['/usr/lib64/libopencv_core.so'])

valIn = 1
ctypes_lib.my_fct.argtypes = [ctypes.c_int]
ctypes_lib.my_fct.restype = ctypes.c_int
valOut = ctypes_lib.my_fct(valIn)
print(valIn, valOut)

del ctypes_lib

相关问题