Python日志记录被阻塞:达多格

velaa5lx  于 2023-04-28  发布在  Python
关注(0)|答案(1)|浏览(154)

Python中有一个自定义日志处理程序,名为DataDogCustomLogHandler
这是初始化记录器的函数:

os.environ["DD_API_KEY"] = 'XXXXX'
os.environ["DD_SITE"] = 'XXXXX'
host_name = socket.gethostname()

def init_datadog_logging(service_name: str = None, env_name: str = None, min_log_level: int = logging.INFO):
    tags = f'service: {service_name}, host: {host_name}, environment: {env_name}'
    logging.basicConfig()
    logging.getLogger().setLevel(min_log_level)

    logger = logging.getLogger()
    if env_name == 'UAT' or env_name == 'Production':
        datadog_custom_handler = DatadogCustomLogHandler(tags=tags, service=service_name, level=min_log_level)
        logger.addHandler(datadog_custom_handler)

我有一个异步函数,它使用aiohttp抓取数据:

logger = logging.getLogger(__name__)

async def http_get(url: str) -> JSONObject:
    logger.info(f'Getting data from {url}')
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.json()

在我的main函数中,我运行如下:

init_datadog_logging('log.test', 'UAT', min_log_level=logging.INFO)
logger = logging.getLogger(__name__)

async def main_async() -> None:
    time_before = time.perf_counter()
    async with aiohttp.ClientSession() as session:
        tasks = []
        for date in dates:
            url = BASE_URL + date
            tasks.append(asyncio.ensure_future(http_get(url)))
        responses = await asyncio.gather(*tasks)

    logger.info(f'Total time taken: {time.perf_counter() - time_before}')

asyncio.run(main_async())

没有logger需要0.97秒,但只要我计算logger.info,它就变成了大约8秒。如何解锁logging库?或者,有没有更好的方法通过http发送datadog日志,而不用python阻塞?

5jdjgkvh

5jdjgkvh1#

您可以从查看Python文档的这一部分开始。基本上,您可以配置日志记录,以便使用的处理程序不会阻塞,方法是写入队列并返回-阻塞代码在(例如)单独的线程中运行。您知道您当前配置的DataDog处理程序是否会阻塞吗?如果是,它们阻塞的是什么?

相关问题