Python中有一个自定义日志处理程序,名为DataDogCustomLogHandler
。
这是初始化记录器的函数:
os.environ["DD_API_KEY"] = 'XXXXX'
os.environ["DD_SITE"] = 'XXXXX'
host_name = socket.gethostname()
def init_datadog_logging(service_name: str = None, env_name: str = None, min_log_level: int = logging.INFO):
tags = f'service: {service_name}, host: {host_name}, environment: {env_name}'
logging.basicConfig()
logging.getLogger().setLevel(min_log_level)
logger = logging.getLogger()
if env_name == 'UAT' or env_name == 'Production':
datadog_custom_handler = DatadogCustomLogHandler(tags=tags, service=service_name, level=min_log_level)
logger.addHandler(datadog_custom_handler)
我有一个异步函数,它使用aiohttp
抓取数据:
logger = logging.getLogger(__name__)
async def http_get(url: str) -> JSONObject:
logger.info(f'Getting data from {url}')
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.json()
在我的main
函数中,我运行如下:
init_datadog_logging('log.test', 'UAT', min_log_level=logging.INFO)
logger = logging.getLogger(__name__)
async def main_async() -> None:
time_before = time.perf_counter()
async with aiohttp.ClientSession() as session:
tasks = []
for date in dates:
url = BASE_URL + date
tasks.append(asyncio.ensure_future(http_get(url)))
responses = await asyncio.gather(*tasks)
logger.info(f'Total time taken: {time.perf_counter() - time_before}')
asyncio.run(main_async())
没有logger
需要0.97秒,但只要我计算logger.info
,它就变成了大约8秒。如何解锁logging
库?或者,有没有更好的方法通过http发送datadog日志,而不用python阻塞?
1条答案
按热度按时间5jdjgkvh1#
您可以从查看Python文档的这一部分开始。基本上,您可以配置日志记录,以便使用的处理程序不会阻塞,方法是写入队列并返回-阻塞代码在(例如)单独的线程中运行。您知道您当前配置的DataDog处理程序是否会阻塞吗?如果是,它们阻塞的是什么?