python-3.x 使用Pydantic设置配置多个YAML文件

szqfcxe2  于 2023-03-31  发布在  Python
关注(0)|答案(1)|浏览(385)

我正在考虑使用pydantic_settings_yaml将YAML配置文件加载到Pydantic模型中。理想情况下,我应该有一个包含环境无关配置的global.yml,然后是一个环境特定配置的env.yml。目前还不清楚pydantic_settings_yaml如何处理这个问题。我希望避免每次加载配置时都创建一个合并的.yml。
下面的代码将把dev.yml或live.yml读入Pydantic模型,但不读入global.yml。

app_config.py

env = os.getenv("ENVIRONMENT", "").lower()

match env:
    case "dev":
        yaml_filename = "dev.yml"
        secrets_sub_dir = "dev"
    case "live":
        yaml_filename = "live.yml"
        secrets_sub_dir = "live"
    case _:
        raise Exception(f"Unrecognised environment: {env}")

class Settings(YamlBaseSettings):
    test_api_one: Dict[str, str]

    class Config:
        secrets_dir = f"/secrets/{secrets_sub_dir}"
        yaml_file = Path(__file__).parent / yaml_filename

config = Settings()

dev.yml

'test_api_one':
  'uri': 'https://dev_api/'
  'secret': <file:dev_secret>

live.yml

'test_api_one':
  'uri': 'https://live_api/'
  'secret': <file:live_secret>

global.yml

'test_api_one':
  'dateFormat': '%Y-%m-%d'
  'dateTimeFormat': '%Y-%m-%dT%H:%M:%SZ'
lx0bsm1f

lx0bsm1f1#

由于我对你提到的一揽子计划有疑问(见我上面的评论),我建议你自己实施。
Pydantic文档解释了如何自定义设置源。为了确保嵌套字典“适当地”更新,您还可以使用非常方便的pydantic.utils.deep_update函数。
显然,您需要安装pyyaml才能使其工作。
下面是一个例子:

from pathlib import Path
from typing import Any

from pydantic import BaseSettings as PydanticBaseSettings
from pydantic.env_settings import SettingsSourceCallable
from pydantic.utils import deep_update
from yaml import safe_load

THIS_DIR = Path(__file__).parent

class BaseSettings(PydanticBaseSettings):
    test_api_one: dict[str, str]

    class Config:
        config_files = [
            Path(THIS_DIR, "global.yaml"),
            Path(THIS_DIR, "dev.yaml"),
        ]

        @classmethod
        def customise_sources(
                cls,
                init_settings: SettingsSourceCallable,
                env_settings: SettingsSourceCallable,
                file_secret_settings: SettingsSourceCallable
        ) -> tuple[SettingsSourceCallable, ...]:
            return init_settings, env_settings, config_file_settings

def config_file_settings(settings: PydanticBaseSettings) -> dict[str, Any]:
    config: dict[str, Any] = {}
    if not isinstance(settings, BaseSettings):
        return config
    for path in settings.Config.config_files:
        if not path.is_file():
            print(f"No file found at `{path.resolve()}`")
            continue
        print(f"Reading config file `{path.resolve()}`")
        if path.suffix in {".yaml", ".yml"}:
            config = deep_update(config, load_yaml(path))
        else:
            print(f"Unknown config file extension `{path.suffix}`")
    return config

def load_yaml(path: Path) -> dict[str, Any]:
    with Path(path).open("r") as f:
        config = safe_load(f)
    if not isinstance(config, dict):
        raise TypeError(
            f"Config file has no top-level mapping: {path}"
        )
    return config

现在假设这些是有问题的文件:
global.yaml

test_api_one:
  date_format: '%Y-%m-%d'
  date_time_format: '%Y-%m-%dT%H:%M:%SZ'

dev.yaml

test_api_one:
  uri: 'https://dev_api/'
  secret: dev

执行print(BaseSettings().json(indent=4))的输出是这样的:
一个三个三个一个

相关问题