我需要创建一个Shell,它利用jq从一个相当大的json文件中提取一些数据

z9ju0rcb  于 2023-02-17  发布在  Shell
关注(0)|答案(1)|浏览(101)

所以我一直在努力解决这个问题。
这是JSON文件中的一部分数据,其余的数据使用相同的结构。

`"data": {
 "master.services": {
  "host1": [
    {
      "cert_cn": "CN=host1-signer1:",
      "days_remaining": 412,
      "expiry": "2024-03-18 17:21:41",
      "health": "ok",
      "issuer": "CN=host1-signer1",
      "path": "/etc/host1/ca.crt",
      "serial": xxxxxxxxxxx,
      "serial_hex": "xxxxxxxx"
    },
    {
      "cert_cn": "CN=host1-signer2",
      "days_remaining": 544,
      "expiry": "2024-07-28 12:47:50",
      "health": "ok",
      "issuer": "CN=host1-signer2",
      "path": "/etc/host1/server.crt",
      "serial": 12,
      "serial_hex": "0xc"
    },
    {
      "cert_cn": "CN=host3-signer1",
      "days_remaining": 544,
      "expiry": "2024-07-28 12:47:52",
      "health": "ok",
      "issuer": "CN=host3-signer1",
      "path": "/etc/host3/peer.crt",
      "serial": 14,
      "serial_hex": "0xe"
    }
  ],
  "host4": [
    {
      "cert_cn": "CN=host4-signer1",
      "days_remaining": 729,
      "expiry": "2025-01-29 17:10:11",
      "health": "ok",
      "issuer": "xxxxxxxxxxxxxxxxxxxxxxxxx",
      "path": "/etc/origin/master/admin.host4",
      "serial": 6,
      "serial_hex": "0x6"
        }
       ],
 }

`

我试着编写一个使用jq提取以下数据的shell:

"master-services(name of master Service)":
      host1:....
        cert_cn:....
        days_remaining:....
        expiry:.......
      host2:.....
        cert_cn:....
        days_remaining:....
        expiry:.......

等等......我已经提取了host 1、cert_cn等的数据......但是我就是不能在所属数据前面得到主服务名。我想这是因为主服务名在{}括号中,因此被定义为一个对象,而不是数组......此外,我已经尝试过了,如果有人有任何提示,我将非常感谢......
此外,第一次张贴,所以如果还有什么需要,或我需要删除的东西,请告诉我!
编辑:为了澄清,有多个“master.services“,有多个“host“。我需要一个jq命令,它可以在shell中执行,输出:“主服务”(以便可以识别将要用完的证书),然后是bei主机,后面只跟有“cert_cn”,“days_remaining”和“expiry”,如上所示。输出需要是xml格式,但那个问题会被处理,在我解决这个问题之后。到目前为止我遇到的问题,是“master.service”中的点,这使得直接寻址它们变得不可能。之后,我试图通过做如下操作来导航JSON:jq -r '.data | .[1] | {cert_cn, days_remaining, expiry}'但是因为“master.service”是在“{}”括号中而不是[]括号中,所以我不能用数字索引它们,因为它们是对象而不是数组。已经感谢您的评论,并为这个不清楚的问题感到抱歉,我还在学习英语。

lf5gs5x2

lf5gs5x21#

上面给出的示例文件不是一个有效的JSON。包含未加引号的字符串,缺少左大括号和几个缺少的右大括号。右大括号在这里(假设它是一个名为duda_jq_stackoverflow.json的文件)。
如果你执行

oscar@toreyes-i7:/tmp$ cat duda_jq_stackoverflow.json

显示内容为:

{
    "data": {
         "master.services": {
             "host1": [
               {
                 "cert_cn": "CN=host1-signer1:",
                 "days_remaining": 412,
                 "expiry": "2024-03-18 17:21:41",
                 "health": "ok",
                 "issuer": "CN=host1-signer1",
                 "path": "/etc/host1/ca.crt",
                 "serial": "xxxxxxxxxxx",
                 "serial_hex": "xxxxxxxx"
               },
               {
                 "cert_cn": "CN=host1-signer2",
                 "days_remaining": 544,
                 "expiry": "2024-07-28 12:47:50",
                 "health": "ok",
                 "issuer": "CN=host1-signer2",
                 "path": "/etc/host1/server.crt",
                 "serial": 12,
                 "serial_hex": "0xc"
               },
               {
                 "cert_cn": "CN=host3-signer1",
                 "days_remaining": 544,
                 "expiry": "2024-07-28 12:47:52",
                 "health": "ok",
                 "issuer": "CN=host3-signer1",
                 "path": "/etc/host3/peer.crt",
                 "serial": 14,
                 "serial_hex": "0xe"
               }
             ],
             "host4": [
               {
                 "cert_cn": "CN=host4-signer1",
                 "days_remaining": 729,
                 "expiry": "2025-01-29 17:10:11",
                 "health": "ok",
                 "issuer": "xxxxxxxxxxxxxxxxxxxxxxxxx",
                 "path": "/etc/origin/master/admin.host4",
                 "serial": 6,
                 "serial_hex": "0x6"
               }
             ]
         }
    }
}

如果你执行

oscar@toreyes-i7:/tmp$ jq '.data["master.services"]'  duda_jq_stackoverflow.json

你会得到原始JSON的下一个片段:

{
  "host1": [
    {
      "cert_cn": "CN=host1-signer1:",
      "days_remaining": 412,
      "expiry": "2024-03-18 17:21:41",
      "health": "ok",
      "issuer": "CN=host1-signer1",
      "path": "/etc/host1/ca.crt",
      "serial": "xxxxxxxxxxx",
      "serial_hex": "xxxxxxxx"
    },
    {
      "cert_cn": "CN=host1-signer2",
      "days_remaining": 544,
      "expiry": "2024-07-28 12:47:50",
      "health": "ok",
      "issuer": "CN=host1-signer2",
      "path": "/etc/host1/server.crt",
      "serial": 12,
      "serial_hex": "0xc"
    },
    {
      "cert_cn": "CN=host3-signer1",
      "days_remaining": 544,
      "expiry": "2024-07-28 12:47:52",
      "health": "ok",
      "issuer": "CN=host3-signer1",
      "path": "/etc/host3/peer.crt",
      "serial": 14,
      "serial_hex": "0xe"
    }
  ],
  "host4": [
    {
      "cert_cn": "CN=host4-signer1",
      "days_remaining": 729,
      "expiry": "2025-01-29 17:10:11",
      "health": "ok",
      "issuer": "xxxxxxxxxxxxxxxxxxxxxxxxx",
      "path": "/etc/origin/master/admin.host4",
      "serial": 6,
      "serial_hex": "0x6"
    }
  ]
}

我建议您使用gron程序来获得您必须使用jq的路径(在输出中,删除每行开头的"json"部分)。
您可以在https://github.com/tomnomnom/gron中获得gron
如果您执行命令

oscar@toreyes-i7:/tmp$ gron duda_jq_stackoverflow.json

你得到

json = {};
json.data = {};
json.data["master.services"] = {};
json.data["master.services"].host1 = [];
json.data["master.services"].host1[0] = {};
json.data["master.services"].host1[0].cert_cn = "CN=host1-signer1:";
json.data["master.services"].host1[0].days_remaining = 412;
json.data["master.services"].host1[0].expiry = "2024-03-18 17:21:41";
json.data["master.services"].host1[0].health = "ok";
json.data["master.services"].host1[0].issuer = "CN=host1-signer1";
json.data["master.services"].host1[0].path = "/etc/host1/ca.crt";
json.data["master.services"].host1[0].serial = "xxxxxxxxxxx";
json.data["master.services"].host1[0].serial_hex = "xxxxxxxx";
json.data["master.services"].host1[1] = {};
json.data["master.services"].host1[1].cert_cn = "CN=host1-signer2";
json.data["master.services"].host1[1].days_remaining = 544;
json.data["master.services"].host1[1].expiry = "2024-07-28 12:47:50";
json.data["master.services"].host1[1].health = "ok";
json.data["master.services"].host1[1].issuer = "CN=host1-signer2";
json.data["master.services"].host1[1].path = "/etc/host1/server.crt";
json.data["master.services"].host1[1].serial = 12;
json.data["master.services"].host1[1].serial_hex = "0xc";
json.data["master.services"].host1[2] = {};
json.data["master.services"].host1[2].cert_cn = "CN=host3-signer1";
json.data["master.services"].host1[2].days_remaining = 544;
json.data["master.services"].host1[2].expiry = "2024-07-28 12:47:52";
json.data["master.services"].host1[2].health = "ok";
json.data["master.services"].host1[2].issuer = "CN=host3-signer1";
json.data["master.services"].host1[2].path = "/etc/host3/peer.crt";
json.data["master.services"].host1[2].serial = 14;
json.data["master.services"].host1[2].serial_hex = "0xe";
json.data["master.services"].host4 = [];
json.data["master.services"].host4[0] = {};
json.data["master.services"].host4[0].cert_cn = "CN=host4-signer1";
json.data["master.services"].host4[0].days_remaining = 729;
json.data["master.services"].host4[0].expiry = "2025-01-29 17:10:11";
json.data["master.services"].host4[0].health = "ok";
json.data["master.services"].host4[0].issuer = "xxxxxxxxxxxxxxxxxxxxxxxxx";
json.data["master.services"].host4[0].path = "/etc/origin/master/admin.host4";
json.data["master.services"].host4[0].serial = 6;
json.data["master.services"].host4[0].serial_hex = "0x6";

在该输出中,我们可以看到jq使用的路径格式。

json.data["master.services"] = {};

并在开始时删除json,并丢弃从=开始的所有内容,剩下的是:

.data["master.services"]

这就是告诉jq提取字段值的路径。
请注意,由于字段名中包含.,因此必须用双引号将字段名括起来,然后依次用方括号[]将字段名括起来。
您可以得到值host1host4弹出:

oscar@toreyes-i7:/tmp$ jq -r '.data["master.services"] | keys_unsorted | .[]'  duda_jq_stackoverflow.json
host1
host4

使用这些值,您可以组合路径并使用jq进行另一个查询。
或者,这也可以成为你实现目标的起点:

oscar@toreyes-i7:/tmp$ jq -r '.data["master.services"]["host1"][] as $nodo | "\("cert_cn = " + $nodo.cert_cn + "\n" + "days_remaining = " +  ($nodo.days_remaining | tostring) + "\n" + "expiry " + $nodo.expiry + "\n")"'  duda_jq_stackoverflow.json
cert_cn = CN=host1-signer1:
days_remaining = 412
expiry 2024-03-18 17:21:41

cert_cn = CN=host1-signer2
days_remaining = 544
expiry 2024-07-28 12:47:50

cert_cn = CN=host3-signer1
days_remaining = 544
expiry 2024-07-28 12:47:52

这能回答你的问题吗?

相关问题