在yaml中从CSV文件生成模板

m0rkklqb  于 2023-10-13  发布在  其他
关注(0)|答案(2)|浏览(115)

我试图创建yaml文件从一个模板,使用我的变量。我的yaml模板看起来像这样

  1. number: {{NUMBER}}
  2. name: {{NAME}}
  3. region: {{REGION}}
  4. storenum: {{STORENUM}}
  5. clients: {{CLIENTS}}
  6. tags: {{TAGS}}
  7. storename: {{STORENAME}}
  8. employee: {{EMPLOYEE}}
  9. products: {{PRODUCTS}}

但我的变量是在一个CSV文件的结构是变量。

  1. Number - Name - Region - Storenum
  2. StoreX - StoreX - New York - 30

我现在有一个小脚本,从一个模板创建变量参数和模板像这样script. shtemplate. yml-fvariables.txt。结果是这样的

  1. number: 37579922
  2. name: Store1
  3. region: New York
  4. storenum: 32
  5. clients: 100
  6. tags: stores
  7. storename: Store newyork
  8. employee: 10
  9. products: 200

但我只能一个接一个地做。有没有办法读取CSV参数并发送到程序并生成例如Template 1,Template 2,..从CSV参数?任何帮助

  1. #!/bin/bash
  2. readonly PROGNAME=$(basename $0)
  3. config_file="<none>"
  4. print_only="false"
  5. silent="false"
  6. usage="${PROGNAME} [-h] [-d] [-f] [-s] --
  7. where:
  8. -h, --help
  9. Show this help text
  10. -p, --print
  11. Don't do anything, just print the result of the variable expansion(s)
  12. -f, --file
  13. Specify a file to read variables from
  14. -s, --silent
  15. Don't print warning messages (for example if no variables are found)
  16. examples:
  17. VAR1=Something VAR2=1.2.3 ${PROGNAME} test.txt
  18. ${PROGNAME} test.txt -f my-variables.txt
  19. ${PROGNAME} test.txt -f my-variables.txt > new-test.txt"
  20. if [ $# -eq 0 ]; then
  21. echo "$usage"
  22. exit 1
  23. fi
  24. if [[ ! -f "${1}" ]]; then
  25. echo "You need to specify a template file" >&2
  26. echo "$usage"
  27. exit 1
  28. fi
  29. template="${1}"
  30. if [ "$#" -ne 0 ]; then
  31. while [ "$#" -gt 0 ]
  32. do
  33. case "$1" in
  34. -h|--help)
  35. echo "$usage"
  36. exit 0
  37. ;;
  38. -p|--print)
  39. print_only="true"
  40. ;;
  41. -f|--file)
  42. config_file="$2"
  43. ;;
  44. -s|--silent)
  45. silent="true"
  46. ;;
  47. --)
  48. break
  49. ;;
  50. -*)
  51. echo "Invalid option '$1'. Use --help to see the valid options" >&2
  52. exit 1
  53. ;;
  54. # an option argument, continue
  55. *) ;;
  56. esac
  57. shift
  58. done
  59. fi
  60. vars=$(grep -oE '\{\{[A-Za-z0-9_]+\}\}' "${template}" | sort | uniq | sed -e 's/^{{//' -e 's/}}$//')
  61. if [[ -z "$vars" ]]; then
  62. if [ "$silent" == "false" ]; then
  63. echo "Warning: No variable was found in ${template}, syntax is {{VAR}}" >&2
  64. fi
  65. fi
  66. # Load variables from file if needed
  67. if [ "${config_file}" != "<none>" ]; then
  68. if [[ ! -f "${config_file}" ]]; then
  69. echo "The file ${config_file} does not exists" >&2
  70. echo "$usage"
  71. exit 1
  72. fi
  73. source "${config_file}"
  74. fi
  75. var_value() {
  76. eval echo \$$1
  77. }
  78. replaces=""
  79. # Reads default values defined as {{VAR=value}} and delete those lines
  80. # There are evaluated, so you can do {{PATH=$HOME}} or {{PATH=`pwd`}}
  81. # You can even reference variables defined in the template before
  82. defaults=$(grep -oE '^\{\{[A-Za-z0-9_]+=.+\}\}' "${template}" | sed -e 's/^{{//' -e 's/}}$//')
  83. for default in $defaults; do
  84. var=$(echo "$default" | grep -oE "^[A-Za-z0-9_]+")
  85. current=`var_value $var`
  86. # Replace only if var is not set
  87. if [[ -z "$current" ]]; then
  88. eval $default
  89. fi
  90. # remove define line
  91. replaces="-e '/^{{$var=/d' $replaces"
  92. vars="$vars
  93. $current"
  94. done
  95. vars=$(echo $vars | sort | uniq)
  96. if [[ "$print_only" == "true" ]]; then
  97. for var in $vars; do
  98. value=`var_value $var`
  99. echo "$var = $value"
  100. done
  101. exit 0
  102. fi
  103. # Replace all {{VAR}} by $VAR value
  104. for var in $vars; do
  105. value=$(var_value $var | sed -e "s;\&;\\\&;g" -e "s;\ ;\\\ ;g") # '&' and <space> is escaped
  106. if [[ -z "$value" ]]; then
  107. if [ $silent == "false" ]; then
  108. echo "Warning: $var is not defined and no default is set, replacing by empty" >&2
  109. fi
  110. fi
  111. # Escape slashes
  112. value=$(echo "$value" | sed 's/\//\\\//g');
  113. replaces="-e 's/{{$var}}/${value}/g' $replaces"
  114. done
  115. escaped_template_path=$(echo $template | sed 's/ /\\ /g')
  116. eval sed $replaces "$escaped_template_path"
polkgigr

polkgigr1#

但我只能一个接一个地做。有没有办法读取CSV参数并发送到程序并生成例如Template 1,Template 2,..从CSV参数?任何帮助
我不能拿出一个完整的解决方案,但我有一个解决方案,可以创建多个文件。把它融入你的剧本。
file.csv

  1. Number,Name,Region,Storenum,Clients,Tags,Storename,Employee,Products
  2. 88899223,Store1,New York,30,100,stores,Store newyork,10,200
  3. 37579922,Store2,Chicago,30,1000,stores,Store Chicago,10,200
  4. 77777777,Store3,New Orleans,309,100,stores,Store New Orleans,10,200
  5. 55555555,Store4,New Jersey,50,100,stores,Store Jersey,10,200
  6. 44444444,Store5,Connecticut,90,100,stores,Store Connecticut,10,200
  7. 33333333,Store6,Michigan,,900,stores,Store Michigan,10,200
  8. 22222222,Store7,Texas,30,200,stores,,10,200

Template.yaml

  1. number: {{NUMBER}}
  2. name: {{NAME}}
  3. region: {{REGION}}
  4. storenum: {{STORENUM}}
  5. clients: {{CLIENTS}}
  6. tags: {{TAGS}}
  7. storename: {{STORENAME}}
  8. employee: {{EMPLOYEE}}
  9. products: {{PRODUCTS}}
  1. #!/usr/bin/env bash
  2. n=1
  3. skip=0
  4. start=-1
  5. while IFS=, read -r number name region storenum clients tags storename employee products; do
  6. if ((start++ >= skip)); then
  7. export NUMBER="${number:-none}" NAME="${name:-none}" REGION="${region:-none}" \
  8. STORENUM="${storenum:-none}" CLIENTS="${clients:-none}" TAGS="${tags:-none}" \
  9. STORENAME="${storename:-none}" EMPLOYEE="${employee:-none}" PRODUCTS="${products:-none}"
  10. sed 's/{{/$/;s/}}//g' Template.yaml |
  11. envsubst '$NUMBER $NAME $REGION $STORENUM $CLIENTS $TAGS $STORENAME $EMPLOYEE $PRODUCTS' > "Template$n"
  12. ((n++))
  13. fi
  14. done < file.csv

通过运行以下命令检查创建的文件:

  1. tail -n+1 Template[0-9]*

输出为:

  1. ==> Template1 <==
  2. number: 88899223
  3. name: Store1
  4. region: New York
  5. storenum: 30
  6. clients: 100
  7. tags: stores
  8. storename: Store newyork
  9. employee: 10
  10. products: 200
  11. ==> Template2 <==
  12. number: 37579922
  13. name: Store2
  14. region: Chicago
  15. storenum: 30
  16. clients: 1000
  17. tags: stores
  18. storename: Store Chicago
  19. employee: 10
  20. products: 200
  21. ==> Template3 <==
  22. number: 77777777
  23. name: Store3
  24. region: New Orleans
  25. storenum: 309
  26. clients: 100
  27. tags: stores
  28. storename: Store New Orleans
  29. employee: 10
  30. products: 200
  31. ==> Template4 <==
  32. number: 55555555
  33. name: Store4
  34. region: New Jersey
  35. storenum: 50
  36. clients: 100
  37. tags: stores
  38. storename: Store Jersey
  39. employee: 10
  40. products: 200
  41. ==> Template5 <==
  42. number: 44444444
  43. name: Store5
  44. region: Connecticut
  45. storenum: 90
  46. clients: 100
  47. tags: stores
  48. storename: Store Connecticut
  49. employee: 10
  50. products: 200
  51. ==> Template6 <==
  52. number: 33333333
  53. name: Store6
  54. region: Michigan
  55. storenum: none
  56. clients: 900
  57. tags: stores
  58. storename: Store Michigan
  59. employee: 10
  60. products: 200
  61. ==> Template7 <==
  62. number: 22222222
  63. name: Store7
  64. region: Texas
  65. storenum: 30
  66. clients: 200
  67. tags: stores
  68. storename: none
  69. employee: 10
  70. products: 200
  71. ==> Template8 <==
  72. number: 22222222
  73. name: Store7
  74. region: Texas
  75. storenum: 30
  76. clients: 200
  77. tags: stores
  78. storename: none
  79. employee: 10
  80. products: 200
  • 上面的脚本需要envsubstsed,因为exportbash shell的内置。
  • 如果字段中没有值,则默认值为none
  • 警告:对于大文件/数据大小,shell本身就很慢。
展开查看全部
cuxqih21

cuxqih212#

这从来没有得到一个公认的答案。我提供第二个部分解决方案。
从这个开始:

  1. Import-Csv MyFile.csv | ConvertTo-Json | OutFile myfile.json

最后的outfile只是一个虚拟文件,它将管道结果存储在一个文件中。你可以对管道输出做任何你想做的事情。
这里有两件事是错误的:它引入了一些你不想要的引号。它生成JSON而不是YAML。
Powershell Gallery中有一个名为Powershell-Yaml的模块。它包含一个类似于ConvertTo-Json的工具。你可以导入这个模块,然后把它修复成你想要的样子。

相关问题