windows 在多个缓存路径中使用GitHub缓存操作？

nuypyhwy 于 2023-03-31 发布在 Windows

关注(0)|答案(3)|浏览(158)

我试图使用官方的GitHub缓存操作（https://github.com/actions/cache）来缓存一些二进制文件，以加快我的一些工作流程，但是当指定多个缓存路径时，我无法让它工作。
下面是我使用单个缓存路径设置的一个简单的工作测试：有一个操作用于写入该高速缓存，另一个操作用于阅读缓存（两者都在不同的工作流中执行，但在同一个仓库和分支上）。首先执行write-action，并创建一个文件“subdir/a.txt”，然后使用“actions/cache@v2”操作将其缓存：

# Test with single path
    - name: Create file
      shell: bash
      run: |
        mkdir subdir
        cd subdir
        printf '%s' "Lorem ipsum" >> a.txt
        
    - name: Write cache (Single path)
      uses: actions/cache@v2
      with:
        path: "D:/a/cache_test/cache_test/**/*.txt"
        key: test-cache-single-path

read-action检索该高速缓存，递归地打印目录中所有文件的列表以确认它已经从缓存中恢复了文件，然后打印缓存的txt文件的内容：

- name: Get cached file
      uses: actions/cache@v2
      id: get-cache
      with:
        path: "D:/a/cache_test/cache_test/**/*.txt"
        key: test-cache-single-path
    
    - name: Print files
      shell: bash
      run: |
        echo "Cache hit: ${{steps.get-cache.outputs.cache-hit}}"
        cd "D:/a/cache_test/cache_test"
        ls -R
        cat "D:/a/cache_test/cache_test/subdir/a.txt"

这工作没有任何问题。
现在，该高速缓存操作的描述包含一个指定多个缓存路径的示例：

- uses: actions/cache@v2
    with:
      path: | 
        path/to/dependencies
        some/other/dependencies 
      key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }}

但是当我在示例操作中尝试时，它失败了。在新的write-action中，我创建了两个文件，“subdir/a.txt”和“subdir/b.md”，然后通过指定两个路径来缓存它们：

# Test with multiple paths
    - name: Create files
      shell: bash
      run: |
        mkdir subdir
        cd subdir
        printf '%s' "Lorem ipsum" >> a.txt
        printf '%s' "dolor sit amet" >> b.md

    #- name: Write cache (Multi path)
      uses: actions/cache@v2
      with:
        path: |
          "D:/a/cache_test/cache_test/**/*.txt"
          "D:/a/cache_test/cache_test/**/*.md"
        key: test-cache-multi-path

新的read-action与旧的相同，但也指定了两个路径：

# Read cache
    - name: Get cached file
      uses: actions/cache@v2
      id: get-cache
      with:
        path: |
          "D:/a/cache_test/cache_test/**/*.txt"
          "D:/a/cache_test/cache_test/**/*.md"
        key: test-cache-multi-path
    
    - name: Print files
      shell: bash
      run: |
        echo "Cache hit: ${{steps.get-cache.outputs.cache-hit}}"
        cd "D:/a/cache_test/cache_test"
        ls -R
        cat "D:/a/cache_test/cache_test/subdir/a.txt"
        cat "D:/a/cache_test/cache_test/subdir/b.md"

这一次，我仍然得到该高速缓存已被读取的确认：

Cache restored successfully
Cache restored from key: test-cache-multi-path
Cache hit: true

但是，“ls -R”没有列出这些文件，并且“cat”命令会失败，因为这些文件不存在。
我的错误在哪里？用该高速缓存操作指定多个路径的正确方法是什么？

windows

来源：https://stackoverflow.com/questions/69821869/using-github-cache-action-with-multiple-cache-paths

3条答案

按热度按时间

5cnsuln71#

我做了一些修改就能使它工作;

使用相对路径而不是绝对路径
使用密钥内容的散列

看起来至少bash的绝对路径看起来像这样：

/d/a/so-foobar-cache/so-foobar-cache/cache_test/cache_test/subdir

其中so-foobar-cache是存储库的名称。

.github/workflows/foobar.yml

name: Store and Fetch cached files
on: [push]
jobs:
  store:
    runs-on: windows-2019
    steps:
      - name: Create files
        shell: bash
        id: store
        run: |
          mkdir -p 'cache_test/cache_test/subdir'
          cd 'cache_test/cache_test/subdir'
          echo pwd $(pwd)
          printf '%s' "Lorem ipsum" >> a.txt
          printf '%s' "dolor sit amet" >> b.md
          cat a.txt b.md
      - name: Store in cache
        uses: actions/cache@v2
        with:
          path: |
            cache_test/cache_test/**/*.txt
            cache_test/cache_test/**/*.md
          key: multiple-files-${{ hashFiles('cache_test/cache_test/**') }}
      - name: Print files (A)
        shell: bash
        run: |
          echo "Cache hit: ${{steps.store.outputs.cache-hit}}"
          find cache_test/cache_test/subdir
          cat cache_test/cache_test/subdir/a.txt
          cat cache_test/cache_test/subdir/b.md

  fetch:
    runs-on: windows-2019
    needs: store
    steps:
      - name: Restore
        uses: actions/cache@v2
        with:
          path: |
            cache_test/cache_test/**/*.txt
            cache_test/cache_test/**/*.md
          key: multiple-files-${{ hashFiles('cache_test/cache_test/**') }}
          restore-keys: |
            multiple-files-${{ hashFiles('cache_test/cache_test/**') }}
            multiple-files-
      - name: Print files (B)
        shell: bash
        run: |
          find cache_test -type f | xargs -t grep -e.

日志

$ gh run view 1446486801 

✓ master Store and Fetch cached files · 1446486801
Triggered via push about 3 minutes ago

JOBS
✓ store in 5s (ID 4171907768)
✓ fetch in 10s (ID 4171909690)

第一份工作

$ gh run view 1446486801 --log --job=4171907768 | grep -e Create  -e Store -e Print
store   Create files    2021-11-10T22:59:32.1396931Z ##[group]Run mkdir -p 'cache_test/cache_test/subdir'
store   Create files    2021-11-10T22:59:32.1398025Z mkdir -p 'cache_test/cache_test/subdir'
store   Create files    2021-11-10T22:59:32.1398695Z cd 'cache_test/cache_test/subdir'
store   Create files    2021-11-10T22:59:32.1399360Z echo pwd $(pwd)
store   Create files    2021-11-10T22:59:32.1399936Z printf '%s' "Lorem ipsum" >> a.txt
store   Create files    2021-11-10T22:59:32.1400672Z printf '%s' "dolor sit amet" >> b.md
store   Create files    2021-11-10T22:59:32.1401231Z cat a.txt b.md
store   Create files    2021-11-10T22:59:32.1623649Z shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
store   Create files    2021-11-10T22:59:32.1626211Z ##[endgroup]
store   Create files    2021-11-10T22:59:32.9569082Z pwd /d/a/so-foobar-cache/so-foobar-cache/cache_test/cache_test/subdir
store   Create files    2021-11-10T22:59:32.9607728Z Lorem ipsumdolor sit amet
store   Store in cache  2021-11-10T22:59:33.9705422Z ##[group]Run actions/cache@v2
store   Store in cache  2021-11-10T22:59:33.9706196Z with:
store   Store in cache  2021-11-10T22:59:33.9706815Z   path: cache_test/cache_test/**/*.txt
store   Store in cache  cache_test/cache_test/**/*.md
store   Store in cache  
store   Store in cache  2021-11-10T22:59:33.9708499Z   key: multiple-files-25c0e6413e23766a3681413625169cee1ca3a7cd2186cc1b1df5370fb43bce55
store   Store in cache  2021-11-10T22:59:33.9709961Z ##[endgroup]
store   Store in cache  2021-11-10T22:59:35.1757943Z Received 260 of 260 (100.0%), 0.0 MBs/sec
store   Store in cache  2021-11-10T22:59:35.1761565Z Cache Size: ~0 MB (260 B)
store   Store in cache  2021-11-10T22:59:35.1781110Z [command]C:\Windows\System32\tar.exe -z -xf D:/a/_temp/653f7664-e139-4930-9710-e56942f9fa47/cache.tgz -P -C D:/a/so-foobar-cache/so-foobar-cache
store   Store in cache  2021-11-10T22:59:35.2069751Z Cache restored successfully
store   Store in cache  2021-11-10T22:59:35.2737840Z Cache restored from key: multiple-files-25c0e6413e23766a3681413625169cee1ca3a7cd2186cc1b1df5370fb43bce55
store   Print files (A) 2021-11-10T22:59:35.3087596Z ##[group]Run echo "Cache hit: "
store   Print files (A) 2021-11-10T22:59:35.3088324Z echo "Cache hit: "
store   Print files (A) 2021-11-10T22:59:35.3088983Z find cache_test/cache_test/subdir
store   Print files (A) 2021-11-10T22:59:35.3089571Z cat cache_test/cache_test/subdir/a.txt
store   Print files (A) 2021-11-10T22:59:35.3090176Z cat cache_test/cache_test/subdir/b.md
store   Print files (A) 2021-11-10T22:59:35.3104465Z shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
store   Print files (A) 2021-11-10T22:59:35.3106449Z ##[endgroup]
store   Print files (A) 2021-11-10T22:59:35.3494703Z Cache hit: 
store   Print files (A) 2021-11-10T22:59:35.4456032Z cache_test/cache_test/subdir
store   Print files (A) 2021-11-10T22:59:35.4456852Z cache_test/cache_test/subdir/a.txt
store   Print files (A) 2021-11-10T22:59:35.4459226Z cache_test/cache_test/subdir/b.md
store   Print files (A) 2021-11-10T22:59:35.4875011Z Lorem ipsumdolor sit amet
store   Post Store in cache 2021-11-10T22:59:35.6109511Z Post job cleanup.
store   Post Store in cache 2021-11-10T22:59:35.7899690Z Cache hit occurred on the primary key multiple-files-25c0e6413e23766a3681413625169cee1ca3a7cd2186cc1b1df5370fb43bce55, not saving cache.

第二份工作

$ gh run view 1446486801 --log --job=4171909690  | grep -e Restore -e Print
fetch   Restore 2021-11-10T22:59:50.8498516Z ##[group]Run actions/cache@v2
fetch   Restore 2021-11-10T22:59:50.8499346Z with:
fetch   Restore 2021-11-10T22:59:50.8499883Z   path: cache_test/cache_test/**/*.txt
fetch   Restore cache_test/cache_test/**/*.md
fetch   Restore 
fetch   Restore 2021-11-10T22:59:50.8500449Z   key: multiple-files-
fetch   Restore 2021-11-10T22:59:50.8501079Z   restore-keys: multiple-files-
fetch   Restore multiple-files-
fetch   Restore 
fetch   Restore 2021-11-10T22:59:50.8501644Z ##[endgroup]
fetch   Restore 2021-11-10T22:59:53.1143793Z Received 257 of 257 (100.0%), 0.0 MBs/sec
fetch   Restore 2021-11-10T22:59:53.1145450Z Cache Size: ~0 MB (257 B)
fetch   Restore 2021-11-10T22:59:53.1163664Z [command]C:\Windows\System32\tar.exe -z -xf D:/a/_temp/30b0dc24-b25f-4713-b3d3-cecee7116785/cache.tgz -P -C D:/a/so-foobar-cache/so-foobar-cache
fetch   Restore 2021-11-10T22:59:53.1784328Z Cache restored successfully
fetch   Restore 2021-11-10T22:59:53.5197756Z Cache restored from key: multiple-files-
fetch   Print files (B) 2021-11-10T22:59:53.5483939Z ##[group]Run find cache_test -type f | xargs -t grep -e.
fetch   Print files (B) 2021-11-10T22:59:53.5484730Z find cache_test -type f | xargs -t grep -e.
fetch   Print files (B) 2021-11-10T22:59:53.5498140Z shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
fetch   Print files (B) 2021-11-10T22:59:53.5498674Z ##[endgroup]
fetch   Print files (B) 2021-11-10T22:59:55.8119800Z grep -e. cache_test/cache_test/subdir/a.txt cache_test/cache_test/subdir/b.md
fetch   Print files (B) 2021-11-10T22:59:56.1777887Z cache_test/cache_test/subdir/a.txt:Lorem ipsum
fetch   Print files (B) 2021-11-10T22:59:56.1784138Z cache_test/cache_test/subdir/b.md:dolor sit amet
fetch   Post Restore    2021-11-10T22:59:56.3890391Z Post job cleanup.
fetch   Post Restore    2021-11-10T22:59:56.5481739Z Cache hit occurred on the primary key multiple-files-, not saving cache.

赞(0）回复(0）举报 2023-03-31

vsdwdz232#

来到这里，看看我是否可以缓存多个二进制文件.我看到有一个单独的工作流推缓存和另一个检索.我们有一个单独的用例，我们需要安装某些依赖.在这里共享相同.

用例

您的工作流需要gcc和python3才能运行。（依赖项也可以是任何其他依赖项）
您有一个安装依赖项./install-dependencies.sh的脚本，并为脚本提供适当的env，如ENV_INSTALL_PYTHON=true或ENV_INSTALL_GCC=true

注意事项

./install-dependencies.sh负责在路径~/bin中安装依赖项，并在同一路径中生成可执行二进制文件。它还确保使用新的二进制路径更新$PATH环境变量
我们不需要将check cache和install binaries复制两次（因为我们现在有两个二进制文件），而是只需要一个即可完成。
该高速缓存密钥名称python-gcc-cache-key可以是任何名称，但必须确保它是唯一的。
第三步- name: install python, gcc负责创建名为python-gcc-cache-key的键，如果没有找到它，即使我们在这一步中没有提到这个keyname。
第一步是 checkout 包含./ www.example.com脚本的存储库install-dependencies.sh。

工作流程

name: Install dependencies
on: [push]
jobs:
  install_dependencies:
    runs-on: ubuntu-latest
    name: Install python, gcc
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
## python, gcc installation
# Check if python, gcc if present in worker cache
      - name: python, gcc cache
        id: python-gcc-cache
        uses: actions/cache@v2
        with:
          path: |
            ~/bin/python
            ~/bin/gcc
          key: python-gcc-cache-key
 #  Install python, gcc if was not found in cache       
      - name: install python, gcc
        if: steps.python-gcc-cache.outputs.cache-hit != 'true'
        working-directory: .github/workflows
        env:
          ENV_INSTALL_PYTHON: true
          ENV_INSTALL_GCC: true
        run: |
          ./install-dependencies.sh

      - name: validate python, gcc
        working-directory: .github/workflows
        run: |
          ENV_INSTALL_BINARY_DIRECTORY_LINUX="$HOME/bin"
          export PATH="$ENV_INSTALL_BINARY_DIRECTORY_LINUX:$PATH"
          python3 --version
          gcc --version

福利

这将取决于你试图安装的二进制文件。对我们来说，每次缓存命中时节省的时间几乎是50sec。

赞(0）回复(0）举报 2023-03-31

kyxcudwk3#

我的用途是 * 缓存pip和APT包 *，在Sphinx中编译文档。虽然多个路径可以开箱即用，但重要的部分是将apt缓存链接到用户空间。工作流程如下：

name: docs

on: [push, pull_request, workflow_dispatch]

jobs:
  docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
      - name: cache dependencies
        id: cache_deps
        uses: actions/cache@v3
        env:
            cache-name: cache-dependencies
        with:
          path: |
            .cache/apt/archives
            .venv
          key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('.github/workflows/*') }}
      - name: Install dependencies fresh
        if: ${{ steps.cache_deps.outputs.cache-hit != 'true' }}
        run: |
          python -m venv .venv
          source .venv/bin/activate
          pip install jupyter-book
          pip install sphinxcontrib-plantuml
          echo "Dir::Cache \"$PWD/.cache/apt\";" | sudo tee -a /etc/apt/apt.conf
          sudo mkdir -p .cache/apt/archives/partial
          sudo apt-get -o Dir::Cache::archives=archives install plantuml
      - name: Install dependencies cache
        run: |
          sudo dpkg -i .cache/apt/archives/*.deb
          sudo chown -Rv $(whoami) .cache/apt
      - name: Compile Docs
        run: |
          source .venv/bin/activate
          jupyter-book build docs
      - name: Deploy to gh-pages
        uses: peaceiris/actions-gh-pages@v3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_branch: gh-pages
          publish_dir: ./docs/_build/html

demonstrated to work here .

赞(0）回复(0）举报 2023-03-31

我来回答

windows 在多个缓存路径中使用GitHub缓存操作？

3条答案

.github/workflows/foobar.yml

用例

注意事项

工作流程

福利

相关问题

热门标签

最新问答