Cache reports 'Up to date' while it is not

Ilya's Avatar

Ilya

05 Jun, 2024 11:26 PM

See the following build:
  https://ci.appveyor.com/project/blp/ovs/build/job/mros01o229s56l8y

We have a build cache configured for a folder:
cache:
- C:\ovs-build-downloads -> appveyor.yml

The file in the folder gets updated during the build, but at the end of a build we have:
Updating build cache...
Cache 'C:\ovs-build-downloads' - Up to date

So, the results of the cache update are lost. The next build again detects that the cache needs an update and performs an expensive update again.

  1. Support Staff 1 Posted by Owen McDonnell on 06 Jun, 2024 02:27 PM

    Owen McDonnell's Avatar

    It looks like the hashes were updated yesterday and based on hashed not matching you are performing the download.
    Any ideas why the hashes are now different?

  2. 2 Posted by Ilya Maximets on 06 Jun, 2024 08:49 PM

    Ilya Maximets's Avatar

    (looks my comment can't pass through moderation, so replying via email)

    What we do is that we get the list of available OpenSSL versions from
    https://raw.githubusercontent.com/slproweb . We find the latest one.
    We compare the hash of the latest one with the hash of installer we have
    in a cached C:\ovs-build-downloads. If they do not match, we remove the
    old installer and download a new one. We expect the new installer to be
    stored back to the build cache.

    Yesterday the new OpenSSL 3.0.14 became available, so we compared the hash
    and it didn't match our currently cached 3.0.13. So the cached installer
    was removed and a new installer for 3.0.14 was downloaded. However, for
    some reason, the caching mechanism didn't recognize that file was changed
    in C:\ovs-build-downloads and didn't upload the new 3.0.14 installer into
    build cache. So, every build re-downloads it and cache doesn't actually
    work.

    On 6/6/24 16:27, Owen McDonnell wrote:

  3. Support Staff 3 Posted by Owen McDonnell on 06 Jun, 2024 09:52 PM

    Owen McDonnell's Avatar

    Sorry, I think i misunderstood the problem at first.
    As long as you have declared a dependency that is the dependency that is used to determine when to update the cache.
    Remove the -> appveyor.yml from the cache section of your config file and it should update as expected.

  4. 4 Posted by ilya.maximets on 07 Jun, 2024 10:34 AM

    ilya.maximets's Avatar

    (sending this for the third time, sorry if you received all the copies, but neither email
    nor web interface seem to work consistently)

    > Remove the -> appveyor.yml from the cache section of your config file and it should
    > update as expected.

    Hmm. This sounds strange. I though that -> dependency only
    affects the restoration of the cache. As the docs say:

      cache:
      - C:\ProgramData\chocolatey\lib -> appveyor.yml

      This will mean invalidate C:\ProgramData\chocolatey\lib whenever
      appveyor.yml is changed. Dependency check is performed on build
      start and cache item won’t be restored if dependency changed.
      At the end of successful build cache item will be updated.

    This doesn't say that item will not be updated if appveyor.yml
    did not change.

    We want the cache to be invalidated whenever appveyor.yml changes
    and we want the cache to be updated whenever content of the
    C:\ovs-build-downloads changes as well. How can we express that?

  5. Support Staff 5 Posted by Owen McDonnell on 10 Jun, 2024 05:11 PM

    Owen McDonnell's Avatar

    The cache dependency was created in order to skip time consuming CRC32 calculation.
    If you want it to be done at both the start and finish you might try something like this:

    cache:
    - C:\ovs-build-downloads -> appveyor.yml
    - C:\ovs-build-downloads
    
    The order is important here.

    But I feel compelled to ask, given that simply putting the folder as the cache with no dependency will update exactly as you expect, what would be the situation in which you want to invalidate your cache based on a change in the appveyor.yml?

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac