discuss.circleci.com Open in urlscan Pro
2602:fd3f:3:ff02::4b Public Scan

Back to summary

URL:
https://discuss.circleci.com/t/skip-job-if-cache-exists/28726
Submission: On November 20 via api (November 20th 2023, 2:52:20 am UTC) from FI — Scanned from FI

Form analysis
1 forms found in the DOM

POST /login

<form id="hidden-login-form" method="post" action="/login" style="display: none;">
  <input name="username" type="text" id="signin_username">
  <input name="password" type="password" id="signin_password">
  <input name="redirect" type="hidden">
  <input type="submit" id="signin-button" value="Kirjaudu">
</form>

Text Content

CircleCI Discuss

SKIP JOB IF CACHE EXISTS

Build Environment
cache
jtmoon79 Maaliskuu 2, 2019, 1.23am 1

I want to skip a job if a particular cache already exists.

In my case, the first job that runs builds the cache. The cache rarely changes.
So each run of the workflow runs the job “cache_rebuild”. The job
“cache_rebuild” tries to save that cache. Most often the cache already exists
(Circle CI reports “Skipping cache generation, cache already exists”). The job
“cache_rebuild” isn’t necessary most of the time.

How do I skip the job “cache_rebuild” if a specific cache already exists?

This is similar to this post regarding skipping a build.

levlaz Maaliskuu 2, 2019, 7.55pm 2

There is no way to skip a build conditionally like this, but what you could do
is wrap the steps up in a script that first checks if the cache has changed, and
if it has not return an exit code of 0 rather than continuing to regenerate the
cache.

jtmoon79 Maaliskuu 11, 2019, 6.28am 3

@levlaz how would I check if the cache has changed from within a script?

drazisil Maaliskuu 11, 2019, 1.06pm 4

That depends on your cache key. If you are are using a checksum you would check
that to see if it has changed from what it was when you restored it.

Can you share your config.yml file?

1 tykkäys

jtmoon79 Maaliskuu 11, 2019, 7.06pm 5

> Can you share your config.yml file?

github.com

JTMOON79/COVERLOVIN2/BLOB/MASTER/.CIRCLECI/CONFIG.YML

# Python CircleCI 2.1 configuration file
#
# environment variable CACHE_VERSION value is set at
# https://circleci.com/gh/jtmoon79/coverlovin2/edit#env-vars
#
---
version: 2.1
jobs:
prep_workspace_cache:
docker:
# see https://circleci.com/docs/2.0/circleci-images/
- image: circleci/python:3.7.1
steps:
- checkout
- run:
name: prepare a Circle CI workspace, prepare a Circle CI cache, save environment variables for pipenv in the workspace
command: |
set -x

echo -e "\n\n### record information about the shell context\n\n"

This file has been truncated. show original

thomasbertet Elokuu 30, 2019, 6.10pm 6

Hi there, I would like to do the same.
Did you manage to do something @jtmoon79 ?

jtmoon79 Elokuu 30, 2019, 7.38pm 7

thomasbertet:

> Did you manage to do something @jtmoon79 ?

Nope.

halfer Syyskuu 1, 2019, 9.50am 8

jtmoon79:

> @levlaz how would I check if the cache has changed from within a script?

It looks like you are using cache-{{ .Environment.CACHE_VERSION }}-{{ .Branch
}}-{{ arch }}-{{ checksum "Pipfile.lock" }}. I assume that if you do a cache
restore and the files do not appear, does that not mean that the hash has
changed? In other words, the key lookup failed and your dependency fetch has to
do an operation from scratch?

tzookb Joulukuu 31, 2019, 7.17am 9

I had to skip several jobs in cache cache exists.
So I did this trick:

lint_code:
executor: my-executor
steps:
- skip_if_cache_exists:
skiptype: "linting"
- run: do job stuff
- save_cache_flag:
skiptype: "linting"

and the commands that Im using above: skip_if_cache_exists, save_cache_flag

commands:
skip_if_cache_exists:
description: |
a command to exit the job for selected branch
parameters:
skiptype:
description: type of job to skip
type: string
steps:
- restore_cache:
key: skipcheck-<<parameters.skiptype>>-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}
- run:
name: if cache exists exit
command: |
FILE=~/cachedflags/job.<<parameters.skiptype>>.flag
if test -f "$FILE"; then
echo "$FILE exist"
circleci step halt
else
echo "$FILE doesnt exist"
fi
save_cache_flag:
description: |
a command that will create the cache
parameters:
skiptype:
description: type of job to skip
type: string
steps:
- run:
name: create job flag file
command: mkdir -p ~/cachedflags/ && touch ~/cachedflags/job.<<parameters.skiptype>>.flag
- save_cache:
key: skipcheck-<<parameters.skiptype>>-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}
paths:
- ~/cachedflags/job.<<parameters.skiptype>>.flag

1 tykkäys

CJBridges Kesäkuu 29, 2020, 2.49pm 10

Hi @tzookb - thanks for sharing - I found your commands in the last few days
when trying to check if a cache already existed. Here’s my adjustment to allow
for:

1. Category (e.g. src, bundle, node_modules)
2. Original cache key used as part of the “marker” cache key, rather than a new
key. Explicit matching.
3. Output for each command now includes the category and more descriptive text
so that you can separate the original cache from the marker cache quickly on
the UI. This especially helps as not everyone I work with will need to know
about the underlying mechanism.
4. Renamed cache commands

I couldn’t get code formatting to play nice today in the forum, so here it is in
a gist:

gist.github.com

HTTPS://GIST.GITHUB.COM/CJBRIDGES/86A5AB831F2763A4ED699061BF902DAC

CIRCLECI_COMMAND_FOR_CACHE_OPTIMIZATION.YML

commands:
halt_if_cache_exists:
description: Halt a build if a cache already exists, without downloading the entire cache. Match only exact key. Pair with mark_cache_existence.
parameters:
category:
description: friendly name of the sort of cache (e.g. bundle, src)
type: string
key:
description: hash key where underlying data would be stored

This file has been truncated. show original

* Aloitus
* Alueet
* UKK/Ohjeet
* Käyttöehdot
* Tietosuojaseloste

Palvelun tarjoaa Discourse, toimii parhaiten, kun JavaScript on käytössä

Siirry pääsisältöön

* circleci.com
* Documentation
* Feature Requests
* Support

RekisteröidyKirjaudu
*

* All Topics
* Unanswered Questions
* Suggested Topics

* Events

* Lisää

Alueet
* Announcements
* API
* Build Environment
* Community
* Deployments
* Ecosystem
* Feedback & Bug Reports
* Integrations and Accounts
* Helpful Resources
* Kaikki alueet

Tunnisteet
* circle-yml
* docker
* nodejs
* ios
* 2.0
* Kaikki tunnisteet

SKIP JOB IF CACHE EXISTS

Build Environment
cache

Olet valinnut 0 viestiä.

valitse kaikki

peruuta valinta

maalis 2019
1 / 10
maalis 2019

kesä 2020

jtmoon79
1
maalis 19

I want to skip a job if a particular cache already exists.

How do I skip the job “cache_rebuild” if a specific cache already exists?

This is similar to this post regarding skipping a build.

* LUOTU

maalis 19

* VIIMEISIN VASTAUS

kesä 20
* 9

VASTAUSTA

* 9,6 k

KATSELUA

* 7

KÄYTTÄJÄÄ

* 2

TYKKÄYSTÄ

* 2

LINKKIÄ

* 4

levlazUser
maalis 19

8 päivää myöhemmin
jtmoon79
maalis 19

@levlaz how would I check if the cache has changed from within a script?

2 vastausta

drazisilRegular
maalis 19

That depends on your cache key. If you are are using a checksum you would check
that to see if it has changed from what it was when you restored it.

Can you share your config.yml file?

jtmoon79
maalis 19

> Can you share your config.yml file?

github.com

JTMOON79/COVERLOVIN2/BLOB/MASTER/.CIRCLECI/CONFIG.YML 168

echo -e "\n\n### record information about the shell context\n\n"

This file has been truncated. show original

6 kuukautta myöhemmin
thomasbertet
elo 19

Hi there, I would like to do the same.
Did you manage to do something @jtmoon79 ?

jtmoon79
elo 19

thomasbertet:

> Did you manage to do something @jtmoon79 ?

Nope.

halfer
syys 19

jtmoon79:

> @levlaz how would I check if the cache has changed from within a script?

4 kuukautta myöhemmin
tzookb
joulu 19

I had to skip several jobs in cache cache exists.
So I did this trick:

lint_code:
executor: my-executor
steps:
- skip_if_cache_exists:
skiptype: "linting"
- run: do job stuff
- save_cache_flag:
skiptype: "linting"

and the commands that Im using above: skip_if_cache_exists, save_cache_flag

6 kuukautta myöhemmin
CJBridges
kesä 20

Hi @tzookb - thanks for sharing - I found your commands in the last few days
when trying to check if a cache already existed. Here’s my adjustment to allow
for:

I couldn’t get code formatting to play nice today in the forum, so here it is in
a gist:

gist.github.com

HTTPS://GIST.GITHUB.COM/CJBRIDGES/86A5AB831F2763A4ED699061BF902DAC 416

CIRCLECI_COMMAND_FOR_CACHE_OPTIMIZATION.YML

This file has been truncated. show original

Vastaa

discuss.circleci.com Open in urlscan Pro 2602:fd3f:3:ff02::4b Public Scan

Form analysis 1 forms found in the DOM

POST /login

Text Content

discuss.circleci.com Open in urlscan Pro
2602:fd3f:3:ff02::4b Public Scan

Form analysis
1 forms found in the DOM