Skip to content

Implementation of python's caching#266

Merged
MaksimZhukov merged 13 commits into
actions:mainfrom
dmitry-shibanov:v-dmshib/add-cache-support
Nov 17, 2021
Merged

Implementation of python's caching#266
MaksimZhukov merged 13 commits into
actions:mainfrom
dmitry-shibanov:v-dmshib/add-cache-support

Conversation

@dmitry-shibanov

Copy link
Copy Markdown
Contributor

In scope of this pull request we add possibility to cache python dependencies. We added cache input parameter to the action. For now, input will accept the following values:

  • pip - enable caching for pip dependencies
  • pipenv - enable caching for pipenv dependencies

ADR: #247
Related issue: #226

Description

Action will try to search requirements.txt for pip or Pipfile.lock for pipenv files in the repository and throw error if no one is found. The hash of found files will be used as part of cache key (the same approach like actions/cache recommends). The following key cache will be used:

  • For pip: setup-python-${{ runner.os }}-pip-${{ hashFiles('') }}
  • For pipenv: setup-python-${{ runner.os }}-python-${{resolved-python-version}}-pipenv-${{ hashFiles('') }}

Action will cache:

  • Pip (global cache directory retrieved via pip cache dir)
  • Pipenv (virtualenv directories $HOME/.virtualenvs for Windows and $HOME/.local/share/virtualenvs for Linux, macOS)

Besides we add cache-dependency-path input. This input will accept an array or regex of dependency files. The field will accept a path (relative to the repository root) to dependency files. If the provided path contains wildcards, the action will search all matching files and calculate a common hash like the ${{ hashFiles('**/requirements-dev.txt') }} YAML construction does.

Example of yml

  • For pip
 uses: actions/setup-python@v2
  with:
    python-version: '3.9'
    cache: 'pip'
  • For pipenv:
 uses: actions/setup-python@v2
  with:
    python-version: '3.9'
    cache: 'pipenv'

Note:

  • It's not breaking change because it requires additional field and nothing changes by default.
  • License tests are falling. But we have changes to fix it. We've decided to stay with implementation, because approximately 60 licenses were changed. When changes are reviewed, we will add licenses to the existing pull request.

What's done:

  • - Implementation of python's caching
  • - Adding tests for python's caching
  • - Adding documentation
  • - Update licenses

@yaananth yaananth left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just minor clean up comments, didn't look closely at core logics yet. I will defer it to someone else or I can come back and take a peek something this week.

Thanks!

Comment thread .github/workflows/e2e-cache.yml
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread __tests__/cache-restore.test.ts Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread action.yml Outdated
Comment thread .github/workflows/e2e-cache.yml Outdated
Comment thread .github/workflows/e2e-cache.yml Outdated
Comment thread .github/workflows/e2e-cache.yml Outdated
Comment thread src/setup-python.ts Outdated
@merwok

merwok commented Nov 12, 2021

Copy link
Copy Markdown

Tox is another wildly popular tool that manages its own virtual envs and installs dependencies.
Could there be an option to cache .tox directory too? (not exclusive with pip caching but combinable)

Comment thread src/setup-python.ts Outdated
@konradpabjan

Copy link
Copy Markdown
Contributor

Tox is another wildly popular tool that manages its own virtual envs and installs dependencies.
Could there be an option to cache .tox directory too? (not exclusive with pip caching but combinable)

@merwok In a future PR this is something that can definitely be added. I'd be happy to review any contributor PR in the future that adds this support. For the scope of this PR though we're just focusing first on pip and pipenv

@konradpabjan

Copy link
Copy Markdown
Contributor

@dmitry-shibanov This is looking great! Lets fix the failing Licensed check and there is that cacheDepencies mispelling and IMO we're good to go 🚀

@yaananth yaananth left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some questions/suggestions to get more clarity, but 👍

Comment thread src/cache-distributions/pipenv-cache.ts
Comment thread src/setup-python.ts
@MaksimZhukov MaksimZhukov merged commit 280924f into actions:main Nov 17, 2021
@hugovk

hugovk commented Nov 17, 2021

Copy link
Copy Markdown
Contributor

This is great, thank you! And for the new release: https://github.com/actions/setup-python/releases/tag/v2.3.0

Please could you also update the v2 tag to point to v2.3.0?

@dmitry-shibanov

Copy link
Copy Markdown
Contributor Author

Hello @hugovk. Thank you. I think we'll update v2 tag tomorrow. We want to proceed with some extra validation.

@dmitry-shibanov

Copy link
Copy Markdown
Contributor Author

Hello @hugovk. We've updated the v2 tag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants