Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions .coveragerc

This file was deleted.

5 changes: 0 additions & 5 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,10 @@ instructions.
**Merge checklist:**
- [ ] All reviewers approved
- [ ] CI build is green
- [ ] Version bumped
- [ ] Changelog record added
- [ ] Commits are squashed
- [ ] PR author is listed in AUTHORS

**Post merge:**
- [ ] Create a tag
- [ ] Check new version is pushed to PyPi after tag-triggered build is
finished.
- [ ] Delete working branch (if not needed anymore)

**Author concerns:** List any concerns about this PR - inelegant
Expand Down
17 changes: 8 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
name: Python CI

on:
push:
branches: [master]
pull_request:
branches:
- '**'
workflow_call:

jobs:
run_tests:
name: tests
runs-on: ${{ matrix.os }}
permissions:
contents: read
strategy:
matrix:
os: [ubuntu-latest]
Expand All @@ -19,21 +20,19 @@ jobs:

steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: setup python
uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
- name: Install uv
uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
with:
enable-cache: true
python-version: ${{ matrix.python-version }}

- name: Install pip
run: pip install -r requirements/pip.txt

- name: Install Dependencies
run: pip install -r requirements/ci.txt
run: uv sync --locked --group ci

- name: Run Tests
env:
TOXENV: ${{ matrix.toxenv }}
run: tox
run: uv run --locked tox

- name: Run coverage
if: matrix.python-version == '3.12' && matrix.toxenv == 'django42'
Expand Down
31 changes: 0 additions & 31 deletions .github/workflows/publish_pypi.yml

This file was deleted.

117 changes: 117 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
name: Semantic Release

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.github/PULL_REQUEST_TEMPLATE.md still has "Version bumped" and "Changelog record added" in the merge checklist, and "Create a tag" and the PyPI check after merge. Those should come out of the template since PSR handles the version, tag and release now.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 01c577c.


on:
push:
branches: [master]

jobs:
# The same aggregate a pull request has to pass, so the release runs exactly
# the checks that were required to merge -- and picks up any package added to
# ci.yml later without needing a change here.
run_ci:
uses: ./.github/workflows/ci.yml

release:
# Every package this workflow publishes has to build before we tag
# anything. The GitHub release and the PyPI uploads cannot be taken back,
# so a package that only fails to build in its publish job would leave the
# release half-finished.
needs: [run_ci]
runs-on: ubuntu-latest
if: github.ref_name == 'master'
concurrency:
group: ${{ github.workflow }}-release-${{ github.ref_name }}
cancel-in-progress: false

permissions:
contents: write

steps:
# Note: We checkout the repository at the branch that triggered the workflow.
# Python Semantic Release will automatically convert shallow clones to full clones
# if needed to ensure proper history evaluation. However, we forcefully reset the
# branch to the workflow sha because it is possible that the branch was updated
# while the workflow was running, which prevents accidentally releasing un-evaluated
# changes.
- name: Setup | Checkout Repository on Release Branch
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.ref_name }}

- name: Setup | Force release branch to be at workflow sha
run: |
git reset --hard ${{ github.sha }}

- name: Action | Semantic Version Release
id: release
# Adjust tag with desired version if applicable.
uses: python-semantic-release/python-semantic-release@b700dbeb1f431a0fcc30a79f3f0869407fe07278 # v10.7.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
git_committer_name: "github-actions"
git_committer_email: "[email protected]"
changelog: "false"
# Commit, tag, push and build, but don't create the GitHub release.
# We create it ourselves in the next step so that the distributions
# are attached before the release is published. See that step for why.
vcs_release: "false"

# This repo has immutable releases enabled, which freezes a release's
# assets the moment it is published, so assets cannot be attached
# afterwards. Everything we want on the release has to be built by now
# and passed to this one command, which creates the release as a draft,
# uploads the assets, and only then publishes it:
# https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/immutable-releases
- name: Publish | Create GitHub Release with Assets
if: steps.release.outputs.released == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Reuse the release notes python-semantic-release generated for us.
RELEASE_NOTES: ${{ steps.release.outputs.release_notes }}
TAG: ${{ steps.release.outputs.tag }}
run: |
# Output the release notes to a file
printf '%s' "$RELEASE_NOTES" > "$RUNNER_TEMP/release_notes.md"
# Creates the release as a draft, uploads the assets, and then
# publishes it -- all within this one command.
gh release create "$TAG" \
--verify-tag \
--title "$TAG" \
--notes-file "$RUNNER_TEMP/release_notes.md" \
dist/*

- name: Upload | Distribution Artifacts
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: steps.release.outputs.released == 'true'
with:
name: distribution-artifacts
path: dist
if-no-files-found: error

outputs:
released: ${{ steps.release.outputs.released || 'false' }}
version: ${{ steps.release.outputs.version }}

publish_to_pypi:
# 1. Separate out the publish step from the github release step to run each step at
# the least amount of token privilege
# 2. Also, publishing can fail, and its better to have a separate job if you need to retry
# and it won't require reversing the release.
runs-on: ubuntu-latest
needs: release
if: github.ref_name == 'master' && needs.release.outputs.released == 'true'

permissions:
contents: read
id-token: write

steps:
- name: Setup | Download Build Artifacts
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
id: artifact-download
with:
name: distribution-artifacts
path: dist

- name: Publish to PyPi
uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
3 changes: 3 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
Change Log
----------

This changelog is no longer maintained. Release notes are published only on

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Publishing a Release" section in README.rst still says to tag by hand, that Travis does the release, and not to create a GitHub Release, which contradicts this note. The README is now the whole PyPI description so that text will show up there too. Update the section to say that merges to master are released automatically by python-semantic-release based on the conventional commit types, with release notes on the GitHub Releases page.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 01c577c.

the `GitHub Releases page <https://github.com/openedx/edx-celeryutils/releases>`_.

..
All enhancements and patches to edx-celeryutils will be documented
in this file. It adheres to the structure of http://keepachangelog.com/ ,
Expand Down
4 changes: 1 addition & 3 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,4 @@ include CHANGELOG.rst
include CONTRIBUTING.rst
include LICENSE.txt
include README.rst
include requirements/base.in
recursive-include celery_utils *.html *.png *.gif *js *.css *jpg *jpeg *svg *py
include requirements/constraints.txt
recursive-include src/celery_utils *.html *.png *.gif *js *.css *jpg *jpeg *svg *py
39 changes: 5 additions & 34 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -35,47 +35,18 @@ docs: ## generate Sphinx HTML documentation, including API docs
tox -e docs
$(BROWSER) docs/_build/html/index.html

COMMON_CONSTRAINTS_TXT=requirements/common_constraints.txt
.PHONY: $(COMMON_CONSTRAINTS_TXT)
$(COMMON_CONSTRAINTS_TXT):
wget -O "$(@)" https://raw.githubusercontent.com/edx/edx-lint/master/edx_lint/files/common_constraints.txt || touch "$(@)"

upgrade: export CUSTOM_COMPILE_COMMAND=make upgrade
upgrade: $(COMMON_CONSTRAINTS_TXT)
## update the requirements/*.txt files with the latest packages satisfying requirements/*.in
pip install -qr requirements/pip-tools.txt
pip-compile --upgrade --allow-unsafe --rebuild -o requirements/pip.txt requirements/pip.in
pip-compile --rebuild --upgrade -o requirements/pip-tools.txt requirements/pip-tools.in
pip install -qr requirements/pip.txt
pip install -qr requirements/pip-tools.txt
pip-compile --rebuild --upgrade -o requirements/base.txt requirements/base.in
pip-compile --rebuild --upgrade -o requirements/dev.txt requirements/dev.in requirements/quality.in
pip-compile --rebuild --upgrade -o requirements/doc.txt requirements/base.in requirements/doc.in
pip-compile --rebuild --upgrade -o requirements/quality.txt requirements/quality.in
pip-compile --rebuild --upgrade -o requirements/test.txt requirements/base.in requirements/test.in
pip-compile --rebuild --upgrade -o requirements/ci.txt requirements/ci.in
# Let tox control the Django version for tests
grep -e "^amqp==\|^anyjson==\|^billiard==\|^celery==\|^kombu==\|^click-didyoumean==\|^click-repl==\|^click==\|^prompt-toolkit==\|^vine==" requirements/base.txt > requirements/celery50.txt
sed -i.tmp '/^[d|D]jango==/d' requirements/test.txt
sed -i.tmp '/^djangorestframework==/d' requirements/test.txt
sed -i.tmp '/^amqp==/d' requirements/test.txt
sed -i.tmp '/^anyjson==/d' requirements/test.txt
sed -i.tmp '/^billiard==/d' requirements/test.txt
sed -i.tmp '/^celery==/d' requirements/test.txt
sed -i.tmp '/^kombu==/d' requirements/test.txt
sed -i.tmp '/^vine==/d' requirements/test.txt
rm requirements/test.txt.tmp
upgrade: ## update the uv lockfile with the latest packages satisfying pyproject.toml
uv run --with edx-lint edx_lint write_uv_constraints pyproject.toml
uv lock --upgrade

quality: ## check coding style with pycodestyle and pylint
tox -e quality

requirements: ## install development environment requirements
pip install -qr requirements/pip.txt
pip install -qr requirements/dev.txt --exists-action w
pip-sync requirements/base.txt requirements/dev.txt requirements/private.* requirements/test.txt
uv sync --group dev

test: clean ## run tests in the current virtualenv
py.test tests/ celery_utils/
py.test tests/ src/celery_utils/

diff_cover: test
diff-cover coverage.xml
Expand Down
8 changes: 2 additions & 6 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,8 @@ Documentation
Publishing a Release
--------------------

After a PR merges, a new version of the package will automatically be released by Travis when the commit is tagged. Use::

git tag -a X.Y.Z -m "Releasing version X.Y.Z"
git push origin X.Y.Z

Do **not** create a Github Release, or ensure its message points to the CHANGELOG.rst.
Merges to master are released automatically by python-semantic-release, which picks the version from the
conventional commit types. Release notes are on the `GitHub Releases <https://github.com/openedx/edx-celeryutils/releases>`_ page.

License
-------
Expand Down
5 changes: 0 additions & 5 deletions celery_utils/__init__.py

This file was deleted.

10 changes: 6 additions & 4 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,12 @@
import os
from subprocess import check_call
from datetime import datetime
from importlib.metadata import version as get_version
import sys

import django

VERSION = get_version('edx-celeryutils')

# Configure Django for autodoc usage
django.setup()
Expand Down Expand Up @@ -81,9 +83,9 @@
# built documents.
#
# The short X.Y version.
version = '0.1.0'
version = VERSION
# The full version, including alpha/beta/rc tags.
release = '0.1.0'
release = VERSION

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand Down Expand Up @@ -499,8 +501,8 @@ def on_init(app): # pylint: disable=unused-argument
# If we are, assemble the path manually
bin_path = os.path.abspath(os.path.join(sys.prefix, 'bin'))
apidoc_path = os.path.join(bin_path, apidoc_path)
check_call([apidoc_path, '-o', docs_path, os.path.join(root_path, 'celery_utils'),
os.path.join(root_path, 'celery_utils/migrations')])
check_call([apidoc_path, '-o', docs_path, os.path.join(root_path, 'src', 'celery_utils'),
os.path.join(root_path, 'src', 'celery_utils', 'migrations')])


def setup(app):
Expand Down
7 changes: 4 additions & 3 deletions docs/getting_started.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
Getting Started
===============

If you have not already done so, create/activate a `virtualenv`_. Unless otherwise stated, assume all terminal code
below is executed within the virtualenv.
`uv`_ manages the development environment. ``make requirements`` creates a ``.venv`` directory in the
repository and installs the dependencies there; activate it with ``source .venv/bin/activate``. Unless
otherwise stated, assume all terminal code below is executed within that environment.

.. _virtualenv: https://virtualenvwrapper.readthedocs.org/en/latest/
.. _uv: https://docs.astral.sh/uv/getting-started/installation/


Install dependencies
Expand Down
4 changes: 2 additions & 2 deletions docs/testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ Testing
=======

edx-celeryutils has an assortment of test cases and code quality
checks to catch potential problems during development. To run them all in the
version of Python you chose for your virtualenv:
checks to catch potential problems during development. To run them all in the ``.venv``
environment:

.. code-block:: bash

Expand Down
Loading
Loading