Skip to content

CI/CD Workflow Architecture

Overview

The FiberPath CI/CD system has been completely reorganized for v3 to:

  • Eliminate redundancy and improve maintainability
  • Establish clear naming conventions
  • Separate concerns (CI vs packaging vs deployment)
  • Add missing automation (PyPI publishing, coordinated releases)
  • Reduce GitHub Actions minutes usage

Structure

Composite Actions (.github/actions/)

Reusable setup steps used across multiple workflows:

  1. setup-python/

  2. Sets up Python 3.11 with uv package manager

  3. Creates virtual environment
  4. Installs dependencies with caching
  5. Configurable dependency groups

  6. Used by: backend-ci, backend-publish, docs-ci, docs-deploy

  7. setup-node/

  8. Sets up Node.js 20

  9. Configures npm cache
  10. Installs GUI dependencies

  11. Used by: gui-ci, gui-packaging

  12. setup-rust/

  13. Sets up Rust toolchain
  14. Configures cargo cache
  15. Optionally installs Linux Tauri dependencies
  16. Used by: gui-ci, gui-packaging

Workflows (.github/workflows/)

CI Workflows (run on every push/PR)

backend-ci.yml - Python Backend CI

  • Triggers: Push to main, PRs affecting Python code
  • Jobs:
  • Lint & Type Check (Ruff, MyPy)
  • Test on ubuntu/macos/windows (pytest)
  • Status: Backend CI

gui-ci.yml - GUI CI

  • Triggers: Push to main, PRs affecting GUI code
  • Jobs:
  • Lint & Type Check (ESLint, TypeScript)
  • Test & Build (Vitest, coverage, Vite build)
  • Status: GUI CI

docs-ci.yml - Documentation CI

  • Triggers: Push to main, PRs affecting docs
  • Jobs:
  • Validate MkDocs build (--strict)
  • Status: Docs CI

Deployment Workflows

docs-deploy.yml - Documentation Deployment

  • Triggers: Push to main affecting docs/, manual dispatch
  • Jobs:
  • Build MkDocs site
  • Deploy to GitHub Pages
  • Permissions: contents:read, pages:write, id-token:write
  • Status: Docs Deploy

Packaging Workflows

gui-packaging.yml - GUI Installer Creation

  • Triggers: Push to main affecting GUI, manual dispatch, releases
  • Jobs:
  • Build installers for Windows/macOS/Linux (Tauri)
  • Upload artifacts (30 day retention)
  • Upload release assets (if triggered by release)
  • Timeout: 45 minutes per OS
  • Status: GUI Packaging

Publishing Workflows

backend-publish.yml - PyPI Publishing

  • Triggers: GitHub releases, manual dispatch
  • Jobs:
  • Verify version matches tag
  • Build distribution packages
  • Publish to PyPI (trusted publishing)
  • Environment: pypi (with deployment protection)
  • Permissions: contents:read, id-token:write
  • Status: Backend Publish

Release Orchestration

release.yml - Coordinated Release Management

  • Triggers: Manual dispatch with version input
  • Jobs:
  • Validate version format and tag availability
  • Create GitHub release with auto-generated notes
  • Trigger backend-publish (PyPI)
  • Trigger gui-packaging (installers)
  • Inputs:
  • version: Semantic version (e.g., 0.3.14)
  • prerelease: Boolean flag
  • Permissions: contents:write, id-token:write
  • Status: Release

Workflow Naming Convention

Format: {component}-{purpose}.yml

  • Component: backend | gui | docs | release
  • Purpose: ci | packaging | publish | deploy

Trigger Strategy

Workflow Push (main) Pull Request Manual Release
backend-ci ✅ ✅ ❌ ❌
gui-ci ✅ ✅ ❌ ❌
docs-ci ✅ ✅ ❌ ❌
docs-deploy ✅ ❌ ✅ ❌
gui-packaging ✅ ❌ ✅ ✅
backend-publish ❌ ❌ ✅ ✅
release ❌ ❌ ✅ ❌

Path Filters

Workflows only run when relevant files change:

  • backend-ci: fiberpath/**, fiberpath_api/**, fiberpath_cli/**, tests/**, pyproject.toml
  • gui-ci: fiberpath_gui/**, workflow files, composite actions
  • docs-ci: docs/**, mkdocs.yml, CONTRIBUTING.md, README.md
  • docs-deploy: Same as docs-ci
  • gui-packaging: fiberpath_gui/**, workflow files, composite actions

Release Process

Manual Release Steps

  1. Prepare Release

  2. Update version in pyproject.toml

  3. Update version in fiberpath_gui/src-tauri/Cargo.toml
  4. Update CHANGELOG.md with release notes

  5. Commit changes: git commit -m "Prepare release 0.3.14"

  6. Trigger Release Workflow

  7. Go to Actions → Release → Run workflow

  8. Enter version: 0.3.14
  9. Set prerelease: false

  10. Click "Run workflow"

  11. Automated Steps

  12. Validates version format

  13. Checks if tag exists
  14. Verifies version in pyproject.toml
  15. Creates Git tag (e.g., v0.3.14)
  16. Creates GitHub release with auto-generated notes
  17. Triggers backend-publish (PyPI)

  18. Triggers gui-packaging (installers)

  19. Post-Release

  20. Verify PyPI upload: pypi.org/project/fiberpath
  21. Download GUI installers from GitHub release
  22. Test installers on Windows/macOS/Linux
  23. Announce release

Archived Workflows

Previous workflows moved to .github/workflows/archive/:

  • ci.yml - Old monolithic Python CI (combined lint + test)
  • gui.yml - Old GUI checks + packaging (redundant with gui-tests.yml)
  • gui-tests.yml - Old GUI testing (redundant with gui.yml)
  • docs-site.yml - Old docs deployment (replaced by docs-ci + docs-deploy)

Improvements Over Previous System

Issue Previous New Solution
Redundancy gui.yml and gui-tests.yml both ran linting/building Single gui-ci.yml with all checks
Naming Inconsistent (ci.yml, gui.yml, docs-site.yml) Consistent {component}-{purpose}.yml
Setup duplication Python/Node/Rust setup repeated in every workflow Composite actions (DRY principle)
PyPI publishing Manual process Automated with trusted publishing
Release coordination Manual trigger of each workflow Single release.yml orchestrates all
Documentation No workflow docs This document + badges in README

Caching Strategy

Python (setup-python)

  • Virtual environment: .venv/ cached by OS + Python version + pyproject.toml hash
  • uv cache: Built-in caching from astral-sh/setup-uv@v3

Node.js (setup-node)

  • npm modules: ~/.npm cached by actions/setup-node@v4 with package-lock.json hash

Rust (setup-rust)

  • Cargo artifacts: ~/.cargo/ + target/ cached by OS + Cargo.lock hash
  • Binaries, registry, git index

Future Enhancements

  • [ ] Add workflow status badges to README.md
  • [ ] Document in CONTRIBUTING.md
  • [ ] Add Codecov integration for coverage tracking
  • [ ] Add dependabot for workflow action updates
  • [ ] Add workflow matrix testing for multiple Python versions
  • [ ] Add performance benchmarking workflow
  • [ ] Add security scanning (Snyk, SAST)
  • [ ] Add automatic changelog generation
  • [ ] Add automatic version bumping

Troubleshooting

Workflow Not Triggering

  1. Check path filters - workflow only runs if affected files changed
  2. Verify branch name matches trigger configuration
  3. Check workflow file syntax (YAML validation)
  4. Look for workflow run history in Actions tab

Composite Action Errors

  1. Verify action.yml exists in correct location
  2. Check inputs are passed correctly from workflow
  3. Ensure shell type is specified (bash/pwsh)
  4. Review action logs for specific error messages

Failed Builds

  1. Check if dependencies changed (update lock files)
  2. Verify environment variables are set correctly
  3. Look for OS-specific issues (Linux deps for Tauri)
  4. Review cache invalidation (might need manual clear)

PyPI Publishing Failures

  1. Verify trusted publishing is configured in PyPI project settings
  2. Check version matches tag (validation step)
  3. Ensure workflow has id-token:write permission
  4. Verify environment "pypi" exists with protection rules

References