Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor as a python package #6

Merged
merged 55 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
eb2c207
refactor: reorganize repo as a python package
kelly-sovacool Aug 7, 2024
f1d007f
refactor: create main functions for python scripts
kelly-sovacool Aug 7, 2024
42a234c
ci: setup testing infra
kelly-sovacool Aug 7, 2024
d87063c
ci: numpydoc validation
kelly-sovacool Aug 7, 2024
ed3927f
chore: configure numpydoc validation
kelly-sovacool Aug 7, 2024
b0691aa
fix: syntax error
kelly-sovacool Aug 8, 2024
600f2ae
chore: copy python project .gitignore from GitHub
kelly-sovacool Aug 8, 2024
d7070a2
fix: syntax for defining scripts
kelly-sovacool Aug 8, 2024
8b88116
chore: bump black version
kelly-sovacool Aug 8, 2024
97fc5c2
refactor: move standalone scripts to scripts/ and modules to src/ccbr…
kelly-sovacool Aug 8, 2024
8837150
feat: add shell_run() and exec_in_context() + basic tests
kelly-sovacool Aug 8, 2024
73d3c41
test: a few basic script tests
kelly-sovacool Aug 8, 2024
2271e27
style: run black
kelly-sovacool Aug 8, 2024
0b21a41
ci: add codecov
kelly-sovacool Aug 8, 2024
599677d
ci: continue on error
kelly-sovacool Aug 9, 2024
fb3e40b
ci: continue if linting fails
kelly-sovacool Aug 9, 2024
bf0e8f5
test: fix which_vpn test
kelly-sovacool Aug 9, 2024
c53dcc0
ci: run pytest only on src/
kelly-sovacool Aug 9, 2024
3f90b07
test: fix which_vpn extension
kelly-sovacool Aug 9, 2024
fb10b74
ci: always run tests, even if prior step fails
kelly-sovacool Aug 9, 2024
4526f75
ci: specify pkg name for pytest-cov
kelly-sovacool Aug 9, 2024
543291f
ci: separate lint and build jobs
kelly-sovacool Aug 9, 2024
c4a287c
ci: separate lint and build jobs
kelly-sovacool Aug 9, 2024
d687f68
ci: fix syntax
kelly-sovacool Aug 9, 2024
c774829
ci: fix black usage
kelly-sovacool Aug 9, 2024
b03e28b
docs: add CI badges
kelly-sovacool Aug 9, 2024
1e9b7e6
chore: Merge branch 'master' into refactor-pkg
kelly-sovacool Aug 12, 2024
821e73e
ci: 🤖 black formatting
github-actions[bot] Aug 12, 2024
34684cf
docs: add gh templates from ccbr/actions
kelly-sovacool Aug 12, 2024
66c1e14
feat: add pipeline utility functions
kelly-sovacool Aug 12, 2024
9280b71
chore: move VERSION file to top-level of repo
kelly-sovacool Aug 12, 2024
9f28f07
fix: help msg for peek
kelly-sovacool Aug 12, 2024
f5ac08e
feat: add basic CLI entrypoint based on CCBR_NextflowTemplate
kelly-sovacool Aug 12, 2024
1944b0a
chore: update project description
kelly-sovacool Aug 12, 2024
61b82e5
feat: add cite subcommand
kelly-sovacool Aug 12, 2024
fd94be5
test: add tests for python CLIs in src/
kelly-sovacool Aug 12, 2024
f90e728
refactor: use importlib to get the version from pyproject.toml
kelly-sovacool Aug 12, 2024
1abdb56
test: fix shell_run import statement
kelly-sovacool Aug 12, 2024
fbbd57d
feat: dynamically show list of CLI commands available
kelly-sovacool Aug 12, 2024
4fbd4b3
docs: dynamically create readme with quarto
kelly-sovacool Aug 12, 2024
7bad592
ci: cache pip deps
kelly-sovacool Aug 12, 2024
50cc198
ci: use setup-python v5
kelly-sovacool Aug 12, 2024
f5b915a
chore: create MANIFEST.in
kelly-sovacool Aug 12, 2024
6643b62
fix: attempt to fix data finding problems via symlinks
kelly-sovacool Aug 12, 2024
384e95b
ci: checkout branch if triggered by PR
kelly-sovacool Aug 12, 2024
3ba6b88
ci: 🤖 render readme
github-actions[bot] Aug 12, 2024
ee93290
docs: ensure blank lines around readme headers
kelly-sovacool Aug 12, 2024
4c110fd
chore: Merge branch 'refactor-pkg' of https://github.com/CCBR/Tools i…
kelly-sovacool Aug 12, 2024
37ed8bf
docs: re-render readme
kelly-sovacool Aug 12, 2024
b6dc979
ci: 🤖 render readme
github-actions[bot] Aug 12, 2024
629e40c
ci: specify quarto version to fix header rendering
kelly-sovacool Aug 12, 2024
8e60a35
ci: 🤖 render readme
github-actions[bot] Aug 12, 2024
c923005
feat: add spooker
kelly-sovacool Aug 12, 2024
1d8647a
chore: Merge branch 'refactor-pkg' of https://github.com/CCBR/Tools i…
kelly-sovacool Aug 12, 2024
9364554
ci: 🤖 render readme
github-actions[bot] Aug 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
249 changes: 249 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
# Contributing to CCBR Tools

## Proposing changes with issues

If you want to make a change, it's a good idea to first
[open an issue](https://code-review.tidyverse.org/issues/)
and make sure someone from the team agrees that it’s needed.

If you've decided to work on an issue,
[assign yourself to the issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/assigning-issues-and-pull-requests-to-other-github-users#assigning-an-individual-issue-or-pull-request)
so others will know you're working on it.

## Pull request process

We use [GitHub Flow](https://docs.github.com/en/get-started/using-github/github-flow)
as our collaboration process.
Follow the steps below for detailed instructions on contributing changes to
CCBR Tools.

![GitHub Flow diagram](./img/GitHub-Flow_bg-white.png)

### Clone the repo

If you are a member of [CCBR](https://github.com/CCBR),
you can clone this repository to your computer or development environment.
Otherwise, you will first need to
[fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo)
the repo and clone your fork. You only need to do this step once.

```sh
git clone https://github.com/CCBR/tools
```

> Cloning into 'tools'... <br>
> remote: Enumerating objects: 1136, done. <br>
> remote: Counting objects: 100% (463/463), done. <br>
> remote: Compressing objects: 100% (357/357), done. <br>
> remote: Total 1136 (delta 149), reused 332 (delta 103), pack-reused 673 <br>
> Receiving objects: 100% (1136/1136), 11.01 MiB | 9.76 MiB/s, done. <br>
> Resolving deltas: 100% (530/530), done. <br>

```sh
cd tools
```

### If this is your first time cloning the repo, you may need to install dependencies

- Install the python dependencies with pip

```sh
pip install .[[dev,test]]
```

- Install [`pre-commit`](https://pre-commit.com/#install) if you don't already
have it. Then from the repo's root directory, run

```sh
pre-commit install
```

This will install the repo's pre-commit hooks.
You'll only need to do this step the first time you clone the repo.

### Create a branch

Create a Git branch for your pull request (PR). Give the branch a descriptive
name for the changes you will make, such as `iss-10` if it is for a specific
issue.

```sh
# create a new branch and switch to it
git branch iss-10
git switch iss-10
```

> Switched to a new branch 'iss-10'

### Make your changes

Edit the code, write and run tests, and update the documentation as needed.

#### test

Changes to the **python package** code will also need unit tests to demonstrate
that the changes work as intended.
We write unit tests with pytest and store them in the `tests/` subdirectory.
Run the tests with `python -m pytest`.

#### document

If you have added a new feature or changed the API of an existing feature,
you will likely need to update the documentation in `docs/`.

### Commit and push your changes

If you're not sure how often you should commit or what your commits should
consist of, we recommend following the "atomic commits" principle where each
commit contains one new feature, fix, or task.
Learn more about atomic commits here:
<https://www.freshconsulting.com/insights/blog/atomic-commits/>

First, add the files that you changed to the staging area:

```sh
git add path/to/changed/files/
```

Then make the commit.
Your commit message should follow the
[Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
specification.
Briefly, each commit should start with one of the approved types such as
`feat`, `fix`, `docs`, etc. followed by a description of the commit.
Take a look at the [Conventional Commits specification](https://www.conventionalcommits.org/en/v1.0.0/#summary)
for more detailed information about how to write commit messages.

```sh
git commit -m 'feat: create function for awesome feature'
```

pre-commit will enforce that your commit message and the code changes are
styled correctly and will attempt to make corrections if needed.

> Check for added large files..............................................Passed <br>
> Fix End of Files.........................................................Passed <br>
> Trim Trailing Whitespace.................................................Failed <br>
>
> - hook id: trailing-whitespace <br>
> - exit code: 1 <br>
> - files were modified by this hook <br> > <br>
> Fixing path/to/changed/files/file.txt <br> > <br>
> codespell................................................................Passed <br>
> style-files..........................................(no files to check)Skipped <br>
> readme-rmd-rendered..................................(no files to check)Skipped <br>
> use-tidy-description.................................(no files to check)Skipped <br>

In the example above, one of the hooks modified a file in the proposed commit,
so the pre-commit check failed. You can run `git diff` to see the changes that
pre-commit made and `git status` to see which files were modified. To proceed
with the commit, re-add the modified file(s) and re-run the commit command:

```sh
git add path/to/changed/files/file.txt
git commit -m 'feat: create function for awesome feature'
```

This time, all the hooks either passed or were skipped
(e.g. hooks that only run on R code will not run if no R files were
committed).
When the pre-commit check is successful, the usual commit success message
will appear after the pre-commit messages showing that the commit was created.

> Check for added large files..............................................Passed <br>
> Fix End of Files.........................................................Passed <br>
> Trim Trailing Whitespace.................................................Passed <br>
> codespell................................................................Passed <br>
> style-files..........................................(no files to check)Skipped <br>
> readme-rmd-rendered..................................(no files to check)Skipped <br>
> use-tidy-description.................................(no files to check)Skipped <br>
> Conventional Commit......................................................Passed <br> > [iss-10 9ff256e] feat: create function for awesome feature <br>
> 1 file changed, 22 insertions(+), 3 deletions(-) <br>

Finally, push your changes to GitHub:

```sh
git push
```

If this is the first time you are pushing this branch, you may have to
explicitly set the upstream branch:

```sh
git push --set-upstream origin iss-10
```

> Enumerating objects: 7, done. <br>
> Counting objects: 100% (7/7), done. <br>
> Delta compression using up to 10 threads <br>
> Compressing objects: 100% (4/4), done. <br>
> Writing objects: 100% (4/4), 648 bytes | 648.00 KiB/s, done. <br>
> Total 4 (delta 3), reused 0 (delta 0), pack-reused 0 <br>
> remote: Resolving deltas: 100% (3/3), completed with 3 local objects. <br>
> remote: <br>
> remote: Create a pull request for 'iss-10' on GitHub by visiting: <br>
> remote: https://github.com/CCBR/tools/pull/new/iss-10 <br>
> remote: <br>
> To https://github.com/CCBR/tools <br> > <br> > [new branch] iss-10 -> iss-10 <br>
> branch 'iss-10' set up to track 'origin/iss-10'. <br>

We recommend pushing your commits often so they will be backed up on GitHub.
You can view the files in your branch on GitHub at
`https://github.com/CCBR/tools/tree/<your-branch-name>`
(replace `<your-branch-name>` with the actual name of your branch).

### Create the PR

Once your branch is ready, create a PR on GitHub:
<https://github.com/CCBR/tools/pull/new/>

Select the branch you just pushed:

![Create a new PR from your branch](./img/new-PR.png)

Edit the PR title and description.
The title should briefly describe the change.
Follow the comments in the template to fill out the body of the PR, and
you can delete the comments (everything between `<!--` and `-->`) as you go.
Be sure to fill out the checklist, checking off items as you complete them or
striking through any irrelevant items.
When you're ready, click 'Create pull request' to open it.

![Open the PR after editing the title and description](./img/create-PR.png)

Optionally, you can mark the PR as a draft if you're not yet ready for it to
be reviewed, then change it later when you're ready.

### Wait for a maintainer to review your PR

We will do our best to follow the tidyverse code review principles:
<https://code-review.tidyverse.org/>.
The reviewer may suggest that you make changes before accepting your PR in
order to improve the code quality or style.
If that's the case, continue to make changes in your branch and push them to
GitHub, and they will appear in the PR.

Once the PR is approved, the maintainer will merge it and the issue(s) the PR
links will close automatically.
Congratulations and thank you for your contribution!

### After your PR has been merged

After your PR has been merged, update your local clone of the repo by
switching to the main branch and pulling the latest changes:

```sh
git checkout main
git pull
```

It's a good idea to run `git pull` before creating a new branch so it will
start from the most recent commits in main.

## Helpful links for more information

- [GitHub Flow](https://docs.github.com/en/get-started/using-github/github-flow)
- [semantic versioning guidelines](https://semver.org/)
- [changelog guidelines](https://keepachangelog.com/en/1.1.0/)
- [tidyverse code review principles](https://code-review.tidyverse.org)
- [reproducible examples](https://www.tidyverse.org/help/#reprex)
45 changes: 45 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Bug report
description: Report something that is broken or incorrect
labels: bug
body:
- type: markdown
attributes:
value: |
Before you post this issue, please check the documentation:

<https://ccbr.github.io/CCBR/Tools>

- type: textarea
id: description
attributes:
label: Description of the bug
description: A clear and concise description of what the bug is.
validations:
required: true

- type: textarea
id: command_used
attributes:
label: Command used and terminal output
description: Steps to reproduce the behaviour. Please paste the code you used
render: console
placeholder: |
$ ccbr_tools ...

Some output where something broke

- type: textarea
id: files
attributes:
label: Relevant files
description: |
Please drag and drop any relevant files here. Create a `.zip` archive if the extension is not allowed.

- type: textarea
id: system
attributes:
label: System information
description: |
- Version of CCBR Tools _(eg. 1.0, 1.8.2)_
- Python version _(eg. 3.11)_
- Environment _(eg. local macOS, biowulf HPC)_
4 changes: 4 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
contact_links:
- name: Discussions
url: https://github.com/CCBR/Tools/discussions
about: Please ask and answer questions here.
11 changes: 11 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: Feature request
description: Suggest an idea for the tool
labels: enhancement
body:
- type: textarea
id: description
attributes:
label: Description of feature
description: Please describe your suggestion for a new feature. It might help to describe a problem or use case, plus any alternatives that you have considered.
validations:
required: true
22 changes: 22 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## Changes

<!--
Provide a summary of what is included in this Pull Request (PR).
-->

## Issues

<!--
Reference any issues related to this PR.
If this PR fixes any issues, [use a keyword](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)
when referring to the issue.
-->

## PR Checklist

(~Strikethrough~ any points that are not applicable.)

- [ ] This comment contains a description of changes with justifications, with any relevant issues linked.
- [ ] Write unit tests for any new features, bug fixes, or other code changes.
- [ ] Update docs if there are any API changes.
- [ ] Update `CHANGELOG.md` with a short description of any user-facing changes and reference the PR number. Guidelines: https://keepachangelog.com/en/1.1.0/
58 changes: 58 additions & 0 deletions .github/workflows/auto-format.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: auto-format

on:
workflow_dispatch:
pull_request:
paths:
- "src/**"
- "README.qmd"

env:
GH_TOKEN: ${{ github.token }}

jobs:
auto-format:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.11"]

steps:
- uses: actions/checkout@v4
if: github.event_name == 'pull_request'
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.ref }}

- uses: actions/checkout@v4
if: github.event_name == 'push'
with:
fetch-depth: 0
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
- name: Install dependencies
run: |
python -m pip install .[dev] --upgrade pip
- name: format
uses: psf/black@stable
with:
options: "--verbose"
use_pyproject: true
- name: commit & push
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add .
git commit -m "ci: 🤖 black formatting" && git push || echo "nothing to commit"
- uses: quarto-dev/quarto-actions/setup@v2
with:
version: 1.4.515
- name: quarto render readme
run: |
quarto render README.qmd
git add README.md
git commit -m "ci: 🤖 render readme" && git push || echo "nothing to commit"
Loading