Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Typecheck some API interfaces #593

Merged
merged 28 commits into from
Sep 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
19e2e99
Add typechecking and action config
mfisher87 Sep 4, 2024
4cc2eb9
Typecheck granules module
mfisher87 Sep 4, 2024
041c5ba
Correct invalid type annotation
mfisher87 Aug 14, 2024
a43a25e
Flesh out CMR params & EGI subset types
mfisher87 Aug 26, 2024
3fc95f1
Define generic type annotation for Parameters class
mfisher87 Aug 26, 2024
b9c77e8
Partially annotate the Query class
mfisher87 Sep 4, 2024
da851aa
Add miscellaneous annotation
mfisher87 Aug 26, 2024
2da7a0b
Add all dependencies to typecheck environment
mfisher87 Aug 27, 2024
c8ba01a
Add notes to pyright config
mfisher87 Aug 27, 2024
27f4641
Remove pyright ignores for modules without any errors
mfisher87 Sep 4, 2024
181099f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 4, 2024
e6e7fe2
Clarify use of term "EGI" in docstring
mfisher87 Sep 4, 2024
0440fa9
Type valid data product short names
mfisher87 Sep 4, 2024
df78ca2
Fixup API parameter types
mfisher87 Sep 5, 2024
ee70196
Fixup missed name change
mfisher87 Sep 5, 2024
434c28a
Use Python 3.9-compatible annotation style
mfisher87 Sep 5, 2024
a36d17c
Correct product name type name
mfisher87 Sep 12, 2024
a8996d1
Narrow argument types
mfisher87 Sep 12, 2024
dfe4c09
Set pyright's Python version to 3.9
mfisher87 Sep 12, 2024
fddf23a
Unblock upgrading sphinx extensions by replacing sphinx-panels
mfisher87 Sep 12, 2024
e155a9b
Parse and correctly generate docs from numpy-style docstrings
mfisher87 Sep 12, 2024
51b3492
Fix typo
mfisher87 Sep 13, 2024
7f60849
Autogenerate docs for types
mfisher87 Sep 13, 2024
4cb5956
Tell the typechecker to trust us on the type of reqparams
mfisher87 Sep 13, 2024
7a423a8
Tweak style of panels in index page
mfisher87 Sep 13, 2024
6a4acd9
Rename "Specific" classes to "Required"
mfisher87 Sep 13, 2024
e41aed0
Mark EGI subset shape params NotRequired
mfisher87 Sep 13, 2024
81d60dd
Merge branch 'development' into refactor-and-typecheck-api-interfaces
JessicaS11 Sep 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/typecheck.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Typecheck

on:
pull_request:
push:
branches:
- main
- development


jobs:
test:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: actions/setup-python@v5
with:
python-version: "3.10"
mfisher87 marked this conversation as resolved.
Show resolved Hide resolved

- name: Install package and test dependencies
run: |
python -m pip install .[complete]
JessicaS11 marked this conversation as resolved.
Show resolved Hide resolved
python -m pip install -r requirements-dev.txt

- uses: jakebailey/pyright-action@v2
31 changes: 30 additions & 1 deletion doc/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,17 @@
# ones.
extensions = [
"sphinx.ext.autodoc",
# IMPORTANT: napoleon must be loaded before sphinx_autodoc_typehints
# https://github.com/tox-dev/sphinx-autodoc-typehints/issues/15
"sphinx.ext.napoleon",
"sphinx_autodoc_typehints",
"sphinx.ext.autosectionlabel",
"numpydoc",
# "sphinx.ext.autosummary",
"myst_nb",
"contributors", # custom extension, from pandas
"sphinxcontrib.bibtex",
"sphinx_panels",
"sphinx_design",
# "sphinx.ext.imgconverter", # this extension should help the latex svg warning, but results in an error instead
]
myst_enable_extensions = [
Expand Down Expand Up @@ -79,6 +83,31 @@
nb_execution_mode = "off"
suppress_warnings = ["myst.header"] # suppress non-consecutive header warning


# -- Options for Napoleon docstring parsing ----------------------------------
napoleon_google_docstring = False
napoleon_numpy_docstring = True
napoleon_use_admonition_for_examples = True
napoleon_use_admonition_for_notes = True


# -- Options for autodoc -----------------------------------------------------

# Show the typehints in the description of each object instead of the signature.
autodoc_typehints = "description"


# -- Options for autodoc typehints--------------------------------------------

# Replace Union annotations with union operator "|"
always_use_bars_union = True
# always_document_param_types = True

# Show the default value for a parameter after its type
typehints_defaults = "comma"
typehints_use_return = True


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
Expand Down
143 changes: 83 additions & 60 deletions doc/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,86 +31,109 @@ ICESat-2 datasets to enable scientific discovery.
To further enhance data discovery, we have developed the QUEST module to facilitate querying of ICESat-2 data and complimentary Argo oceanographic data, with additional dataset support expected in the future.


.. panels::
:card: + intro-card text-center
:column: col-lg-4 col-md-4 col-sm-6 col-xs-12 p-2
:img-top-cls: pl-2 pr-2 pt-2 pb-2
JessicaS11 marked this conversation as resolved.
Show resolved Hide resolved
.. grid:: 1 2 2 3
:gutter: 3
:class-container: sd-text-center

---
:img-top: https://cdn-icons-png.flaticon.com/128/2498/2498074.png
.. grid-item-card::
:img-top: https://cdn-icons-png.flaticon.com/128/2498/2498074.png
:class-img-top: sd-p-2
:class-card: sd-shadow-md

**Getting Started**
^^^^^^^^^^^^^^^^^^^
**Getting Started**
^^^^^^^^^^^^^^^^^^^

New to ICESat-2 or icepyx?
Learn how to install icepyx and use it to jumpstart your project today.
Check out our gallery of examples, too!
New to ICESat-2 or icepyx?
Learn how to install icepyx and use it to jumpstart your project today.
Check out our gallery of examples, too!

.. link-button:: install_ref
:type: ref
:text: Installation Instructions
:classes: stretched-link btn-outline-primary btn-block
.. button-ref:: install_ref
:ref-type: ref
:color: primary
:outline:
:expand:

---
:img-top: https://cdn-icons-png.flaticon.com/128/3730/3730041.png
Installation Instructions

**User Guide**
^^^^^^^^^^^^^^
.. grid-item-card::
:img-top: https://cdn-icons-png.flaticon.com/128/3730/3730041.png
:class-img-top: sd-p-2
:class-card: sd-shadow-md

The user guide provides in-depth information on the tools and functionality
available for obtaining and interacting with ICESat-2 data products.
**User Guide**
^^^^^^^^^^^^^^

.. link-button:: api_doc_ref
:type: ref
:text: Software Docs
:classes: stretched-link btn-outline-primary btn-block
The user guide provides in-depth information on the tools and functionality
available for obtaining and interacting with ICESat-2 data products.

---
:img-top: https://cdn-icons-png.flaticon.com/512/4230/4230997.png
.. button-ref:: api_doc_ref
:ref-type: ref
:color: primary
:outline:
:expand:

**Development Guide**
^^^^^^^^^^^^^^^^^^^^^
Software Docs

Have an idea or an ancillary dataset to contribute to icepyx? Go here for information on best practices
for developing and contributing to icepyx.
.. grid-item-card::
:img-top: https://cdn-icons-png.flaticon.com/512/4230/4230997.png
:class-img-top: sd-p-2
:class-card: sd-shadow-md

.. link-button:: dev_guide_label
:type: ref
:text: Development Guide
:classes: stretched-link btn-outline-primary btn-block
**Development Guide**
^^^^^^^^^^^^^^^^^^^^^

---
:img-top: https://cdn-icons-png.flaticon.com/128/1283/1283342.png
Have an idea or an ancillary dataset to contribute to icepyx? Go here for information on best practices
for developing and contributing to icepyx.

**Get in Touch**
^^^^^^^^^^^^^^^^
.. button-ref:: dev_guide_label
:ref-type: ref
:color: primary
:outline:
:expand:

icepyx is more than just software!
We're a community of data producers, managers, and users
who collaborate openly and share code and skills
for every step along the entire data pipeline. Find resources for
your questions here!
Development Guide

.. link-button:: contact_ref_label
:type: ref
:text: Get Involved!
:classes: stretched-link btn-outline-primary btn-block
.. grid-item-card::
:img-top: https://cdn-icons-png.flaticon.com/128/1283/1283342.png
:class-img-top: sd-p-2
:class-card: sd-shadow-md

---
:img-top: https://icesat-2.gsfc.nasa.gov/sites/default/files/MissionLogo_0.png
:img-top-cls: pl-2 pr-2 pt-4 pb-4
**Get in Touch**
^^^^^^^^^^^^^^^^

**ICESat-2 Resources**
^^^^^^^^^^^^^^^^^^^^^^
icepyx is more than just software!
We're a community of data producers, managers, and users
who collaborate openly and share code and skills
for every step along the entire data pipeline. Find resources for
your questions here!

Curious about other tools for working with ICESat-2 data?
Want to share your resource?
Check out the amazing work already in progress!
.. button-ref:: contact_ref_label
:ref-type: ref
:color: primary
:outline:
:expand:

.. link-button:: resource_ref_label
:type: ref
:text: ICESat-2 Resource Guide
:classes: stretched-link btn-outline-primary btn-block
Get Involved!

.. grid-item-card::
:img-top: https://icesat-2.gsfc.nasa.gov/sites/default/files/MissionLogo_0.png
:class-img-top: sd-p-2
:class-card: sd-shadow-md

**ICESat-2 Resources**
^^^^^^^^^^^^^^^^^^^^^^

Curious about other tools for working with ICESat-2 data?
Want to share your resource?
Check out the amazing work already in progress!

.. button-ref:: resource_ref_label
:ref-type: ref
:color: primary
:outline:
:expand:

ICESat-2 Resource Guide


.. toctree::
Expand Down
1 change: 1 addition & 0 deletions doc/source/user_guide/documentation/icepyx.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@ Diagrams are updated automatically after a pull request (PR) is approved and bef
read
variables
components
types
11 changes: 11 additions & 0 deletions doc/source/user_guide/documentation/types.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Types
=====

.. automodule:: icepyx.core.types
:members:
:undoc-members:
:exclude-members: CMRParamsBase,CMRParamsWithBbox,CMRParamsWithPolygon

.. COMMENT. `exclude-members` specified above is required because those models
contain symbols ('[', ']') in some keys, which sphinx doesn't like.
See: https://github.com/sphinx-doc/sphinx/issues/11039
JessicaS11 marked this conversation as resolved.
Show resolved Hide resolved
81 changes: 68 additions & 13 deletions icepyx/core/APIformatting.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
# Generate and format information for submitting to API (CMR and NSIDC)
"""Generate and format information for submitting to API (CMR and NSIDC)."""

import datetime as dt
from typing import Any, Generic, Literal, TypeVar, Union, overload

from icepyx.core.types import (
CMRParams,
EGIParamsSubset,
EGIRequiredParams,
)

# ----------------------------------------------------------------------
# parameter-specific formatting for display
Expand Down Expand Up @@ -183,12 +190,56 @@ def to_string(params):
return "&".join(param_list)


ParameterType = Literal["CMR", "required", "subset"]
# DevGoal: When Python 3.12 is minimum supported version, migrate to PEP695 style
T = TypeVar("T", bound=ParameterType)

JessicaS11 marked this conversation as resolved.
Show resolved Hide resolved

class _FmtedKeysDescriptor:
"""Enable the Parameters class' fmted_keys property to be typechecked correctly.

See: https://github.com/microsoft/pyright/issues/3071#issuecomment-1043978070
"""

@overload
def __get__(
self,
instance: 'Parameters[Literal["CMR"]]',
owner: Any,
JessicaS11 marked this conversation as resolved.
Show resolved Hide resolved
) -> CMRParams: ...

@overload
def __get__(
self,
instance: 'Parameters[Literal["required"]]',
owner: Any,
) -> EGIRequiredParams: ...

@overload
def __get__(
self,
instance: 'Parameters[Literal["subset"]]',
owner: Any,
) -> EGIParamsSubset: ...

def __get__(
self,
instance: "Parameters",
owner: Any,
) -> Union[CMRParams, EGIRequiredParams, EGIParamsSubset]:
"""
Returns the dictionary of formatted keys associated with the
parameter object.
"""
return instance._fmted_keys


# ----------------------------------------------------------------------
# DevNote: Currently, this class is not tested!!
# DevGoal: this could be expanded, similar to the variables class, to provide users with valid options if need be
# DevGoal: currently this does not do much by way of checking/formatting of other subsetting options (reprojection or formats)
# it would be great to incorporate that so that people can't just feed any keywords in...
class Parameters:
class Parameters(Generic[T]):
"""
JessicaS11 marked this conversation as resolved.
Show resolved Hide resolved
Build and update the parameter lists needed to submit a data order

Expand All @@ -206,7 +257,14 @@ class Parameters:
on the type of query. Must be one of ['search','download']
"""

def __init__(self, partype, values=None, reqtype=None):
fmted_keys = _FmtedKeysDescriptor()

def __init__(
self,
partype: T,
values=None,
reqtype=None,
):
assert partype in [
"CMR",
"required",
Expand Down Expand Up @@ -242,15 +300,7 @@ def poss_keys(self):

# return self._wanted

@property
def fmted_keys(self):
"""
Returns the dictionary of formatted keys associated with the
parameter object.
"""
return self._fmted_keys

def _get_possible_keys(self):
JessicaS11 marked this conversation as resolved.
Show resolved Hide resolved
def _get_possible_keys(self) -> dict[str, list[str]]:
"""
Use the parameter type to get a list of possible parameter keys.
"""
Expand Down Expand Up @@ -347,7 +397,7 @@ def check_values(self):
else:
return False

def build_params(self, **kwargs):
def build_params(self, **kwargs) -> None:
"""
Build the parameter dictionary of formatted key:value pairs for submission to NSIDC
in the data request.
Expand Down Expand Up @@ -443,3 +493,8 @@ def build_params(self, **kwargs):
k = "Boundingshape"

self._fmted_keys.update({k: kwargs["spatial_extent"]})


CMRParameters = Parameters[Literal["CMR"]]
RequiredParameters = Parameters[Literal["required"]]
SubsetParameters = Parameters[Literal["subset"]]
Loading
Loading