[meta] Ibis expression API stability

[lostmygithubaccount]

Acceptance criteria: the Ibis user-facing expression API is stabilized for 10.0+. While we will always reserve the right to make breaking changes, we are now in the place where stability is important for production dependencies. Organizations choosing Ibis should feel confident their code will work with minimal effort when upgrading to new versions.

TODOs:

P1

dtype or type or cast

• bug: many Operations can't handle NULL optional arguments #8833

refactor

• refactor: deprecate NA/fillna/dropna in favor of name that uses null instead of na #7276 (associated PR: refactor: deprecate fillna and dropna api #9300)
• refactor: remove ibis.NA  #9311 (associated PR: refactor: remove ibisNA #9344)
• feat(api): standardize the position of null values in sorted columns and provide an api for specifying null order #7252 (associated PR feat(api): add nulls_first=False argument to order_by #9385)

dtype or type or cast

• Possibility to extract isoyear and isoweekday (or weekday) from a timestamp column. #8989 (draft PR open feat(api): isoyear method #9034)
• feat: Support type arg to ibis.array and ibis.map #8289 (PR open feat: improve map(), struct(), array() #8666 needs work)

P2

refactor or deprecate or breaking changes

• refactor(ux): ensure that all known and knowable type errors raise UnsupportedBackendType instead of letting code pass through to the backend #8333

dtype or type

• bug: dt.highest_precedence([dt.uint64, dt.int32]) gives uint64, not int64 #7331
• feat: allow specifying Optional and/or T | None in UDFs #7133

other

• feat: consolidate ibis.case(), Value.case(), Value.cases(), Value.substitute() #7280 (PR in progress feat(api): move from .case() to .cases() #9039) (new PR waiting review: feat(api): move from .case() to .cases() #9096)

[ncclementi]

Weekly Update

06/05/2024

• bug: many Operations can't handle NULL optional arguments #8833 (closed via chore: add box to scalar expr in interactive mode #9265)

• [meta] Ibis expression API stability #8996 (PR under review refactor: deprecate fillna and dropna api #9300)

• Possibility to extract isoyear and isoweekday (or weekday) from a timestamp column. #8989 (closed via community PR feat(api): isoyear method #9034)

• feat: Support type arg to ibis.array and ibis.map #8289 is being broken in multiple PR that Nick Crews is working on. Some have being merged:

#9305
#9306
#9302

The main one is still work in progress. #8666

[ncclementi]

Weekly Update

06/13/2024

Closed as complete:

• refactor: deprecate NA/fillna/dropna in favor of name that uses null instead of na #7276
• refactor: remove ibis.NA  #9311

Started and in progress:

• feat(api): standardize the position of null values in sorted columns and provide an api for specifying null order #7252

Note:
#8289 is being further splitted #8666 (comment)

[ncclementi]

closing in favor of #9638