Feature request: unpack

[elyase]

This is similar to itertools.starmap but without the map part. It would allow the following:

pipe(
   data,
   unpack(f), # ---> call f(*args) instead of f(args)
   list
)

Currently you need to do it via lambda:

data = ((1, 2, 3, 4), ('a', 'b', 'c', 'd'), ('🤔', '🙈', '😌',  '💕'))
pipe(
    data,
    lambda x: zip(*x),
    list
)
# [(1, 'a', '🤔'), (2, 'b', '🙈'), (3, 'c', '😌'), (4, 'd', '💕')]

Also requested in funcy and StackOverflow

[eriknw]

Thanks, @elyase, good suggestion. I'm open to the idea. The solution with lambda isn't bad imho and is what I would usually go with, but let's explore this some more.

We already have toolz.apply, which is a simple function such that apply(f, 1, 2) is the same as f(1, 2). It could be reasonable to introduce an old-style apply that unpacks the arguments. What's a good name? starapply, because it's similar to starmap? applyunpack? Let's see how this looks:

from toolz import pipe, curry

@curry
def starapply(func, args, kwargs={}):
    return func(*args, **kwargs)

data = ((1, 2, 3, 4), ('a', 'b', 'c', 'd'), ('🤔', '🙈', '😌',  '💕'))

pipe(
   data,
   starapply(f),
   list
)

This uses a curried version of starapply, which is a common style when using pipe.

I don't think this looks bad at all. In fact, I kind of like it, and the name starapply makes sense to me. Thoughts?

[mentalisttraceur]

I think starapply is a great name for this.

The only downside is that I could see someone unfamiliar with this idea misinterpreting it as:

the result of star-applying f (whatever that means, I'm not sure yet) is going to be called with this argument

rather than as the more correct

this argument will be star-applied to f (* splatted when f is called).

I think that's probably fine though. In as little as there is any convention for naming stuff that does this at all in Python, star{{ name }} as the splatting equivalent of ``{{ name }}` is that convention.

I've been thinking about names or better interfaces for this pattern for about a year now, and I think starapply is the best I've seen, especially in the context of a library which already has the apply function that toolz.functoolz does.

[mentalisttraceur]

starcall might be a good name too.

[mjpieters]

Has any progress been made on this?

I currently work around this by using curried.reduce(some_curried_function):

from pytoolz import curried, curry

def starapply(func, *args, **kwargs):
     # turn func(*args, **kwargs) into f(more) -> func(*args, *more, **kwargs)
     return curried.reduce(curry(func, *args, **kwargs))

demo:

>>> import datetime
>>> this_year = starapply(datetime.date, 2022)
>>> args = (9, 28)
>>> this_year(args)  # not *args!
datetime.date(2022, 9, 28)

This works for my purposes but it won't work for functions that accept fewer args than passed in (the error depends on the return value at the point enough arguments have been passed in), plus I am not sure my version can be made into a curried function.

[mentalisttraceur]

Looks like there is an apply in PyPI that brings back the apply that came built-in with old Python versions.

So you could just do

from functools import partial

from apply import apply

def starcall(f):
    return partial(apply, f)

If that's all that's desired, then if I was the toolz maintainer I might be inclined to not bother with it, because it's just two lines on top of existing functionality.

However, I'm thinking maybe there's something better we can do here!

Imagine a starcall where either of these works:

• starcall(f)([1, 2])
• starcall(f)({"foo":1, "bar":2})

In other words, the signature of the callable returned by starcall would be (args_or_kwargs), and ideally the implementation would then check if the argument object can take a * or a **, and apply the right one accordingly.

Since dictionaries can take both * and **, it should probably check for compatibility with ** first, and then fall back to *. I did a bit of digging, and apparently the most correct (most matching CPython's actual ** splatting behavior) ahead-of-time check for **-compatibility is to do hasattr(args_or_kwargs, 'keys') and hasattr(type(args_of_kwargs), '__getitem__').

(As much as I'd love to test for * and ** compat by simply doing the splat and catching the TypeError, f itself might raise a TypeError... a dummy function that does nothing would solve that, if we're fine with accidentally breaking obscure edge-cases where args_or_kwargs has side-effects when iterated or can only be iterated once.)

So now we're building up to something pretty useful that also happens to be just complex and long enough to be worth "getting right" once and then reusing, but also doesn't require pulling in any fancy dependencies to get right - so a pretty good value proposition for toolz I think:

class starcall:
    __slots__ = ('function',)

    def __init__(self, function):
        if not callable(function):
            raise TypeError('starcall argument must be callable')
        self.function = function

    def __call__(self, args_or_kwargs):
        splat_as_kwargs = True
        try:
            args_or_kwargs.keys
            type(args_or_kwargs).__getitem__
        except AttributeError:
            splat_as_kwargs = False
        if splat_as_kwargs:
            return self.function(**args_or_kwargs)
        return self.function(*args_or_kwargs)

[mentalisttraceur]

Perhaps overkill, but we could also fully generalize the starcall sketch from the end of my last comment, to support passing through any combination of arguments, by adding a class that can hold both *-splatted and **-splatted arguments:

class Arguments:
    __slots__ = ('args', 'kwargs')

    def __init__(self, /, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs

    @classmethod
    def wrap(cls, args_or_kwargs):
        if isinstance(args_or_kwargs, Arguments):
            return cls(*args_or_kwargs, **args_or_kwargs)
        try:
            args_or_kwargs.keys
            type(args_or_kwargs).__getitem__
        except AttributeError:
            return cls(*args_or_kwargs)
        return cls(**args_or_kwargs)

    def __iter__(self):
        return iter(self.args)

    def keys(self):
        return self.kwargs.keys()

    def __getitem__(self, key):
        return self.kwargs[key]

And then starcall.__call__ can be simplified, since all the tricky logic moved into Arguments.wrap:

class starcall:
    __slots__ = ('function',)

    def __init__(self, function):
        if not callable(function):
            raise TypeError('starcall argument must be callable')
        self.function = function

    def __call__(self, args_or_kwargs):
        arguments = Arguments.wrap(args_or_kwargs)
        return self.function(*arguments, **arguments)

With that in place, in a pipe(f, starcall(g)), f can do either of these:

return (1, 4.2)  # or Arguments(1, 4.2)

return {'foo': 1, 'bar': 4.2}  # or Arguments(foo=1, bar=4.2)

return Arguments(1, bar=4.2)

to cause g to get called as g(1, 4.2), g(foo=1, bar=4.2), and g(1, bar=4.2).

[mentalisttraceur]

Actually I'm on the fence about Arguments being directly splattable: starcall could just as easily look like this:

class starcall:
    __slots__ = ('function',)

    def __init__(self, function):
        if not callable(function):
            raise TypeError('starcall argument must be callable')
        self.function = function

    def __call__(self, args_or_kwargs):
        arguments = Arguments.wrap(args_or_kwargs)
        return self.function(*arguments.args, **arguments.kwargs)

and then the arguments class doesn't need those three special methods just to be splattable:

class Arguments:
    __slots__ = ('args', 'kwargs')

    def __init__(self, /, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs

    @classmethod
    def wrap(cls, args_or_kwargs):
        if isinstance(args_or_kwargs, Arguments):
            return args_or_kwargs
        try:
            args_or_kwargs.keys
            type(args_or_kwargs).__getitem__
        except AttributeError:
            return cls(*args_or_kwargs)
        return cls(**args_or_kwargs)

(Also have mixed feelings about calling those attributes args and kwargs vs something like positional and keyword, but I'm trusting that the standard abbreviated names are better for understanding, especially outside of fluent English and regardless of how the arguments object is named, because they're so universal in Python.)

[mentalisttraceur]

Oh! I overcomplicated it, starcall can just be a regular (curried) function:

@curry
def starcall(function, args_or_kwargs):
    arguments = Arguments.wrap(args_or_kwargs)
    return function(*arguments.args, **arguments.kwargs)

The currying enables the pipe(..., starcall(f), ...) case.

---

Now, that doesn't support the example @mjpieters had:

>>> this_year = starapply(datetime.date, 2022)
>>> args = (9, 28)
>>> this_year(args)  # not *args!
datetime.date(2022, 9, 28)

But I think that's fine, because that example's starapply is just also reimplementing partial application. In situations where we want that, we can do starcall = compose(starcall, partial), and also either of these alternatives works:

>>> this_year = starcall(partial(datetime.date, 2022))
>>> args = (9, 28)
>>> this_year(args)  # not *args!
datetime.date(2022, 9, 28)

>>> date = curry(datetime.date)
>>> this_year = starcall(date(2022))
>>> args = (9, 28)
>>> this_year(args)  # not *args!
datetime.date(2022, 9, 28)

[mentalisttraceur]

Inspired by #557 , an idea for enhancing the Arguments object I've been drafting here:

Add a build method which builds the arguments instance from every value or name-value tuple yielded by an iterable:

class Arguments:
    __slots__ = ('args', 'kwargs')

    def __init__(self, /, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs

    @classmethod
    def wrap(cls, args_or_kwargs):
        if isinstance(args_or_kwargs, Arguments):
            return args_or_kwargs
        try:
            args_or_kwargs.keys
            type(args_or_kwargs).__getitem__
        except AttributeError:
            return cls(*args_or_kwargs)
        return cls(**args_or_kwargs)

    @classmethod
    def from_iterable(cls, iterable):
        args = []
        kwargs = {}
        for argument in iterable:
            if isinstance(argument, tuple) and len(argument) == 2:
                name, value = argument
                if name is not None:
                    kwargs[name] = value
                    continue
            args.append(argument)
        return cls(*args, **kwargs)

Of course starcall itself remains the same, but functions now have another ergonomic option for building arguments for starcall, and this pairs well with the composed proposed in #557:

@composed(Arguments.from_iterable)
def foo():
    yield 123
    yield 'bar', 456

def example(*args, **kwargs):
    print(args, kwargs)

f = compose(starcall(example), foo)

f()  # prints: (123,) {'bar': 456}

Of course, maybe it would be even better to just provide a decorator version of Arguments.from_iterable... perhaps @Arguments.builder or (inspired by #557 ) @collect_arguments.

Also, wrap should probably be renamed for symmetry and clarity to from_splattable.

(Not going back to my earlier idea of from_args_or_kwargs, because I've now realized that that older name is also ambiguous: from just that name itself it's not obvious if you would use from_args(args) or from_args(*args).)