Allow typevars in do expressions

[sgaure]

(@c42f edit:) Upstream syntax discussion: JuliaLang/julia#54915

There are several ways to create functions in julia. One of them is the do syntax. You can't dispatch on do functions, but now and then the need comes up to capture the types of variables. Although this is easy with typeof inside the function body, it turns out to be quite easy to enable the use of where clauses in the do syntax. This would make a do function similar to the other function definitions.

The reason it's easy is that such expressions are already being parsed, but a minor detail prevents it from being accepted as a valid expression:

julia-1.10.4> Meta.show_sexpr(:(f() do (x::T) where {T}; T(x) end))
(:do, (:call, :f), (:->, (:tuple, (:where, (:(::), :x, :T), :T)), (:block,
      :(#= REPL[1]:1 =#),
      (:call, :T, :x)
    )))

The where clause becomes encapsulated in a tuple. The reason is that after the parser sees do, it looks for a comma separated list. After the list is parsed, i.e. on ; or linefeed, the list is encapsulated in a tuple. This means that the where clause becomes an argument to the function, which fails as a syntax error. By modifying the parsing of do expressions to avoid inserting tuple if the newly parsed list is a where, the parse result becomes:

julia-this-PR> Meta.show_sexpr(:(f() do (x::T) where {T}; T(x) end))
(:do, (:call, :f), (:->, (:where, (:(::), :x, :T), :T), (:block,
      :(#= REPL[1]:1 =#),
      (:call, :T, :x)
    )))

This is a valid :-> expression, just like an anonymous function definition ((x::T) where T) -> T(x) currently is.

The PR is a one-line change to parser.jl, isolated to parsing what follows do. It does not enable any new or very useful functionality, it merely will make functions defined in do quite similar to other function definitions.

I don't think this change will break anything, as the construction currently results in a ERROR: syntax: ... failure.

[LilithHafner]

I think it is best for proposed syntax changes to start as or have an associated issue in the JuilaLang/julia repo for viability.

[LilithHafner]

LGTM, technically.

[LilithHafner]

Suggested change

	"f(x) do (y::T) where T body end" => "(call f x (do (where (parens (::-i y T)) T) (block body)))"
	"f(x) do y::T where T body end" => "(call f x (do (tuple (::-i y (where T T))) (block body)))"
	"f(x) do (y::T) where T body end" => "(call f x (do (where (parens (::-i y T)) T) (block body)))"

Maybe add a test for the case without parenthesis (IIUC the where clause will bind tighter than the ::).

[LilithHafner]

Actually, as @c42f points out, the relative precedence of :: and where is already context dependent:

julia> parsestmt(SyntaxNode, "function f()::T where T 0 end")
line:col│ tree                                   │ file_name
   1:1  │[function]
   1:10 │  [where]
   1:10 │    [::-i]
   1:10 │      [call]
   1:10 │        f
   1:15 │      T
   1:23 │    T
   1:24 │  [block]
   1:25 │    0


julia> parsestmt(SyntaxNode, "f()::T where T = 0")
line:col│ tree                                   │ file_name
   1:1  │[=]
   1:1  │  [::-i]
   1:1  │    [call]
   1:1  │      f
   1:6  │    [where]
   1:6  │      T
   1:14 │      T
   1:18 │  0

So in this case, I think we should make :: bind tighter than where so parentheses are not necessary here.

Reconsidered

[LilithHafner]

See also: JuliaLang/julia#55159

And also, as a syntax change this would need to be gated on VERSION.