Pen programming language

Pen is the parallel, concurrent, and functional programming language focused on application programming following Go's philosophy. It aims for further simplicity, testability, and portability to empower team (v. individual) and/or long-term (v. short-term) productivity.

Its syntax, type system, effect system, and module system are fashioned to achieve those goals being simple and easy to grasp for both newcomers and experts. One of the biggest differences from the other functional languages is polymorphism without generics.

Pen provides the two built-in functions of go and race to represent many concurrent/parallel computation patterns. Thanks to its syntax, type system, and the state-of-the-art reference counting garbage collection, programs are always memory safe and data-race free.

System libraries and runtime in Pen are detachable from applications. Thanks to this, Pen can compile the same applications even for WebAssembly and WASI. Pen also provides Rust/C FFI to reuse existing libraries written in those languages.

import Core'Number
import Os'File

# The `\` prefix for λ denotes a function.
findAnswer = \(kind string) number {
  # Secret source...

  21
}

main = \(ctx context) none {
  # The `go` function runs a given function in parallel.
  # `x` is a future for the computed value.
  x = go(\() number { findAnswer("humanity") })
  y = findAnswer("dolphins")

  _ = File'Write(ctx, File'StdOut(), Number'String(x() + y))

  none
}

Install

Pen is available via Homebrew.

brew install pen-lang/pen/pen

For more information, see Install.

Examples

See the examples directory.

Documentation

• Getting started
• Guides
  • Building an executable
  • Creating a library
  • Using a library
• Language reference
  • Syntax
  • Types

Comparison with Go

Overview

	Pen	Go
Domain	Application programming	System programming
Paradigm	Functional	Imperative / object-oriented
Memory management	Reference counting	Concurrent mark-and-sweep
System library	Your choice!	Built-in
Values	Immutable	Mutable

Runtime

	Pen	Go
Context switch	Continuations	Platform dependent
Concurrent computation	Built-in functions	go expression
Synchronization	Futures, lazy lists	Channels, concurrent data structures
Data race prevention	Built into GC	Dynamic analysis
Resource management	Built into GC	defer statement
Error handling	error type, ? operator	error type, multi-value return
Exception	None	panic and recover functions

Types

	Pen	Go
Number	number (IEEE 754)	int, float64, ...
Sequence	[number] (lazy list)	[]int (array or slice)
Map	{string: number}	map[string]int
Optional value	none, union types	null pointer (or zero value)
Function	\(number, boolean) string	func(int, bool) string
Union	number | string	Interface
Top type	any	any (interface{})
Interface	Records	Interface
Futures	Functions (thunks)	None
Concurrent queue	[number], built-in functions	chan int

The \ (lambda, λ) notation in function types and literals originates from other functional programming languages like Haskell.

Technical design

Polymorphism without generics

Pen explicitly omit generics (or specifically parametric polymorphism for user-defined functions and types) from its language features as well as the original Go. It is one of the biggest experiments in the language as most of existing functional languages have generics as their primary features.

Instead, we explore polymorphism with other language features, such as generic constructs (e.g. list comprehension and pattern matches,) subtyping, top types, reflection, code generation, and so on. A belief behind this decision is that Pen can achieve the same flexibility as other languages reducing complexity of the language itself. For the same reason, we don't adopt macros as we believe they are too powerful for humanity to handle.

Dynamic effect system

Pen does not adopt any formal effect system of algebraic effects or monads. Instead, Pen rather uses a simple rule to manage side effects: all effects are passed down from the main functions to child functions. So unless we pass those impure functions to other functions explicitly, they are always pure. As such, Pen is an impure functional programming language although all runtime values are immutable. However, it still provides many of the same benefits purely functional languages do, such as determinicity and testability.

The reason we do not adopt any formal and statically provable effect system is to keep the language and its type system simple and lean for the purpose of improving developer productivity and software development scalability; we want to make Pen accessible and easy to learn for both newbie and expert programmers.

Context switch

Like Go, every function in Pen is suspendable and can be called asynchronously. This is realized by intermediate representation compiled into Continuation Passing Style (CPS) which also enables proper tail calls. Thus, Pen implements context switch without any platform-dependent codes for slight sacrifice of performance while Go requires logic written in assembly languages.

Currently, Pen does not use delimited continuations for the following reasons.

• Traditional continuations are sufficient for our use cases, such as asynchronous programming.
• Delimited continuations require heap allocations although the second-class continuations do not.

Reference counting GC

Pen implements the Perceus reference counting as its GC. Thanks to the state-of-the-art ownership-based RC algorithm, programs written in Pen performs much less than traditional RC where every data transfer or mutation requires counting operations. In addition, the algorithm reduces heap allocations significantly for records behind unique references, which brings practical performance without introducing unsafe mutability.

See also How to Implement the Perceus Reference Counting Garbage Collection.

Inductive values

TBD

Stackful coroutines

TBD

Contributing

Pen is under heavy development. Feel free to post Issues and Discussions!

Workflows

Installing from source

See Install.

Building crates

tools/build.sh

Running unit tests

tools/unit_test.sh

Running integration tests

tools/build.sh
tools/integration_test.sh

Running benchmarks

Those benchmarks include ones written in both Pen and Rust.

tools/benchmark.sh

Linting crates

tools/lint.sh

Formatting crates

tools/format.sh

Directory structure

• cmd: Commands
  • pen: pen command
• lib: Libraries for compiler, formatter, documentation generator, etc.
  • app: Platform-agnostic application logic for pen command
  • infra: Platform-dependent logic for pen command
  • ast: Abstract Syntax Tree (AST) types
  • hir: High-level Intermediate Representation (HIR) types and semantics
  • mir: Mid-level Intermediate Representation (MIR)
  • ast-hir: AST to HIR compiler
  • hir-mir: HIR to MIR compiler
  • mir-fmm: MIR to F-- compiler
• packages: Packages written in Pen
  • core: Package for platform-independent algorithms and data structures
  • os: Package for a common OS interface
• tools: Developer and CI tools
• doc: Documentation at pen-lang.org

License

Pen is dual-licensed under MIT and Apache 2.0.