Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: threading support #4559

Draft
wants to merge 17 commits into
base: dev
Choose a base branch
from
Draft

WIP: threading support #4559

wants to merge 17 commits into from

Conversation

aykevl
Copy link
Member

@aykevl aykevl commented Oct 28, 2024

Work in progress. Many necessary parts are not yet implemented, but this gives context for PRs I want to make in the future.

See #2446 for motivation (in particular, why this focuses on Linux first) and for a list of things that need to be implemented to make multithreading work in TinyGo.

Copy link

github-actions bot commented Oct 28, 2024

Size difference with the dev branch:

Binary size difference
 flash                          ram
 before   after   diff          before   after   diff
  75776   75528   -248  -0.33%    7480    7472     -8  -0.11% tinygo build -size short -o ./build/test.hex -target=p1am-100 ./examples/p1am/main.go
 338356  338116   -240  -0.07%   15396   15368    -28  -0.18% tinygo build -size short -o ./build/test.hex -target=matrixportal-m4 -stack-size 8kb ./examples/net/webstatic/
 286056  285860   -196  -0.07%   17812   17820      8   0.04% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/webserver/
 287076  286920   -156  -0.05%   18212   18204     -8  -0.04% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/mqttclient/paho/
  13720   13712     -8  -0.06%    4940    4940      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/sx126x/lora_rxtx/
  83056   83048     -8  -0.01%    6588    6588      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/lora/lorawan/atcmd/
  16844   16844      0   0.00%    4324    4324      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/adafruit4650
  61320   61320      0   0.00%    6196    6196      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adt7410/main.go
   9532    9532      0   0.00%    4756    4756      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adxl345/main.go
  13528   13528      0   0.00%    6796    6796      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/amg88xx
   8644    8644      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/main.go
   8332    8332      0   0.00%    2320    2320      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/at24cx/main.go
   8088    8088      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bh1750/main.go
   7396    7396      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/blinkm/main.go
  70472   70472      0   0.00%    3660    3660      0   0.00% tinygo build -size short -o ./build/test.hex -target=pinetime     ./examples/bma42x/main.go
  63868   63868      0   0.00%    6196    6196      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmi160/main.go
  27388   27388      0   0.00%    4788    4788      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp180/main.go
  63920   63920      0   0.00%    6228    6228      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp280/main.go
  12176   12176      0   0.00%    4820    4820      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bmp388/main.go
   8164    8164      0   0.00%    3348    3348      0   0.00% tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/sram/main.go
   4580    4580      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/easystepper/main.go
  69144   69144      0   0.00%    6976    6976      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/flash/console/spi
   7172    7172      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/gc9a01/main.go
   7836    7836      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/hcsr04/main.go
   5792    5792      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/customchar/main.go
   5744    5744      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/text/main.go
  10492   10492      0   0.00%    4756    4756      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/hd44780i2c/main.go
  16108   16108      0   0.00%    2360    2360      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hub75/main.go
  10116   10116      0   0.00%    6916    6916      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/basic
  10608   10608      0   0.00%    4868    4868      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/basic
  29488   29488      0   0.00%   38076   38076      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/pyportal_boing
  10136   10136      0   0.00%    6924    6924      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/scroll
  10696   10696      0   0.00%    4876    4876      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/scroll
  11696   11696      0   0.00%    4788    4788      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis3dh/main.go
  26092   26092      0   0.00%    2328    2328      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/lsm303agr/main.go
  12412   12412      0   0.00%    4796    4796      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/lsm6ds3/main.go
  10708   10708      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mag3110/main.go
   9896    9896      0   0.00%    4780    4780      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017/main.go
  10348   10348      0   0.00%    4788    4788      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017-multiple/main.go
   9744    9744      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp3008/main.go
  68496   68496      0   0.00%    6196    6196      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp2515/main.go
   8204    8204      0   0.00%    4756    4756      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mma8653/main.go
   8112    8112      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mpu6050/main.go
  12232   12232      0   0.00%    3352    3352      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/pca9685/main.go
   6188    6188      0   0.00%    3288    3288      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setbuffer/main.go
   5220    5220      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setpixel/main.go
  10516   10516      0   0.00%    3328    3328      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/seesaw
   2825    2825      0   0.00%     558     558      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino ./examples/servo
  13748   13748      0   0.00%    3400    3400      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico     ./examples/sgp30
   8044    8044      0   0.00%    6796    6796      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/shifter/main.go
  57300   57300      0   0.00%    3692    3692      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht3x/main.go
  57364   57364      0   0.00%    3700    3700      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht4x/main.go
  57272   57272      0   0.00%    3692    3692      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/shtc3/main.go
   6648    6648      0   0.00%    2288    2288      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1306/i2c_128x32/main.go
   6100    6100      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1306/spi_128x64/main.go
   5808    5808      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1331/main.go
   6748    6748      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7735/main.go
   6660    6660      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7789/main.go
  16932   16932      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/thermistor/main.go
  10032   10032      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/tm1637/main.go
  10848   10848      0   0.00%    3340    3340      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/touch/capacitive
   9504    9504      0   0.00%    6788    6788      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/fourwire/main.go
  12572   12572      0   0.00%    6984    6984      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/pyportal_touchpaint/main.go
  14956   14956      0   0.00%    4756    4756      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl53l1x/main.go
  13436   13436      0   0.00%    4756    4756      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl6180x/main.go
   6452    6452      0   0.00%    2320    2320      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13/main.go
   6140    6140      0   0.00%    2312    2312      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13x/main.go
   6364    6364      0   0.00%    2320    2320      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd4in2/main.go
  26432   26432      0   0.00%   16732   16732      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/waveshare-epd/epd2in66b/main.go
   6812    6812      0   0.00%    4788    4788      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/ws2812
   5734    5734      0   0.00%    9538    9538      0   0.00% tinygo build -size short -o ./build/test.bin -target=m5stamp-c3          ./examples/ws2812
   1581    1581      0   0.00%     598     598      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino   ./examples/ws2812
   1056    1056      0   0.00%     180     180      0   0.00% tinygo build -size short -o ./build/test.hex -target=digispark ./examples/ws2812
  31816   31816      0   0.00%    4788    4788      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bme280/main.go
  16420   16420      0   0.00%    4732    4732      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/microphone/main.go
  11056   11056      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/buzzer/main.go
  12764   12764      0   0.00%    4788    4788      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/veml6070/main.go
   6728    6728      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/simple/main.go
   8640    8640      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/speed/main.go
   6696    6696      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/simple/main.go
   9252    9252      0   0.00%    4756    4756      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/speed/main.go
   7308    7308      0   0.00%    3320    3320      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-f103rb ./examples/shiftregister/main.go
   6976    6976      0   0.00%    2268    2268      0   0.00% tinygo build -size short -o ./build/test.hex -target=hifive1b ./examples/ssd1351/main.go
  12968   12968      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis2mdl/main.go
   8844    8844      0   0.00%    4764    4764      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/max72xx/main.go
   7240    7240      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/clkout/
  12164   12164      0   0.00%    3304    3304      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/qmi8658c/main.go
  10788   10788      0   0.00%    3288    3288      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/pcf8591/
   8720    8720      0   0.00%    4756    4756      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina260/main.go
  13160   13160      0   0.00%    4788    4788      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina219/main.go
   9300    9300      0   0.00%    5260    5260      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-l432kc ./examples/aht20/main.go
  61416   61416      0   0.00%    8232    8232      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/i2csoft/adt7410/
  10260   10260      0   0.00%    6796    6796      0   0.00% tinygo build -size short -o ./build/test.elf -target=wioterminal ./examples/axp192/m5stack-core2-blinky/
   9008    9008      0   0.00%    3276    3276      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/xpt2046/main.go
  11236   11236      0   0.00%    4252    4252      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/irremote/main.go
  11932   11932      0   0.00%    3324    3324      0   0.00% tinygo build -size short -o ./build/test.hex -target=badger2040 ./examples/uc8151/main.go
  10408   10408      0   0.00%    3356    3356      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/scd4x/main.go
   8516    8516      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=circuitplay-express ./examples/makeybutton/main.go
   9628    9628      0   0.00%    4772    4772      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ds18b20/main.go
  15896   15896      0   0.00%    4880    4880      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/as560x/main.go
   9868    9868      0   0.00%    3296    3296      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu6886/main.go
   7772    7772      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ttp229/main.go
  62196   62196      0   0.00%    3788    3788      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ndir/main_ndir.go
  65180   65180      0   0.00%    6260    6260      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ndir/main_ndir.go
   9296    9296      0   0.00%    3288    3288      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu9150/main.go
  11376   11376      0   0.00%    3324    3324      0   0.00% tinygo build -size short -o ./build/test.hex -target=macropad-rp2040 ./examples/sh1106/macropad_spi
   8436    8436      0   0.00%    3760    3760      0   0.00% tinygo build -size short -o ./build/test.hex -target=macropad-rp2040 ./examples/encoders/quadrature-interrupt
   9760    9764      4   0.04%    4768    4768      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/itsybitsy-m0/main.go
  22152   22160      8   0.04%    3544    3544      0   0.00% tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/time/main.go
 263524  263532      8   0.00%   46760   46760      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/slideshow
  69600   69616     16   0.02%    6376    6376      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/ds3231/main.go
  67540   67556     16   0.02%    6368    6368      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/i2c/main.go
  68076   68092     16   0.02%    6504    6504      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/uart/main.go
  62324   62340     16   0.03%    5952    5952      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/is31fl3731/main.go
  77188   77204     16   0.02%    6344    6344      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/dht/main.go
  36512   36528     16   0.04%    3988    3988      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/pcf8523/
  71208   71224     16   0.02%    6344    6344      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/alarm/
  70744   70760     16   0.02%    6340    6340      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/time/
  71176   71192     16   0.02%    6352    6352      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/timer/
  66752   66768     16   0.02%    4816    4816      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/ndir/main_ndir.go
  65720   65736     16   0.02%    4784    4784      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mcp9808/main.go
  65532   65556     24   0.04%    9012    9012      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/flash/console/qspi
  72620   72644     24   0.03%   10748   10748      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/sdcard/console/
  11768   11796     28   0.24%    6580    6580      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/apds9960/proximity/main.go
  14692   14720     28   0.19%    6580    6580      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/hts221/main.go
  14036   14064     28   0.20%    6580    6580      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/lps22hb/main.go
  10496   10524     28   0.27%    4540    4540      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-bluefruit ./examples/tone
  24676   24704     28   0.11%   13892   13892      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840-sense ./examples/waveshare-epd/epd1in54/main.go
 245452  245516     64   0.03%   10488   10480     -8  -0.08% tinygo build -size short -o ./build/test.hex -target=nano-rp2040 -stack-size 8kb ./examples/net/websocket/dial/
 111500  111572     72   0.06%    7772    7764     -8  -0.10% tinygo build -size short -o ./build/test.hex -target=arduino-mkrwifi1010 -stack-size 8kb ./examples/net/tlsclient/
 119288  119368     80   0.07%    7848    7840     -8  -0.10% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 -stack-size 8kb ./examples/net/tcpclient/
 296580  296668     88   0.03%   13788   13780     -8  -0.06% tinygo build -size short -o ./build/test.hex -target=pyportal -stack-size 8kb ./examples/net/http-get/
 154136  154232     96   0.06%    6540    6532     -8  -0.12% tinygo build -size short -o ./build/test.hex -target=nano-rp2040 -stack-size 8kb ./examples/net/mqttclient/natiu/
  27220   27332    112   0.41%    3648    3640     -8  -0.22% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/microbitmatrix/main.go
  31336   31448    112   0.36%    4548    4540     -8  -0.18% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/ssd1289/main.go
  26948   27068    120   0.45%    5696    5688     -8  -0.14% tinygo build -size short -o ./build/test.hex -target=microbit-v2 ./examples/microbitmatrix/main.go
 103048  103176    128   0.12%    9816    9808     -8  -0.08% tinygo build -size short -o ./build/test.hex -target=metro-m4-airlift -stack-size 8kb ./examples/net/socket/
 116992  117140    148   0.13%   13132   13124     -8  -0.06% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/webclient/
  85868   86052    184   0.21%    5140    5140      0   0.00% tinygo build -size short -o ./build/test.hex -target=challenger-rp2040 ./examples/net/ntpclient/
5326584 5327316    732   0.00%  811790  811674   -116  -0.00%

This adds some non-atomic types that have the same interface as the ones
in sync/atomic.

We currently don't need these to be atomic (because the scheduler is
entirely cooperative), but once we add support for a scheduler with
multiple threads and/or preemptive scheduling we can trivially add some
type aliases under a different build tag in the future when real
atomicity is needed for threading.
This uses the internal/llsync package to convert some non-atomic
operations to pseudo-atomic operations that will become truly atomic on
systems that need this (such as multicore chips and when we implement
true parallelism on Linux).
This moves all scheduler code into a separate file that is only compiled
when there's a scheduler in use (the tasks or asyncify scheduler, which
are both cooperative). The main goal of this change is to make it easier
to add a new "scheduler" based on OS threads.

It also fixes a few subtle issues with `-gc=none`:

  - Gosched() panicked. This is now fixed to just return immediately
    (the only logical thing to do when there's only one goroutine).
  - Timers aren't supported without a scheduler, but the relevant code
    was still present and would happily add a timer to the queue. It
    just never ran. So now it exits with a runtime error, similar to any
    blocking operation.
See the code comments for details. But in short, this implements a futex
for the cooperative scheduler (that is single threaded and
non-reentrant). Similar implementations can be made for basically every
other operating system, and even WebAssembly with the threading
(actually: atomics) proposal.
Using this basic futex implementation means we can use the same
implementation for synchronisation primitives on cooperative and
multicore systems.

For more information on futex across operating systems:
https://outerproduct.net/futex-dictionary.html
PMutex is a mutex when threading is possible, and a dummy mutex-like
object (that doesn't do anything) otherwise.
Code size for the cooperative scheduler is nearly unchanged.
Make sure the object is locked when trying to modify it.
Binary size seems unaffected when not using threading.
@aykevl aykevl force-pushed the threading branch 2 times, most recently from bcee341 to 28f51ca Compare November 2, 2024 10:09
@aykevl
Copy link
Member Author

aykevl commented Nov 2, 2024

Updated after some refactoring (moved internal/llsync into internal/task) and added GC support (stop-the-world scanning by pausing all threads and scanning their stacks).
Main issues remaining are:

  • Some problem with bits/fcntl.h that I need to investigate (see CI failures).
  • Timer support.
  • Fixing tests that assume a deterministic order in which goroutines are run.
  • Fixing signal support.

This is a small refactor to prepare GC marking for multithreaded
stop-the-world.
Using a global lock may be slow, but it is certainly simple and safe.
If this global lock becomes a bottleneck, we can of course look into
making the GC truly support multithreading.
Move common functions to scheduler.go. They will be used both from the
cooperative and from the threads scheduler.
Somewhat surprisingly, this results in smaller code than the old code
with the cooperative (tasks) scheduler. Probably because the new RWMutex
is also simpler.
This actually simplifies the code and avoids a heap allocation in the
call to Wait. Instead, it uses the Data field of the task to store
information on whether a task was signalled early.
Comment on lines +11 to +13
#define FUTEX_WAIT 0
#define FUTEX_WAKE 1
#define FUTEX_PRIVATE 128
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why redefine these here if they can be included from <linux/futex.h>?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Musl (which we use) doesn't provide linux/futex.h.

src/internal/task/futex_linux.c Show resolved Hide resolved

#define FUTEX_WAIT 0
#define FUTEX_WAKE 1
#define FUTEX_PRIVATE 128
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: also afaik the flag itself is called FUTEX_PRIVATE_FLAG, just to be consistent with FUTEX_WAIT and FUTEX_WAKE

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Fixed locally, will be included the next time I push the branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants