Saturday, 19 September 2026

[3 of 3] I benchmarked five Scala libraries. Which code would I rather read?

I started with synthetic benchmarks in part 1. Adding socket I/O in part 2 sent me into Kyo's scheduler. Along the way, Ox and Gears caught my attention. Their TCP throughput was close to cats-effect, they allocated much less, and I found their direct-style code easier to read.

For this last post, I added plain Loom and checked whether lower allocation also meant less memory. The results still don't give me one choice that wins everywhere.

Five runtimes, different answers

These are selected results from the new run, in operations per second. Higher is better. The ± values are JMH's 99.9% confidence-interval half-widths; compare libraries within each column.

Throughput, including runtime entry and task management
Runtime8 workersSpawn/joinBlocking TCPCallback TCP
cats-effect3,060 ± 921,894 ± 85145.10 ± 9.03141.53 ± 3.07
Kyo1,399 ± 265,355 ± 1044.95 ± 0.03139.99 ± 1.73
Loom / JDK3,420 ± 71558 ± 43143.49 ± 5.65137.69 ± 3.44
Ox3,580 ± 230494 ± 51140.29 ± 6.62136.78 ± 3.48
Gears3,214 ± 50377 ± 26140.28 ± 9.81139.25 ± 2.57

The eight-worker case processes 4,096 values, with 64 arithmetic rounds per value. Spawn/join creates a child, waits for it to finish, and repeats 1,000 times. Each TCP operation completes 256 exchanges over 64 persistent loopback connections. A separate server requests a 1 ms delay per response.

Kyo leads the tiny-task spawn/join case, while the callback TCP intervals overlap across all five. The slow Kyo blocking result uses defaults. In part 2, flushing and worker tuning brought it close to CE. That tuned result belongs to the earlier experiment.

All five ran every case shown here. The Loom row uses JDK virtual threads and primitives. The suspended effect-chain tests from part 1 have no equivalent Loom, Ox or Gears benchmark: replacing the chain with a direct loop changes what we measure. I haven't compared cancellation or resource-safety guarantees.

Less allocation didn't answer the memory question

I kept 100,000 tasks waiting, each holding a 1 KiB payload, and measured live heap after full GC and macOS process footprint. I also sampled memory during blocking TCP work at 64 connections. The numbers are medians from three fresh JVMs per case, in MiB. TCP measurements cover only the client.

Memory consumption: lower is better
RuntimeWaiting tasks: live heapWaiting tasks: process footprintActive TCP: process footprint
cats-effect191.0418.5152.5
KyoNot measured
Loom / JDK228.7390.090.3
OxNot measured
GearsNot measured

CE used less live heap for these waiting tasks. Loom had the smaller process footprint, which includes memory beyond live heap objects. So far, the memory harness covers only CE and Loom. I haven't measured Kyo, Ox or Gears here; the blanks don't indicate missing library support.

Both studies ran on 19 September 2026 on one M3 Max with JDK 25.0.3 and Scala 3.8.4: CE 3.7.1, Kyo 1.0.0-RC6, Ox 1.0.6 and Gears 0.3.1. Throughput used three forks, three 1-second warmups and three 1-second measurements, with a fixed 2 GiB G1 heap. Memory used G1 with a 64 MiB initial and 2 GiB maximum heap. These are short runs of these particular implementations on one machine.

What I want to maintain

I still enjoy writing with cats-effect and Kyo. After moving into Tagless Final, though, I sometimes find myself reading through a lot of ceremony to get to the business logic. Making every service generic in F[_] is a choice I can reconsider while still using CE.

That makes me more interested in direct-style concurrency. I often find its control flow easier to follow, even though writing effects is more fun for me. The benchmark results give me reasons to consider both.

AI makes me reconsider what I value

AI tools handled most of my earlier effect-system migration. If I spend more time reviewing generated code and less time writing it, how much should my enjoyment of writing a particular style count?

That even puts Java back on the table for me. Boilerplate bothers me less when I'm not typing all of it, but I still have to read and maintain it. I could live with more verbose code if I found it easier to follow what runs, how it fails and who owns a resource.

Effects and types can help me review code too. A version with fewer combinators might still hide a cancellation or cleanup bug.

I don't have one library to recommend for every project. I'd choose around the workload, the guarantees I need and the integrations available. Among the options that fit, I'd give more weight to what my team can comfortably read and maintain, whoever wrote it.

No comments:

Post a Comment