> OS threads are expensive: an operating system thread typically reserves a megabyte of stack space<p>Why is reserving a megabyte of stack space "expensive"?<p>> and takes roughly a millisecond to create<p>I'm not sure where this number is from, it seems off by a few orders of magnitude. On Linux, thread creation is closer to 10 microseconds.
No mention of ruby which has colorless functions.
How many systems are there that can't just spawn a thread for each task they have to work on concurrently? This has to be a system that is A) CPU or memory bound (since async doesn't make disk or network IO faster) and B) must work on ~tens of thousands of tasks concurrently, i.e. can't just queue up tasks and work on only a small number concurrently. The only meaningful example I can come up with are load balancers, embedded software and perhaps something like browsers. But e.g. an application server implementing a REST API that needs to talk to a database anyway to answer each request doesn't really qualify, since the database connection and the work the database itself does are likely much more resource intensive than the overhead of a thread.
I'm not sure this is correct mental model of what async solves<p>Async precisely improves disk/network I/O-bound applications because synchronous code has to waste a whole thread sitting around waiting for an I/O response (each with its own stack memory and scheduler overhead), and in something like an application server there will be many incoming requests doing so in parallel. Cancellation is also easier with async<p>CPU-bound code would not benefit because the CPU is already busy, and async adds overhead<p>See e.g. <a href="https://learn.microsoft.com/en-us/aspnet/web-forms/overview/performance-and-caching/using-asynchronous-methods-in-aspnet-45#:~:text=On%20the%20web,the%20server%20hardware" rel="nofollow">https://learn.microsoft.com/en-us/aspnet/web-forms/overview/...</a> and <a href="https://learn.microsoft.com/en-us/aspnet/web-forms/overview/performance-and-caching/using-asynchronous-methods-in-aspnet-45#:~:text=In%20general%2C%20use%20asynchronous,service%20request%20to%20complete" rel="nofollow">https://learn.microsoft.com/en-us/aspnet/web-forms/overview/...</a>
Inversion of thought pattern: Why is a thread such a waste that we can't have one per concurrent request? Make threads less wasteful instead. Go took things in this direction.
I have some test code that runs a comparison of Hyper pre-async (aka thread per request) vs async (via Tokio), and the pre-async version is able to process more requests per second in every scenario (I/o, CPU complex tasks, shared memory).<p>I'll publish my results shortly. I did these as baselines because I'm testing finishing the User Managed Concurrency Groups proposal to the linux kernel which is an extension to provide faster kernel threads (which beat both of them)
I read this argument ("async is for I/O-bound applications") often, but it makes no sense to me. If your app is I/O bound, how does reducing the work the (already idling!) CPU has to spend on context switching improve the performance of the system?
IO bound might mean latency but not throughput, so you can up concurrency and add batching, both of which require more concurrent requests in flight to hit your real limit. IO bound might also really mean contention for latches on the database, and different types of requests might hit different tables. Basically, I see people say they're IO bound long before they're at the limit of a single disk, so obviously they are not IO bound. Modern drives are absurdly fast. If everyone were really IO bound, we'd need 1/1000 the hardware we needed 10-15 years ago.
It sounds like you're assuming both pieces are running on the same server, which may not be the case (and if you're bottlenecked on the database it probably shouldn't be, because you'd want to move that work off the struggling database server)<p>Assuming for the sake of argument that they are together, you're still saving stack memory for every thread that isn't created. In fact you could say it <i>allows</i> the CPU to be idle, by spending less time context switching. On top of that, async/await is a perfect fit for OS overlapped I/O mechanisms for similar reasons, namely not requiring a separate blocking thread for every pending I/O (see e.g. <a href="https://en.wikipedia.org/wiki/Overlapped_I/O" rel="nofollow">https://en.wikipedia.org/wiki/Overlapped_I/O</a>, <a href="https://stackoverflow.com/a/5283082" rel="nofollow">https://stackoverflow.com/a/5283082</a>)
The simplest example is that you can easily be wasteful in your use of threads. If you just write blocking code, you will block the thread while waiting on io, and threads are a finite resource.<p>So avoiding that would mean a server can handle more traffic before running into limits based on thread count.
I think it's another case of the whole industry being driven by the needs of the very small number of systems that need to handle >10k concurrent requests.
Async does make nvme io faster because queueing multiple operations on the nvme itself is faster.
Async is a Javascript hack that inexplicably got ported to other languages that didn't need it.<p>The issue arose because Javascript didn't have threads, and processing events from the DOM is naturally event driven. To be fair, it's a rare person who can deal with the concurrency issues threads introduce, but the separate stacks threads provide a huge boon. They allow you to turn event driven code into sequential code.<p><pre><code> window.on_keydown(foo);
// Somewhere far away
function foo(char_event) { process_the_character(char_event.key_pressed) };
</code></pre>
becomes:<p><pre><code> while (char = read())
process_the_character(char);
</code></pre>
The latter is easy to read linear sequence of code that keeps all the concerns in one place, the former rapidly becomes a huge entangled mess of event processing functions.<p>The history of Javascript described in the article is just a series of attempts to replace the horror of event driven code with something that looks like the sequential code found in a normal program. At any step in that sequence, the language could have introduced green threads and the job would have been done. And it would have been done without new syntax and without function colouring. But if you keep refining the original hacks they were using in the early days and don't the somewhat drastic stop of introducing a new concept to solve the problem (separate stacks), you end up where they did - at async and await. Mind you, async and await to create a separate stack of sorts - but it's implemented as a chain objects malloc'ed on the heap instead the much more efficient stack structure.<p>I can see how the javascript community fell into that trap - it's the boiling frog scenario. But Python? Python already had threads - and had the examples of Go and Erlang to show how well then worked compared to async / await. And as for Rust - that's beyond inexplicable. Rust has green threads in the early days and abandoned them in favour of async / await. Granted the original green thread implementation needed a bit of refinement - making every low level choose between event driven and blocking on every invocation was a mistake. Rust now has a green thread implementation that fixes that mistake, which demonstrates it wasn't that hard to do. Yet they didn't do it at the time.<p>It sounds like Zig with its pluggable I/O interface finally got it right - they injected I/O as a dependency injected at compile time. No "coloured" async keywords and compiler monomorphises the right code. Every library using I/O only has to be written once - what a novel concept! It's a pity it didn't happen in Rust.
async/await came out of C# (well at least the JS version of it).<p>There are a bunch of use cases for it outside of implementing concurrency in a single threaded runtime.<p>Pretty much every GUI toolkit I've ever used was single threaded event loop/GUI updates.<p>Green threads are a very controversial design choice that even JVM backed out of.
Yep and I loved when C# introduced it. I worked on a system in C# that predated async/await and had to use callbacks to make the asynchronous code work. It was a mess of overnested code and poor exception handling, since once the code did asynchronous work the call stack became disconnected from where the try-catches could take care of them. async/await allowed me to easily make the code read and function like equivalent synchronous code.
> async/await came out of C# (well at least the JS version of it).<p>Not sure if inspired by it, but async/await is just like Haskells do-notation, except specialized for one type: Promise/Future. A bit of a shame. Do-notation works for so many more types.<p>- for lists, it behaves like list-comprehensions.<p>- for Maybes it behaves like optional chaining.<p>- and much more...<p>All other languages pile on extra syntax sugar for that. It's really beautiful that such seemingly unrelated concepts have a common core.
> Green threads are a very controversial design choice that even JVM backed out of.<p>Did they? Project Loom has stabilized around Java 21, no?
Virtual Threads aren't quite the same as green threads (they don't block the OS thread) and they work extremely well now.
I stand corrected, I stopped keeping track of JVM years ago, was referring to initial green threads implementation.
JavaScript got async in 2017, Python in 2015, and C# in 2012. Python actually had a version of it in 2008 with Twisted's @inlineCallbacks decorator - you used yield instead of await, but the semantics were basically the same.
> And as for Rust - that's beyond inexplicable. Rust has green threads in the early days and abandoned them in favour of async / await.<p>There was a fair bit of time between the two, to the point I'm not sure the latter can be called much of a strong motivation for the former. Green threads were removed pre-1.0 by the end of 2014 [0], while work on async/await proper started around 2017/2018 [1].<p>In addition, I think the decision to remove green threads might be less inexplicable than you might otherwise expect if you consider how Rust's chosen niche changed pre-1.0. Off the top of my head no obligatory runtime and no FFI/embeddability penalties are the big ones.<p>> Rust now has a green thread implementation that fixes that mistake<p>As part of the runtime/stdlib or as a third-party library?<p>[0]: <a href="https://github.com/rust-lang/rust/issues/17325" rel="nofollow">https://github.com/rust-lang/rust/issues/17325</a><p>[1]: <a href="https://without.boats/blog/why-async-rust/" rel="nofollow">https://without.boats/blog/why-async-rust/</a>
> Rust has green threads in the early days and abandoned them in favour of async / await. Granted the original green thread implementation needed a bit of refinement - making every low level choose between event driven and blocking on every invocation was a mistake.<p>That's a mischaraterization. They were abandoned because having green threads introduces non-trivial runtime. It means Rust can't run on egzotic architectures.<p>> It sounds like Zig with its pluggable I/O interface finally got it right<p>That remains to be seen. It looks good, with emphasis on looks. Who knows what interesting design constraints and limitation that entails.<p>Looking at comptime, which is touted as Zig's mega feature, it does come at expense of a more strictly typed system.
What if process_the_character takes multiple seconds waiting on a network request?
Function colouring, deadlocks, silent exception swallowing, &c aren’t introduced by the higher levels, they are present in the earlier techniques too.
Function coloring also only applies to a few select languages. If your runtime allows you can call an async function from a sync function by pausing execution of the current function/thread whenever you're waiting for some async op.<p>Libraries like Tokio (mentioned in the article) have support for this built-in. Goroutines sidestep the issue completely. C# Tasks are batteries included in that regard. In fact function colors aren't an issue in most languages that have async/await. JavaScript is the odd one out, mostly due to being single-threaded. Can't really be made to work in a clean way in existing JS engines.
“Function coloring” is an imaginary issue in the first place. Or rather it's a real phenomenon, but absolutely not limited to async and people don't seem to care about it at all except when talking about async.<p>Take Rust: you return <i>`Result<T,E>`</i>, you are <i>coloring</i> your function the same way as you are when using <i>`async`</i>. Same for Option. Errors as return values in Go: again, function coloring.<p>One of your nested function starts taking a "serverUrl" input parameter instead of reading an environment variable: you've colored your function and you now need to color the entire call stack (taking the url parameter themselves).<p>All of them are exactly as annoying, as you need to rewrite the entire call stack's function signature to accommodate for the change, but somehow people obsess about async in particular as if it was something special.<p>It's not special, it's just the reflection that something can either be explicit and require changing many function signatures at once when making a change, or be implicit (with threads, exceptions or global variables) which is less work, but less explicit in the code, and often more brittle.
Function coloring does not mean that functions take parameters and have return values. <i>Result<T,E></i> is not a color. You can call a function that returns a Result from any other function. Errors as return values do not color a function, they're just return values.<p>Async functions are colored because they force a change in the <i>rest</i> of the call stack, not just the caller. If you have a function nested ten levels deep and it calls a function that returns a Result, and you change that function to no longer return a result because it lost all its error cases, you only have to change the direct callers. If you are ten layers deep in a stack of synchronous functions and suddenly need to make an asynchronous call, the type signature of <i>every individual function in the stack</i> has to change.<p>You might say "well, if I'm ten layers deep in stack of functions that don't return errors and have to make a call that returns the error, well now I have to change the entire stack of functions to return the error", but that's not true. The type change from sync to async is <i>forced</i>. The error is not. You could just discard it. You could handle it somehow in one of the intervening calls and terminate the propagation of the type signature changes half way up. The caller might log the error and then fail to propogate it upwards for any number of reasons. You aren't being <i>forced</i> to this change by the type system. You may be forced to change by the rest of the software engineering situation, but that's not a "color".<p>For similar reasons, the article is incorrect about Go's "context.Context" being a coloration. It's just a function parameter like anything else. If you're ten layers deep into non-Context-using code and you need to call a function that takes a context, you can just pass it one with context.Background() that does nothing context-relevant. You may, for other software engineering reasons, <i>choose</i> to poke that use of a context up the stack to the rest of the functions. It's probably a good idea. But you're not being <i>forced</i> to by the type system.<p>"Coloration" is when you have a change to a function that doesn't just change the way it interacts with the functions that directly call it. It's when the changes <i>forcibly</i> propagate up the entire call stack. Not just when it may be a good idea for other reasons but when the language <i>forces</i> the changes.<p>It is not, in the maximally general sense, limited to async. It's just that sync/async is the only such color that most languages in common use expose.
You can still use a function that returns result in a function that uses option.<p>And result and option usually mean something else. Option is a value or none. None doesn't necessarily means the function failed. Result is the value or an error message. You can have result<option, error><p>That's different then async where you can call the other type.
[dead]
I wish the “Function coloring” meme died. It made sense in the context of the original blog post (which was about callback hell, hence the “4. Red functions are more painful to call” section un the original blog post), but doesn't make sense in the context of async/await. There's literally nothing special with async, it's just an effect among many others.<p>As soon as you start using function arguments instead of using a global variable, you are coloring your function in the exact same way. Yet I don't think anyone would make the case that we should stop using function arguments and use global variables instead…
I think the lesson is to be careful about introducing incompatibility via the type system. When you introduce distinctions, you reduce compatibility. Often that’s deliberate (two functions <i>shouldn’t</i> be interchangeable because it introduces a bug) but the result is lots of incompatible code, and, often, duplicate code.<p>Effects are another way of making functions incompatible, for better or worse. It can be done badly. Java fell into that trap with checked exceptions. They meant well, but it resulted in fragmentation.<p>Sometimes it’s worth making an effort to make functions more compatible by standardizing types. By convention, all functions in Go that return an error use the same type. It gives you less information about what errors can actually happen, but that means the implementation of a function can be modified to return a new error without breaking callers.<p>Another example is standardizing on a string type. There are multiple ways strings can be implemented, but standardization is more important.
You can also use type inference with union types like ZIO. So you could e.g. return a Result where the error type is `DatabaseError | InvalidBirthdayError`. If you're in an error monad anyway, and you add a new error type deep in the call stack, it can just infer itself into the union up the stack to wherever you want to handle it.
That will help locally, but for a published API or a callback function where you don't know the callers, it's still going to break people if you change a union type. It doesn't matter if it's inferred or not.
Async/await will be equivalent to parameters when they are first class and can be passed in as parameters. Language syntax and semantics are not equivalent and colored functions are colored by the syntax. Zig avoided colored functions by doing something very much like this.
[dead]
> This was bad enough that Node.js eventually changed unhandled rejections from a warning to a process crash, and browsers added unhandledrejection events. A feature designed to improve error handling managed to create an entirely new class of silent failures that didn’t exist with callbacks.<p>Java has this too.
The discussion around async await always focuses on asynchronous use-cases, but I see the biggest benefits when writing synchronous code. In JS, <i>not</i> having await in front of a statement means that nothing will interfere with your computation. This simplifies access to shared state without race conditions.<p>The other advantage is a rough classification in the type system. <i>Not</i> marking a function as async means that the author believes it can be run in a reasonable amount of time and is safe to run eg. on a UI main thread. In that sense, the propagation through the call hierarchy is a feature, not a bug.<p>I can see that maintaining multiple versions of a function is annoying for library authors, but on the other hand, functions like fs.readSync shouldn’t even exist. Other code could be running on this thread, so it's not acceptable to just freeze it arbitrarily.
I will agree - async rust on an operating system isn’t all that impressive - it’s a lot easier to just have well defined tasks and manually spawn threads to do the work.<p>However, in embedded rust async functions are amazing! Combine it with a scheduler like rtic or embassy, and now hardware abstractions are completely taken care of. Serial port? Just two layers of abstraction and you have a DMA system that shoves bytes out UART as fast as you can create them. And your terminal thread will only occupy as much time as it needs to generate the bytes and spit them out, no spin locking or waiting for a status register to report ready.
No mention of JVM.. which is a bit odd as recently is kinda solved this problem. Sure, not all use cases, but a lot.<p>It uses N:M threading model - where N virtual threads are mapped to M system threads and its all hidden away from you.<p>All the other languages just leak their abstractions to you, java quietly doesn't.<p>Sure, java is kinda ugly language, you can use a different JVM language, all good.<p>Don't get me wrong, love python, rust, dart etc, but JVM is nice for this.
Async ruined Rust for me, even though I write exactly the kind of highly concurrent servers to which it's supposed to be perfectly suited. It degrades API surfaces to the worst case :Send+Sync+'static because APIs have to be prepared to run on multithreaded executors, and this infects your other Rust types and APIs because each of these async edges is effectively a black hole for the borrow checker.<p>Don't get me started on how you need to move "blocking" work to separate thread pools, including any work that has the potential to take some CPU time, not even necessarily IO. I get it, but it's another significant papercut, and your tail latency can be destroyed if you missed even one CPU-bound algorithm.<p>These may have been the right choices for Rust specifically, but they impair quality of life way too much in the course of normal work. A few years ago, I had hope this would all trend down, but instead it seems to have asymptoted to a miserable plateau.
Is there hope that green threads could solve this?
> It degrades API surfaces to the worst case :Send+Sync+'static because APIs have to be prepared to run on multithreaded executors<p>This isn't true at all. The Send+Sync+'static is basically the limitation of tokio::spawn<p><a href="https://emschwartz.me/async-rust-can-be-a-pleasure-to-work-with-without-send-sync-static/" rel="nofollow">https://emschwartz.me/async-rust-can-be-a-pleasure-to-work-w...</a><p>Change the executor, and the bound changes.
this is dead true to me. I write systems code. Rust is supposed to be a systems language. Because I do work that is effectively always written as if it's in the kernel mode and distributed over the network, everything I do is async by default. And the ergonomics around async are just miserable, littered with half-finished implementations and definitions (i.e. async streams, async traits), and a motley bunch of libraries that are written to use particular modes of tokio that don't necessary play well together. its a giant bodge that would be excusable if that wasn't supposed to be part of the core applicability of the language. not to mention that the whole borrower business becomes largely useless, so you forgot to add Arc+Mutex, and Pin implicitly to your list of wrapper type signatures.<p>what bother me the most, is that aside from async, I _really_ do like the language and appreciate what its trying to do. otherwise I would just turn away from the whole mess. this just landed really badly.
I would really hate to work with a blue/red function system. I would have to label all my functions and get nothing in return. But, labelling my functions with some useful information that I care about, that can tell me interesting things about the function without me having to read the function itself and all the functions that it calls, I'd consider a win. I happen to care about whether my functions do IO or not, so the async label has been nothing short of a blessing.
Zig is just doing vtable-based effect programming. This is the way to go for far more than async, but it also needs aggressive compiler optimization to avoid actual runtime dispatch.
I like async and await.<p>I understand that some devs don’t want to learn async programming. It’s unintuitive and hard to learn.<p>On the other hand I feel like saying “go bloody learn async, it’s awesome and massively rewarding”.
Intuition is relative: when I first encountered unix-style synchronous, threaded IO, I found it awkward and difficult to reason about. I had grown up on the callback-driven classic Mac OS, where you never waited on the results of an IO call because that would freeze the UI; the asynchronous model felt like the normal and straightforward one.
> It’s unintuitive and hard to learn.<p>Funny, because it was supposed to be more intuitive than handling concurrency manually.
It is. A lot.<p>But concurrency is hard and there's so much you syntax can do about it.
It is a tool. Some tools make you more productive <i>after</i> you have learned how to use them.<p>I find it interesting how in software, I repeatedly hear people saying "I should not have to learn, it should all be intuitive". In every other field, it is a given that experts are experts because they learned first.
> I find it interesting how in software, I repeatedly hear people saying "I should not have to learn, it should all be intuitive". In every other field, it is a given that experts are experts because they learned first.<p>Other fields don't have the same ability to produce unlimited incidental complexity, and therefore not the same need to rein it in. But I don't think there's any field which (as a whole) doesn't value simplicity.
I feel like it's missing my point. Using a chainsaw is harder than using a manual saw, but if you need to cut many trees it's a lot more efficient to first learn how to use the chainsaw.<p>Now if you take the chainsaw without spending a second thinking about learning to use it, and start using it like a manual saw... no doubt you will find it worse, but that's the wrong way to approach a chainsaw.<p>And I am not saying that async is "strictly better" than all the alternatives (in many situations the chainsaw is inferior to alternatives). I am saying that it is a tool. In some situations, I find it easier to express what I want with async. In others, I find alternatives better. At the end of the day, I am the professional choosing which tool I use for the job.
Except you're hearing it from someone who doesn't have a problem handling state machines and epoll and manual thread management.
Right but how do you expose your state machine and epoll logic to callers? As a blocking function? As a function that accepts continuations and runs on its own thread? Or with no interface such that anyone who wants to interoperate with you has to modify your state machine?
And that was intuitive and easy to learn?
Frankly, async being non-intuitive does not imply that manual concurrency handling is less so; both are a PITA to do correctly.
Some come to async from callbacks and others from (green)threads.<p>If you come from callbacks it is (almost) purely an upgrade, from threads is it more mixed.
It IS intuitive.<p>After you’ve learned the paradigm and bedded it down with practice.
I can't follow that it's hard to learn and unintuitive
What's awesome or rewarding about it?<p>It forces programmers to learn completely different ways of doing things, makes the code harder to understand and reason about, purely in order to get better performance.<p>Which is <i>exactly</i> the wrong thing for language designers to do. Their goal should be to find better ways to get those performance gains.<p>And the designers of Go and Java did just that.
> It forces programmers to learn completely different ways of doing things, makes the code harder to understand and reason about, purely in order to get better performance.<p>Technically, promises/futures already did that in all of the mentioned languages. Async/await helped make it more user friendly, but the complexity was already there long before async/await arrived
What different way of doing things?<p>If I want sequential execution, I just call functions like in the synchronous case and append .await.
If I want parallel and/or concurrent execution, I spawn futures instead of threads and .await them.
If I want to use locks across await points, I use async locks, anything else?
Surely by section 7 well be talking (or have talked) about effect systems
It’s a slop alright. But it also missed the next mainstream iteration which is Java virtual threads / Goroutines. Those do away with coloring by attacking the root of the problem: that OS threads are expensive.<p>Sure, it comes with its own issues like large stacks (10k copy of nearly the same stack?) and I predict a memory coloring in the future (stack variables or even whole frames that can opt out from being copied across virtual threads).
Because all HN needed was another piece of AI slop incorrectly quoting “what color is your function”…<p>It's 2026 and I'm starting to hate the internet.
[dead]