13 comments

  • zahlman6 minutes ago
    &gt; And computers are big, too. You can buy a 1000MHz machine with 2 gigabytes of RAM and an 1000Mbit&#x2F;sec Ethernet card for $1200 or so. Let&#x27;s see - at 20000 clients, that&#x27;s 50KHz, 100Kbytes, and 50Kbits&#x2F;sec per client. It shouldn&#x27;t take any more horsepower than that to take four kilobytes from the disk and send them to the network once a second for each of twenty thousand clients. (That works out to $0.08 per client, by the way. Those $100&#x2F;client licensing fees some operating systems charge are starting to look a little heavy!) So hardware is no longer the bottleneck.<p>It seems to me that there are far fewer problems nowadays with trying to figure out how to serve a tiny bit of data to many people with those kinds of resources, and more problems with <i>understanding how to make a tiny bit of data relevant</i>.<p>It still absolutely can be. We&#x27;ve just lost touch.
  • hoppp5 hours ago
    We solved it 2 decades ago but then decided to use javascript on the server ...
    • mifreewil3 hours ago
      Node.js uses libuv, which implements strategy 2. mentioned on the linked webpage.<p>&quot;libuv is a multi-platform C library that provides support for asynchronous I&#x2F;O based on event loops. It supports epoll(4), kqueue(2)&quot;
      • hinkley50 minutes ago
        Libuv now supports io_uring but I’m fuzzy on how broadly nodejs is applying that fact. It seems to be a function by function migration with lots of rollbacks.
      • marcosdumay3 hours ago
        Except that it wastes 2 or 3 orders of magnitude in performance and polls all the connections from a single OS thread, locking everything if it has to do extra work on any of them.<p>Picking the correct theoretical architecture can&#x27;t save you if you bog down on every practical decision.
        • mifreewil3 hours ago
          I&#x27;m sure there is plenty of data&#x2F;benchmarks out there and I&#x27;ll let that speak for itself, but I&#x27;ll just point out that there are 2 built-in core modules in Node.js, worker_threads (threads) and cluster (processes) which are very easy to bolt on to an existing plain http app.
          • IgorPartola2 hours ago
            So think of it this way: you want to avoid calling malloc() to increase performance. JavaScript does not have the semantics to avoid this. You also want to avoid looping. JavaScript does not have the semantics to avoid it.<p>If you haven’t had experience with actual performant code JS can seem fast. But it’s is a Huffy bike compared to a Kawasaki H2. Sure it is better than a kid’s trike but it is not a performance system by any stretch of the imagination. You use JS for convenience, not performance.
            • gbuk20131 hour ago
              IIRC V8 actually does some tricks under the hood to avoid malocs which is why Node.js can be be unexpectedly fast (I saw some benchmarks where it was only 4x of equivalent C code) - for example it recycles objects of the same shape (which is why it is beneficial not to modify object structure in hot code paths).
              • hinkley46 minutes ago
                Hidden classes is a whoooole thing. I’ve switched several projects to Maps for lookup tables so as not to poke that bear. Storage is the unhappy path for this.
            • winrid2 hours ago
              (to be fair the memory manager reuses memory, so it&#x27;s not calling out to malloc all the time, but yes a manually-managed impl. will be much more efficient)
              • IgorPartola1 hour ago
                Whichever way you manage memory, it is overhead. But the main problem is the language does not have zero copy semantics so lots of things trigger a memcpy(). But if you also need to call malloc() or even worse if you have to make syscalls you are hosed. Syscalls aren’t just functions, they require a whole lot of orchestration to make happen.<p>JavaScript engines also are also JITted which is better than a straight interpreter but except microbenchmarks worse than compiled code.<p>I use it for nearly all my projects. It is fine for most UI stuff and is OK for some server stuff (though Python is superior in every way). But would never want to replace something like nginx with a JavaScript based web server.
            • jauntywundrkind2 hours ago
              What a sad miserable cult it is, to be obsessed with performance, and to so desperately need to beat up on a language. I don&#x27;t know if people spewing this shit are aware of how dull their radical extremism is?<p>You really don&#x27;t have to go far down the techempower benchmarks to get to JS. Losing let&#x27;s just say 33% performance versus an extremely tuned much more minimalist framework on what is practically a micro benchmark is far from the savage fall from civilization &amp; decadence of man, deserving far less scorn and shame than what the cult of js hate fuels their fires on.<p>I could go on about how JS has benefitted from having incredibly work poured into it, because it is the most popular runtime on the planet, because it&#x27;s everywhere, because there was a hot war for companies to try to beat each other on making <i>their</i> js runtime good (one of the only langs with many runtimes is <i>interesting</i>). It&#x27;s a bit excessive &amp; maybe it should have been a more deserving language perhaps (we can debate endlessly), but man... It just doesn&#x27;t matter. Stressing out, being so mean and nasty (huffy vs Kawasaki, @hoppp&#x27;s even worse top-voted half sentence snark takedown), trying to impress upon people this image that it&#x27;s all just so bad: I think there&#x27;s such a ridiculous off kilter tilting way too hard, and far less of it is about good reasons and valid concerns, and so so so much of it is this bandwagon of negative energy, is radical overconcern.<p>Like most tools &amp; languages, <i>it&#x27;s what you do with it.</i> With JS, we have a problem that (mainstream) software hadn&#x27;t faced before, which is client server architecture, that the client might be a cruddy cheap phone somewhere and&#x2F;or on a dodgy link. We are trying to build experiences that work across systems, with significant user perceived latency sometimes. And so data &amp; systems <i>architecture</i> matters a lot. Trying to get keep the client primed with up to date data that it needs to render, doing client work without blocking&#x2F;while maintaining user responsiveness, are hard fun multi-threaded (webworker) challenges, for those folks that care.<p>And those challenges aren&#x27;t unique to js. Other languages have similar challenges. Trying to multithread a win32 UI to avoid blocking also was a bit of a nightmare, working off main thread. Doing data sync is a nightmare. There&#x27;s so many ways to get stuff wrong. And I think a lot of the js code out there does get it wrong. But we experience hundreds or thousands of websites a week, and crucial tools we use that are js client-server are badly architected. I <i>sympathize</i> with why js has such a bad rap. To me it usually seems like architectural app design issues, that companies are too busy building features to really consider the core, to establish data architectures that don&#x27;t block or lag. And that&#x27;s not a specific js problem.<p>There are faster systems yes, but man, the energy being poured into blaming the worst of the world on JS seem ridiculous to me, like such a sad drag, that avoids any interest or fascination on what is so so interesting &amp; so worthy. The language is the least remarkable part of the equation, practically doesn&#x27;t matter, and the marginal performance levels are (with some exception for very specific cases) almost never a remotely critical factor. Just so so so tired, knowing such enormous and pointless frivolous scorn and disdain is going to overwhelm all conversations, going to take over every thread forever, when it matters so so little, is so very rarely a major determinant.<p>JS does not <i>have</i> to be the thought terminating cliche to every thread ever (and humbly, I&#x27;d assert it doesn&#x27;t deserve that false conviction either. But either way!).
              • hinkley41 minutes ago
                I was a Java programmer out of school. My first specialization became performance. It’s the same problem with JavaScript. There’s at least an order of magnitude speed difference between using your head and just vomiting code, and probably more like a factor of forty. Then use some of the time you saved to do deep work to hit an even two orders of magnitude and it can be fast. If you’re not an idiot that is.<p>bcantril has an anecdote about porting some code to rust and it was way faster than he expected. After digging he found it was using btrees under the hood, which was a big part of the boost. B-trees being a giant pain in the ass to write in the donor language but a solved problem in Rust. It’s a bit like that. You move the effort around and it evens out.
              • mystraline1 hour ago
                Tl;dr.<p>Care to summarize?
                • jauntywundrkind1 hour ago
                  Architecture is far more important than runtime speed. (People are so easily swayed by &quot;JS SUCKS LOL&quot; because of experiences with terrible &amp; careless client-server architectures, far more than js itself being &quot;slow&quot;.)<p>The people ripping into js suck up the interesting energies, and bring nothing of value.
                  • IgorPartola1 hour ago
                    If we are discussing C10K we are by definition discussing performance. JavaScript does not enter this conversation any more than BASIC. Yes of course architecture matters. Nobody has been arguing otherwise. But the point is that if you take the best architecture and implement it in the best available JS environment you are still nowhere close to the same architecture implemented in a systems language in terms of performance. You are welcome to your wishful thinking that you do not need to learn anything besides JavaScript when it comes to this conversation. But no matter how hard you argue it will continue being wishful thinking.<p>We are discussing tech where having a custom userland TCP stack is not just a viable option but nearly a requirement and you are talking about using a lighter JS framework. We are not having the same conversation. I highly recommend you get off Dunning-Kruger mountain by writing even a basic network server using a systems language to learn something new and interesting. It is more effort than flaming on a forum but much more instructive.
                    • jauntywundrkind49 minutes ago
                      Techempower isn&#x27;t perfect, but a 33% loss of speed is really not bad versus best of the best web stacks. Your attempt to belittle is the same tired heaping scorn, when the data shows otherwise. But it&#x27;s so cool to hate!<p>Libuv isn&#x27;t the most perfect networking stack in the planet, no. It could be improved. We could figure out new ways to get io-uring or some eBPF based or even dpdk bypass userland stack running. Being JS does not limit what you do for networking, in any way. At all. It adds some overhead to the techniques you choose, requires some glue layer. So yes, some cpu and memory efficiency losses. And we can debate whether thats 20% or 50% or more. Techempower shows it&#x27;s half an order of magnitude worse (10^0.5, 33%). Techempower is a decent proxy for what is available, what you get, without being extremely finicky.<p>Maybe that is the goal, maybe this really is a shoot for the moon effort where you abandon all existing work to get extreme performance, as c10k was, but that makes the entire lesson not actually useful or practical for almost everyone. And if you want to argue against techempower being a proxy for others, for doing extreme work &amp; what is possible at the limit, you have to consider a more extreme js networking than what comes out of box in node.js or Deno too, into a custom engine there too.<p>It&#x27;s serious sad cope to pretend like it&#x27;s totally different domains, that js is inherently never going to have good networking, that it can&#x27;t leverage the same networking techniques you are trying to vaunt over js. The language just isn&#x27;t that important, the orders of magnitude (&lt;&lt;1 according to the only data&#x2F;evidence in this thread) are not actually super significant.
                  • dilyevsky1 hour ago
                    Nothing about current js ecosystem screams good architecture it’s hacks on hacks and a culture of totally ignoring everything outside of your own little bubble. Reminds me of early 2000s javabeans scene
          • drogus1 hour ago
            Worker threads can&#x27;t handle I&#x2F;O, so a single process Node.js app will still have the connection limit much lower than languages where you can handle I&#x2F;O on multiple threads. Obviously, the second thing you mention, ie. multiple processes, &quot;solves&quot; this problem, but at a cost of running more than one process. In case of web apps it probably doesn&#x27;t matter too much (although it can hurt performance, especially if you cache stuff in memory), but there are things where it just isn&#x27;t a good trade-off.
          • hinkley48 minutes ago
            And I have confirmed to my own satisfaction that both PM2 and worker threads have their own libuv threads. Yes very common in node to run around one instance per core, give or take.
    • wmf3 hours ago
      Node.js is actually pretty good at C10K but it failed at multicore and C10M.
  • trueismywork2 days ago
    With nginx and 256 core Epycs, most single servers can easily do 200k requests per sec. Very few companies have more needs
    • tempest_3 hours ago
      This is how I feel about this industries fetishization of &quot;scalability&quot;.<p>A lot of software time is spent making something scalable when in 2025 I can probably run any site the bottom 99% of most visited sites on the internet on a couple machines and &lt; 40k capital.
      • Animats1 hour ago
        The sites that think they need huge numbers of small network interactions are probably collecting too much detailed data about user interaction. Like capturing cursor movement. That might be worth doing for 1% of users to find hot spots, but capturing it for all of them is wasteful.<p>A lot of analytic data is like that. If you captured it for 1% of users you&#x27;d find out what you needed to know at 1% of the cost.
      • tbrownaw3 hours ago
        &gt; <i>any site the bottom 99% of most visited sites on the internet</i><p>What % is the AWS console, and what counts as &quot;running&quot; it?
        • tempest_2 hours ago
          &gt; What % is the AWS console<p>0%<p>Prior to the recent RAM insanity(a big caveat I know) a 1u supermicro machine with 768GB some NVME storage and twin 32 core Epyc 9004s was ~12K USD. You can get 3 of those and and some redundant 10G network infra(people are literally throwing this out) for &lt; 40k. Then you just have to find a rack&#x2F;internet connection to put them in which would be a few hundred a month.<p>The reality is most sites don&#x27;t need multi region setups, they have very predicable load and 3 of those machines would be massive overkill for many. A lot of people like to think they will lose millions per second of down time, and some sites certainly do but most wont.<p>All of this of course would be using new stuff. If you wanted to use used stuff the most cost effective are the 5 year old second gen xeon scalables that are being dumped by cloud providers. Those are more than enough compute for most they are just really thirsty so you will pay with the power bill.<p>This of course is predicated on assumption you have the skill set to support these machines and that is increasingly becoming less common though as successful companies that started in the last 10 years are starting to do more &quot;hybrid cloud&quot; it is starting to come back around.
      • oblio2 hours ago
        Raw technical excellence doesn&#x27;t rake in billions, despite what IT people keep saying.<p>Otherwise Viaweb would be the shining star of 2025. Instead it&#x27;s a forgotten footnote on a path to programming with money (VC).
    • otterdude3 hours ago
      When people talk about a single server they&#x27;re not talking about one hunk of metal, they&#x27;re talking about 1 server process.<p>This article describes the 10k client connection problem, you should be handling 256K clients :)
      • marcosdumay3 hours ago
        When people talk about a single server they are pretty much talking about either a single physical box with a CPU inside or a VPS using a few processor threads.<p>When they say &quot;most companies can run in a single server, but do backups&quot; they usually mean the physical kind.
        • Maxatar1 hour ago
          The term is absolutely ambiguous and I know I&#x27;ve run into confusion in my own work due to the ambiguity. For the purpose of the C10K, server is intended to mean server process rather than hardware.
    • dilyevsky1 hour ago
      At the time this was written powerful backend server only had like 4 cores. Linux only started adopting SMP like that same year. Also CPU caches were tiny<p>Serving less than 1k qps per core is pretty underwhelming today, at such a high core count you&#x27;d likely hit OS limitations way before you&#x27;re bound by hardware
    • api3 hours ago
      I’m shocked that a 256 core Epyc can’t do millions of requests per second at a minimum. Is it limited by the net connection or is there still this much inefficiency?
      • zipy1243 hours ago
        It almost certainly can, even old intel systems with dual CPU 16 core systems could do 4 and a half million a second [1]. At a certain point network&#x2F;kernel bottlenecks become apparent though, rather than being compute limited.<p>[1]: <a href="https:&#x2F;&#x2F;www.amd.com&#x2F;content&#x2F;dam&#x2F;amd&#x2F;en&#x2F;documents&#x2F;products&#x2F;ethernet-adapters&#x2F;onload&#x2F;onload-nginx-plus-benchmark-results.pdf" rel="nofollow">https:&#x2F;&#x2F;www.amd.com&#x2F;content&#x2F;dam&#x2F;amd&#x2F;en&#x2F;documents&#x2F;products&#x2F;et...</a>
      • tempest_3 hours ago
        Like anything it really depends on what they are doing, if you wanted to just open and close a connection you might run into bottle necks in other parts of the stack before the CPU tops out but the real point is that yea, a single machine is going to be enough.
      • otterdude3 hours ago
        256 Processes x 10k clients (per the article) = 256K RPS
        • mrweasel1 hour ago
          Aren&#x27;t you of by a zero? 10K requests per core &#x2F; per second, time 256 cores is 2.560.000 RPS.<p>There&#x27;s probably going to be some overhead, but it seems like you could do 1M, if you have the bandwidth.
    • intothemild2 days ago
      I can&#x27;t tell if this is sarcasm or not.<p>They didn&#x27;t have this kind of compute back when the article was written. Which is the point in the article.
      • marcosdumay3 hours ago
        The article was written exactly because they had machines capable enough at the time. But the software worked against it on every level.
      • Maxatar1 hour ago
        I don&#x27;t see how you could have read the article and come to this conclusion. The first few sentences of the article even go into detail about how a cheap $1200 consumer grade computer should be able to handle 10,000 concurrent connections with ease. It&#x27;s literally the entire focus of the second paragraph.<p>2003 might seem like ancient history, but computers back then absolutely could handle 10,000 concurrent connections.
      • hinkley1 day ago
        In spring 2005 Azul introduced a 24 core machine tuned for Java. A couple years later they were at 48 and then jumped to an obscene 768 cores which seemed like such an imaginary number at the time that small companies didn’t really poke them to see what the prices were like. Like it was a typo.
        • fweimer3 hours ago
          Before clusters with fast interconnects were a thing, there were quite a few systems that had more than a thousand hardware threads: <a href="https:&#x2F;&#x2F;linuxdevices.org&#x2F;worlds-largest-single-kernel-linux-system&#x2F;" rel="nofollow">https:&#x2F;&#x2F;linuxdevices.org&#x2F;worlds-largest-single-kernel-linux-...</a><p>We&#x27;re slowly getting back to similarly-sized systems. IBM now has POWER systems with more than 1,500 threads (although I assume those are SMT8 configurations). This is a bit annoying because too many programs assume that the CPU mask fits into 128 bytes, which limits the CPU (hardware thread) count to 1,024. We fixed a few of these bugs twenty years ago, but as these systems fell out of use, similar problems are back.
          • alexjplant3 hours ago
            &gt; Driven by 1,024 Dual-Core Intel Itanium 2 processors, the new system will generate 13.1 TFLOPs (Teraflops, or trillions of calculations per second) of compute power.<p>This is equal to the combined single precision GPU and CPU horsepower of a modern MacBook [1]. Really makes you think about how resource-intensive even the simplest of modern software is...<p>[1] <a href="https:&#x2F;&#x2F;www.cpu-monkey.com&#x2F;en&#x2F;igpu-apple_m4_10_core" rel="nofollow">https:&#x2F;&#x2F;www.cpu-monkey.com&#x2F;en&#x2F;igpu-apple_m4_10_core</a>
            • fweimer2 hours ago
              Note that those 13.1 TFLOPs are FP64, which isn&#x27;t supported natively on the MacBook GPU. On the other hand, local&#x2F;per-node memory bandwidth is significantly higher on the MacBook. (Apparently, SGI Altix only had 8.5 to 12.8 GB&#x2F;s.) Total memory bandwidth on larger Altix systems was of course much higher due to the ridiculous node count. Access to remote memory on other nodes could be quite slow because it had to go through multiple router hops.
            • hinkley2 hours ago
              My Apple Watch can blow the doors off a Cray 1. It’s crazy.
      • trueismywork1 day ago
        Half serious. I guess what Iwas saying is that it is that kind of science which is still very useful but more to nginx developers themselves. And most users now dont have to worry about this anymore.<p>Should have prefixed my comment wirh &quot;nowadays&quot;
  • senko45 minutes ago
    The date (2003) is incorrect. The article itself refers to events from 2009, is listed at the bottom of the page as having been last updated in 2014, with a copyright notice spanning 2018, and a minor correction in 2019.
  • alwa3 hours ago
    Apparently this refers to making a web server able to serve 10,000 clients simultaneously.
    • IgorPartola3 hours ago
      It has been long enough that C10K is not in common software engineer vernacular anymore. There was a time when people did not trust async anything. This was also a time when PHP was much more dominant on the web, async database drivers were rare and unreliable, and you had to roll your own thread pools.
  • gnabgib2 days ago
    (2011 &#x2F; 2003)<p>Title: <i>The C10K problem</i><p>Popular in:<p>2014 (112 points, 55 comments) <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=7250432">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=7250432</a><p>2007 (13 points, 3 comments) <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=45603">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=45603</a>
  • _qua3 hours ago
    I personally think it&#x27;s more of a <a href="https:&#x2F;&#x2F;c25k.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;c25k.com&#x2F;</a> time of year.
  • amelius3 hours ago
    Yes. But it&#x27;s easy to reinvent it, with modern OSes and tools.
  • hinkley1 day ago
    I don’t think I even heard of C10K until around 2003.
  • readthenotes12 hours ago
    The internationally famous Unix Network Programming book. An icon, a shibboleth, a cynosure<p><a href="https:&#x2F;&#x2F;youtu.be&#x2F;hjjydz40rNI?si=F7aLOSkLqMzgh2-U" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;hjjydz40rNI?si=F7aLOSkLqMzgh2-U</a><p>(From Wayne&#x27;s World--how we knew the comedians had smart advisors)
  • draw_down3 hours ago
    [dead]