This looks like yet another basic key value store.<p>Benchmarking is a complicated problem, but FoundationDB claims 300,000 reads per second on a single core. TigerBeetle claims 100k-500k TPS on... Some kind of hardware?<p><a href="https://apple.github.io/foundationdb/benchmarking.html" rel="nofollow">https://apple.github.io/foundationdb/benchmarking.html</a>
I think I need a deeper-dive into the "diagonal scaling" presented. From my understanding, this is actually no different from "industry decoupling" he disparages earlier in the presentation. There are even off-the-shelf libraries for LSMs backed by object storage like SlateDB.
I feel the Expression Problem neatly frames the "diagonal scaling" proposition; what system design choices will allow the architecture to scale vertically in what fashion, while also being able to scale what horizontally, <i>without</i> losing strict serialisability.<p>If we add a "vertical" capability, it cannot be at the cost of any existing "horizontal" capability, nor should doing so forfend any <i>future</i> "horizontal" capability. And vice-versa (adding horizontal capability should not mess with vertical ones). The point at which one will break the other is the theoretical design limit of the system.
TigerBeetle's Joran Greef is a teacher / explainer par excellence.
Ah I appreciate your kind words, Aditya!<p>This was a team effort: the object storage connector, the scale test, the visualization, the slides, even provisioning the hardware had its challenges!
Oh most certainly; I am remiss to have not included the group effort in my comment, particularly as a person surrounded by theatre and film making friends.<p>Still, for the same reason, I have some idea of <i>why</i> their productions turn out well (or not). Where "well" is "a story well told", not "successful" as in "did well at the box office". The <i>why</i> is usually one person who keeps asking the questions and making the decisions that take the story from imagination to imagination via screen or floor.<p>Something tells me your doubtlessly excellent "production team" (in film terms) will agree with my original comment :)
I have tried to use tiger beetle in production. haven't been successful yet.<p>nice stuff, multi master replication.<p>user API, super small.<p>doubts about how to do streaming backup.<p>after studying the API and doing some spike architectures I come to the conclusion (I may be wrong):<p>tiger beetle is awesome to keep the account balance. that's it.<p>because you pretty much get the transactions affecting and account and IIRC there was not a lot you can do about how to query them or use them.<p>also I was thinking it would be nice to have something like an account grouping other accounts to answer something like: how much money out user accounts have in this microsecond?<p>I think that was more or less about itm they have some special fields u128 to store ids to the transaction they represent into your actual system<p>and IIRC handle multi currency in different books<p>my conclusion was: I think I don't get it yet. I think I'm missing something. had to write a ruby client for it and build an UI to play with the API and do some transactions and see how it behaved. yet that was my conclusion<p>would be great to have an official UI client
The takeaway is that you can just buy a single beefy server instead of using kubernetes or whatever.
To a first approximation, yes. But, <i>why</i>? And for up to how many hundred terabytes of data can you get away with the single beefy server? Provided you make what design choices?<p>Which leads to the real takeaway which is "Tiger Style": <a href="https://tigerstyle.dev/" rel="nofollow">https://tigerstyle.dev/</a> which I am partial to, along with Rich Hickey's "Hammock Driven Development" <a href="https://www.youtube.com/watch?v=f84n5oFoZBc" rel="nofollow">https://www.youtube.com/watch?v=f84n5oFoZBc</a><p>"Tiger on Hammock" will absolutely smoke the competition.<p>(edit: add links)
> But, why?<p>To keep things simple. My current company is running multiple instances of back-end services for absolutely no fucking reason, and I had to fix numerous race condition bugs for them. I had an interview with a startup where, after I asked why they were using distributed DynamoDB locks in a monolith app with only a single instance running, the person said "it works for us" and got defensive. Later they told me I wasn't experienced enough. I am so frustrated that there appears to be zero basic engineering rigor anywhere I can find nowadays.<p>> And for up to how many hundred terabytes of data can you get away with the single beefy server?<p>Do you even need to store many hundred terabytes of data? I have never encountered a scenario in my career (admittedly not very long so far) where there was a need to store even one terabyte of data. But in case of TigerBeetle, from skimming through the video, it appears they offload the main bulk of data to a "remote storage."
And your beefy server goes down what do you do? Where do you think those object storage live exactly?<p>Kubernetes is not just for scaling, it's a way to standardize all ops.
> <i>And your beefy server goes down what do you do?</i><p>Boot it up again. You'll still have higher availability than AWS, GitHub, OpenAI, Anthropic, and many others.<p>> <i>Where do you think those object storage live exactly?</i><p>On a RAID5 array with hot-swappable disks, of course.<p>(Edit to add: this is just a comment on Kubernetes being invoked whenever someone talks about scalability; I have massive respect for what the TigerBeetle folks are doing)
"Always has been"