This couldn't be more perfectly timed .. I have an Unreal Engine game with both VT100 terminals (for running coding agents) and Z80 emulators, and a serial bridge that allows coding agents to program the CP/M machines:<p><a href="https://i.imgur.com/6TRe1NE.png" rel="nofollow">https://i.imgur.com/6TRe1NE.png</a><p>Thank you for posting! It's unbelievable how someone sometimes just drops something that fits right into what you're doing. However bizarre it seems.
If one would train an actual secret (e.g. a passphrase) into such a model, that a user would need to guess by asking the right questions. Could this secret be easily reverse engineered / inferred by having access to models weights - or would it be safe to assume that one could only get to the secret by asking the right questions?
I don’t know, but your question reminds me of this paper which seems to address it on a lower level: <a href="https://arxiv.org/abs/2204.06974" rel="nofollow">https://arxiv.org/abs/2204.06974</a><p>“Planting Undetectable Backdoors in Machine Learning Models”<p>“ … On the surface, such a backdoored classifier behaves normally, but in reality, the learner maintains a mechanism for changing the classification of any input, with only a slight perturbation. Importantly, without the appropriate "backdoor key", the mechanism is hidden and cannot be detected by any computationally-bounded observer. We demonstrate two frameworks for planting undetectable backdoors, with incomparable guarantees. …”
> this secret be easily reverse engineered / inferred by having access to models weights<p>It could with a network this small. More generally this falls under "interpretability."
Nice - that will fit on a Gameboy cartridge, though bank switching might make it super terrible to run. Each bank is only 16k. You can have a bunch of them, but you can only access one bank at a time (well, technically two - bank 0 is IIRC always accessible).
Awesome. I've just designed and built my own z80 computer, though right now it has 32kb ROM and 32kb RAM. This will definitely change on the next revision so I'll be sure to try it out.
interesting, i am wondering how far can it go if we remove some of these limitations but try to solve some extremely specific problem like generating regex based on user input? i know small models(270M range) can do that but can it be done in say < 10MB range?
Eliza's granddaughter.
This is super cool. Would love to see a Z80 simulator set up with these examples to play with!
In before AI companies buy up all the Z80s and raise the prices to new heights.
For future projects and/or for this project, there are many LLMs available more than good enough to generate that kind of synthetic data (20 Qs) with permissive terms of use. (So you don’t need to stress about breaking TOS / C&D etc)
An LLM in a .com file? Haha made my day
Imagine, this working on a Gameboy, in those days. Would've sounded like magic
I don’t think this could beat an ELIZA-style bot in how magical it feels, given the extreme terseness of its replies.
I love these thought experiments. Looking at the code size, it would have been possible for someone to come up with this back in the days, similar to the idea of a million monkeys on a typewriter eventually producing Shakespeare.
And would have lasted 3 minutes.<p>Speaking of - I remember my first digital camera (Fujitsu 1Mb resolution using SmartMedia)… it used so much power that you could take 20-30 photos and then needed to replace all 4 batteries lol