Llms

Written by Opus, reviewed and edited by me. TL;DR: I built a minimal Qwen3-0.6B in pure PyTorch that runs on Apple Silicon, with a live chat UI that shows KV cache memory and per-token latency side by side. Toggle the cache off and watch attention compute go quadratic in real time. Code: github.com/ricklamers/kvcache-exploration. I kept reading explanations of KV cache that all said roughly the same thing: it trades memory for compute, stores keys and values from prior tokens so attention doesn’t redo O(n²) work each step, makes decoding fast. I’ve heard it, you’ve heard it. But “KV cache saves time” is pedagogically thin. It doesn’t tell you how much time, at what memory cost, or what happens when you just don’t have one. ...

So I kept reading rumors that Gemini 3.0 is accessible through Google AI Studio through A/B testing and the SVGs folks were posting (of Xbox controllers in particular) made me think that they might be right. Gemini 3.0 is one of the most anticipated releases in AI at the moment because of the expected advances in coding performance. Evaluating models is a difficult task, but surprisingly the SVG generation task seems to be a very efficient proxy for gauging model quality as @simonw has shown us using his “pelican riding a bicycle” test. ...

In ML it’s common to measure generalization performance as the delta between the training, validation and test dataset. Training data is what your model actively optimizes against using some optimization procedure (e.g. SGD or conjugate gradient or quadratic programming). The validation data is used to get an unbiased estimate of the performance of the model on unseen data. Since the process of selection (hyper parameter selection, stopping criteria) invariably overfits the model to the validation data at the very end we get an estimate of model performance through the test dataset that should not be used for the process of training a model. ...

tl;dr: github.com/ricklamers/gpt-code-ui and to run it pip install gpt-code-ui && gptcode Alright, I’ll cut right to the chase. I’ve built GPT-Code UI because OpenAI couldn’t be bothered to give me access to their new fancy ChatGPT Code Interpreter. And most likely, you don’t have access either. What is ChatGPT Code Interpreter? Well, it’s basically a REPL (read–eval–print loop) that’s integrated with the coding abilities of ChatGPT. You can do really neat things with it like giving it a file (e.g. a CSV file) after which you can ask it to convert the file or extract some information from the file or even use the file as input for a calculation! If you want a longer explanation you can watch this YouTube video. ...