Proxies are good if you pay API prices.<p>For subscriptions, it is easier to collect the JSONL files from ~/.claude/projects/ and similar directories. They are better organized (e.g., subagents) than what the proxy sees.<p>However, the best value comes from collecting agent sessions across the whole organization. You'll need to scrub credentials and handle security.
I'd love to do something like this, or use API keys, but I still don't understand how people justify the cost of using API keys vs. subscription.<p>I tested it once, I'm in on the 20x claude max ($200/month) and I either max my weekly limit every week or come really close. But it only took 2-3 days to hit $200 spend with api keys.
I'm with you there. I'm on expensive plans with everyone because I run out of tokens and rotate/use the models that are best in their niche...<p>CLI Model Pct Remaining Reset Date
claude all models 15% 3d
claude Fable 50% 3d
grok all models 0% 4d
agy Gemini 36% 5d
agy Claude and GPT models 0% 5d
codex codex 0% 6d<p>And I also just used two of my three get-another-week-free tokens for codex.<p>If you've got an agentic swarm, or are working on multiple big projects at once, or have agents parsing data... you burn through tokens fast. The last four days in API would have cost me well over $3k.
The current crop of flash models are more comparable to sonnet than opus but they are very affordable.<p>The reality is that the subscriptions are heavily subsidized. Pretty much every sub that's not run by one of the money-burning frontier labs has had to drop it or dramatically increase prices and lower limits in the last few months. Cheap tokens are an acquisition tool, not a real product. It sounds dramatic but it really is the drug dealer sales funnel.<p>At some point they're going to drop the real prices in your lab and you're going to have to figure out how to pay that much or pay someone else a lot less. Better to get a handle on it now.
> At some point they're going to drop the real prices in your lab and you're going to have to figure out how to pay that much or pay someone else a lot less.<p>This only applies under the assumption that everyone tokenmaxxes their subscription.<p>I have visibility across a few different Claude team subscriptions. I'd say roughly 10% of people max their subscriptions.<p>I think I've only seen one organization where the under-used subs don't more than pay for the tokenmaxxed ones.<p>And that is ignoring the 90% margin that API inference runs at.
It's not obvious the subsidization will end. Loss leaders are a thing. Most enterprise at this point are paying API pricing<p>To better predict what they'll do with it we'd need to know the proportion of token usage via API vs subscription. Keeping an affordable, slightly unprofitable subscription to get people in isn't a bad idea. They also get marketing benefits from the word of mouth
In the optimistic case you describe, what smart customers want to watch out for is being lured into vibing your projects into such byzantine slop-complexity that subscription plans and trailing models are no longer sufficient to make progress on them.<p>Vibe-coders talk about surrendering understanding to the models, trusting that there will always be a model able to overcome and work through any technical debt they happen to accumulate, but the danger that still lurks there (at least for the naive) is that you'll need increasingly large contexts, increasingly sophisticated models, and increasingly large token spends just to keep that ball rolling.<p>Between profitable enterprise accounts being ineligible for subscription, eager but busy people paying for underutilized subscriptions, and naive vibe-coders falling into their pit trap, you can indeed imagine subsidized subscription accounts being available for a long time and remaining an effective value hack for people careful enough not to fall into the pit trap. But only insofar as they are that careful.
Also they can always find another way to make money. Make subscription users product and sell some analytics like "show most trending project being built this week".
Also, some subset of developers on subscriptions are going to develop products that use the API, and some of those products are going to be successful.
> At some point they're going to drop the real prices in your lab<p>The real price for tokens at a specific level of intelligence is dropping rapidly. Between models improving and new hardware getting faster, I think prices will be going down, not up.<p>> Better to get a handle on it now.<p>I experiment with a lot of providers and models. I strongly disagree that anyone needs to start migrating now in preparation for prices maybe going up in the future.<p>It doesn’t make sense. Deal with a price increase when it happens, if it does.<p>Switching models and providers isn’t some huge lift. I can jump to a new model or even hardness and get a feel for it in a day or two. It’s not some huge undertaking that requires preparation and planning.<p>If prices go up, try something at that point in time. Don’t worry about it until then.
When Claude Code subscriptions really started to catch on, there was a popular idea on HN that Anthropic's end game was that basically every dev adjacent role would need to pay for a $200/mo coding plan.<p>Then, both Anthropic and OpenAI in lockstep developed a enterprise pricing structure that openly coerced large businesses to pay API prices where it was feasible to spend $200/day rather than the comparatively cheap $200/mo.<p>Even if token pricing relative to intelligence does trend down over time, that doesn't mean the pricing structure will actually translate to lower costs, particularly in the cash cow B2B market.
>At some point they're going to drop the real prices in your lab and you're going to have to figure out how to pay that much or pay someone else a lot less. Better to get a handle on it now.<p>So just wait until that happens? What's the point of scrimping and saving now? Not to mention you're paying for a premium in the meantime.
(meta commentary)
really gnarly word choice: "drop" in "they're going to drop the real prices in your lab"?<p>I took your meaning as "they're gonna drop [a bad surprise on you by raising] the real prices"... so at least it's not quite as tortured as the ambiguous "support for version N dropped today" (where you often can't even infer the direction from context).<p>this isn't just a tangent, I'm having trouble understanding the point you're trying to make, which hinges entirely on deciding if you meant "drop" literally
They are betting on people don't max out their subscription.<p>Someone who pays $20 a month may only use 10% of their max budget.
Personally at home I use the free github copilot and then openrouter when that runs out. Last month I used $0.90 of openrouter. Zero so far this month. When my bill starts hitting double digits I'll consider options.<p>Different matter at work but even there we are paying for API costs and have a monthly allowance (starts at $200/month but increases fairly easy to get)
This is true for pretty much every fixed price subscription service in existence
I honestly feel a bit worried Anthropic/OpenAI are going to drop me since I consistently hit my limits
The drug dealer gives you a free hit because it is profitable for him in the long term for you to be addicted to his product, not because he is your friend.
It's weird to me how often this analogy gets used because:<p>1. In most cases, like this one, the product isn't heroin.<p>2. Drug dealers don't actually do that.
Are you sure the product isn't heroin? If we take it away, there's a bunch of people around here at least that will definitely experience withdrawal.
What? (1) Yes.. it's never that [stupid economics, they don't want their purchasers to die.. might as well sell fentanyl], (2) it's always that [hardly any way to propose alternatives.. within the scope of not killing the customer]
Another way to look at it is they are running a business. It's not even close to gambling for example.
You might not have a choice. For example, if you want to get a guarantee from Anthropic that they don't train bases on your prompts, you will need an enterprise deal, which is pay-by-token.
I’m not sure what people are doing with these subscriptions to feel the need to monitor what’s going on this closely. A $100 OpenAI subscription feels effectively infinite for my usage at home
Some people are fine with an ISP-provided router that just works, and some would rather run pfSense, if even just to look at the internals.
Cache TTL will get you anyway. For Claude subscriptions, subagents still have a 5m TTL and before I discovered that, their constant waits for ~10 min tests or similar tasks would destroy my 5h usage. And for my API subscription at work, Claude has a 5m TTL for all stuff, and that's nowhere <i>near</i> long enough. You have to be monitoring this unless you want to absolutely torch your company's budget.
One man’s fun is another man’s “idk what any of these words mean, I pay $20/mo for some company to abstract this complexity away”<p>For clarity, I do like blog posts like these.
Tailscale makes Aperture available for free, and while it's not something you own, it is in your tailnet, so at least you have a certain amount of control over how things access it.
It gives a nice view of what agents are doing, with lots of configurability and extensions, and the team behind it has been updating pretty steadily.<p>I happen to be working on an extension to Aperture, so I have my own reason to promote it. But I do think it's a nice easy solution, and I promoted it to people before I was extending it.
Want to shamelessly self promote my self hosted proxy, supports subs and API usage.<p>Primaily designed as a local LLM ops project, but will definitely do proxying for claude-code, opencode, codex, etc<p><a href="https://llama-dash.dev" rel="nofollow">https://llama-dash.dev</a>
Yup. Setting up liteLLMProxy on some servers as we speak and will start playing around.<p>I can't orchestrate agents in sandboxes without it.<p>1. It can issue virtual keys, so provider API keys never go near the sandbox<p>2. I can see everything<p>3. Guardrails<p>And much more<p>Looking forward to it.
I agree its so important to proxy and capture all the data. I've been very happy with bifrost as a proxy. It's a quality piece of software, unlike LiteLLM. You also get this one interface to access all your models, across all services, including self-hosted, STT models, everything. There are ways to use subscriptions with proxies too, but they are discouraged because of grey-hat uses..
Yeah you should run an LLM proxy, but imagine a world where LiteLLM and all the other LLM shovels weren't buggy vibecoded messes. And with the velocity of updates I feel there has to be vulnerabilities there. Or am I just holding it wrong? Anyone got a good alternative?
Bifrost has been decent, despite being partially shareware
Imagine complaining about “buggy vibecoded messes” that you want to use to produce your own buggy vibe coded mess.
<a href="https://github.com/gumieri/nenya" rel="nofollow">https://github.com/gumieri/nenya</a><p>my own ai gateway/proxy
I don't do this but I do have a system that imports all my conversations with AI (claude code, chatgpt, claude app, etc.) to a storage system I call Chatacombs: a unified storage of most conversations I've had with AI, as detailed as I can easily get.
This is the best one I found <a href="https://github.com/kenn-io/agentsview" rel="nofollow">https://github.com/kenn-io/agentsview</a><p>It is a not proxy. It can clearly show whats going on across all agents. The proxy's are generally inconvenient and yet another point of failure. I think the author got to the resort option quickly
OpenRouter does all this for you. It's pretty nice. Although for apps I use another server to lock down the requests even more and prevent the client from seeing my API key.
Some don't need to. Not yet anyway. There's config, there's control, and then there's setup-theatre as a hobby.
I like the content but I have to say the Mark Manson writing style is off putting.
Seeing some good stuff in the comments, but how are people doing this?
By running my own agent harness that provides detailed API metrics in the GUI and logs very verbosely.
Well that was one of the reasons why i build my own agent/harness - to have actual full observability of what it is doing and why it is doing what it is doing.<p>Don't really understand why other agents to give those insights but well found my solution.
What is the use case for these sorts of things?
Why not just grab it from the kernel?
I am boggled how the first step of an llm after you make an orchestrator isn't some kind of proxy that records your personal internet surfing to be used by your personal AI later. Every college should be running in house AI with their own in house search engines and in house any service. Don't go to any college that wants you to use chatgpt or other corporate nonsense. If they can't make local LLMs work, they are not a real college.
> Own your infrastructure<p>...proceeds to use codex.<p>With just a 12gb VRAM GPU + 64gb of CPU RAM, and you can actually own your own infrastructure. I can't wait for hardware prices to fall so that becomes even more accessible.
Does anyone here use LiteLLM for this? How well does it do? Can I use it with my Claude subscription?
<a href="https://docs.litellm.ai/docs/tutorials/claude_code_max_subscription">https://docs.litellm.ai/docs/tutorials/claude_code_max_subsc...</a>
I use GoModel and its amazing. My favorite features are virtual model, the support for arbitrary request labels, and the nice dashboads / ui. Creator has more sophisticated report generation on the near term roadmap. He's super responsive to feedback and ships fast. (except for his recent trip to the US west coast to meet founders and investors, he's EU based)<p>You can use and control all.your subscriptions or aoikeys through a gateway like this, so you don't get pwned like what just happened with OpenCode (or an OpenRouter like vendor). GoModel has some ambitions like this, but is open source so you can run it on you own infra, with plans for self hosted enterprise as well (there are a few features already like SSO). We are rolling out the free version at work right now.<p>I run GoModel at home and have hyper detailed history of every agent interaction, across models running on my sparks and OpenCode sub. You can call me a committed superfan of this project.<p><a href="https://github.com/ENTERPILOT/GOModel" rel="nofollow">https://github.com/ENTERPILOT/GOModel</a>
[flagged]
[flagged]
[dead]
[flagged]
Easy. I don’t use any of the AI shit.