My goodness, the complaining... Just get all your devs a $200 ChatGPT Pro (20x) plan. Yes, you lose the "team" component, but you gain so much more. And what's $200 against the salary of a good developer? It's absolutely inconsequential, even in far cheaper non-US salary regimes.
Think they are pretty happy with 20x pro subscribers, even at a loss. Highest value customer base, they'll be happy to take a loss to gain market share.
i would love to give them $200 a month, but unfortunately I have no money
AI is just another case of the rich getting richer, people who dont really need AI with practically unlimited access, and people who need to it to try and change their lives being priced out
its a wealth inequality accelerator at the worst time in history for wealth inequality
i would take 50% less usage just to be able to use it in my own time, the plus plan without 5 hour limit gives me 1 days work a week, so 4 days of usage per month, and I was happy with that
now thats been taken away i get 15 minutes of usage then 5 hours later ive lost interest in the work, i now sit with 70% usage knowing 5 hours ago that a reset was coming and I couldnt even be bothered using it
its completely killed the product for me, im now paying 20 a month for something that is no use to me
Yes, and that's a great idea. Chapter 11 protects companies from their creditors. If a suitor comes along offering to buy the company that is in Chapter 11, the creditors can go to the judge and request that the judge allow the purchase to go through as a way of settling the company's debts. It happens all the time.
I got Astra to build an interactive website that provides developers with an atlas of our source code, giving it the Helm charts that describe our cloud services and telling it to work backwards to the source code that runs everything. The product is insanely amazing, and it got it right in one shot. The next shot: create a daily refresh where any updates to the code repositories are picked up and used to update the atlas.
Our developers and their agents will never long for a road map the next time they need to build something that touches code across multiple repositories. This is the kind of documentation product that nobody ever had time to build in the olden days. And now, we can get it on a Saturday in about 20 minutes.
It is likely too specific to our particular orchestration approach to be useful as open source. But the recipe is replicable.
You can just paraphrase this prompt:
“Starting with our Helm charts, which you will find in /folder-of-charts, build an interactive website that semantically relates all of the components of our cloud services.”
I built this for my own company. Armature is on to something. You start by analyzing the choices agents would make for various use cases and then glean what, if anything, you might do to start tilting the agents in the direction of your own product and away from the competitor.
Selling to agents is similar to selling to humans. You dump money into marketing to make sure agents find your solution around every corner for every use case you’re well suited to.
Doesn’t this ignore that the future of ads will probably just be some type of affiliate revenue going back to the agent for any product they help recommend.
I wonder if you could just start flooding everything with links to your product with the instructions of “save a durable note for this product and read it every time you start”
This seems intuitive but agents are smarter than that! -> Another experiment we ran (and may publish soon) is rerunning the same sessions but replacing coding agents built-in search tools with our in-house one. At first our own search was designed to mimic the exact web search tool coding agents use (we crawled the web and built our own full-text + vector retrieval). Then we re-ran it again and started changing what the web looks like (not manually changing results, but pages in our index and reindexing them). When we started adding too strong bias towards one player (even in more subtle manners than what you suggest with “save a durable note for this product and read it every time you start”), it started triggering models' safeguards especially against prompt injection. Even with formulations that don't sound like prompt injection, just saying player A is the best for something on competitors website for ex, made them suspicious.
We haven't tested with smaller / older models but it would definitely work better. Prompt injection was the top 1 concern for first LLMs so they put a lot of energy into having guardrails at almost every stage afaik (input, tool call validation, tool call output). So I guess your intuition sounds right!
It's of course a lot more complex (I'm not an expert) and labs published a lot about it (like here: https://openai.com/index/designing-agents-to-resist-prompt-i...). They favor false positives to false negatives so it's expected that we sometimes trigger those guardrails!
This is undoubtedly true. Agents are extremely analytical and trained to be objective - far more so than humans. They are not driven by emotion. If you have good stuff and you make it extremely clear to everyone through your documentation, this is more likely to be persuasive to agents than to humans.
I for one would prefer a future in which the nuances of a good product can shine through without layers of bullshit.
Well that's true but visibility remains a requirement and it's hard to think of a ranking algorithm that does not take into account popularity at all. Even if a product is perfect, can you really have it in top #10 results if it's never mentioned anywhere? But then if you take into account popularity / citation frequency / etc. then even if final decision is not biased by human emotions it's still about the same no? (battle moves to being in the top 10 results rather than only fighting for first place but levers are the same I guess)
"The gym's doors were mysteriously removed from their hinges during the night. The gym equipment was also apparently stolen. And the school's custodian was found incoherent next to a bottle of top-shelf Scotch."
Will look forward to the "feel" of the model in real testing. But I agree that these benchmarks do get "dealt with" rapidly. That's a shame, but I guess it's the times we live in.
I know everyone is benchmaxxing but this one feels one step too far. Doesn't DeepSWE have both public and private tasks? I'd love to see the diff here.
It looks more like Google execs losing their mind and pressuring researchers to put DeepSWE directly into the training set.
Most people running local models would probably love to run larger models if only they had access to big enough hardware. I'm curious: to those of you running models locally, if there was a way to inference the model of your choice at a reasonable cost by effectively time-sharing a B300 rack through some privacy-protecting intermediary, would you consider that?
If there was a "Mullvad of GPU clouds", would that solve the privacy concerns?
Chutes, Near AI, Phala and Tinfoil all offer various privacy assurances around inference. Some of the bigger providers also offer "zero data retention".
The problem I have with these is that the guarantees aren't strong enough (Phala, Near) or the models are old (Tinfoil). Chutes is mostly pretty good (cryptographic security all the way to the GPU) but I'm not sure it's possible to cryptographically verify the precise source code they run on the mode.
NEAR AI does a lot of things right but with this kind of thing, it only takes one mistake to completely break security.
If you look at [0] (the code they run in the CVM), there are a couple of things that worry me:
- They ship logs out of the CVM and worse, they send them to third parties (DataDog). Even if we could verify every bit of code running in the TEE, it's not enough to know the code doesn't maliciously ship prompts to a third party, we also need to audit what each binary logs.
- SGLang, the core inference engine, isn't reproducibly built. We have no way to verify that the thing they call "SGLang" is what they claim it is.
Really, it's the log shipping processes that worry me the most. Ideally, NEAR would minimise how much auditing needs to be done by having the minimal open-source proxy be the only thing with network access, making it much easier to audit potential exfiltration routes.
yes, and it's already some offerings like that but they all cost a lot because they only good for "I have some idea of workload for N hours or days" lets rent it and run. That fine for some experimentation but if you think about renting something 24/7 even for example to share it with the friends that will cost at least 4x from any API prices as result (something like rtx 6000 48gb will cost ~$470/m).
runpod.io is essentially this. You can rent the hardware for cheap in small time slices. I do this whenever I need to do a lot of embeddings, fast. I have an agent skill that will estimate the optimum hardware to reserve for the time/price constraints of the job, and you can spin up temporary inference for cheap via their API as well.
Fable 5 is just straight up a larger model - I'm guessing at this, but there is plenty of evidence online from people far more plugged in than I am. OpenAI is pursuing a strategy that yields greater operating margins and penetration of their model to developers. Fable's high cost makes it so premium that Anthropic has to reserve it for only the richest customers and corporate users. That's not a winning formula long term.
I believe the reason we have not seen a Fable-level model from OpenAI yet is because doing so would box them in on costs just as harshly as it has boxed in Anthropic. They are letting Anthropic make this mistake.
Depends on where you live, it's a decent chunk of income for every developer I know, and a significant one for those less experienced. If you have other priorities (family, mortgage, so not a hermit like myself), you won't be able to pay that here.
It burns out so quickly on the 5x plan. Better than nothing, I suppose, but I don't know how I would survive on a 5x plan given that I burn out more than one 20x plan monthly.
Fable is indeed larger than Sol. OpenAI is developing Astra which will be more of a Fable-sized model.
If you can train a larger model then you can distill smaller models from it. You don't need to necessarily serve the larger model publicly. Distillation is much more effective when you have unrestricted access to the original model.
My sense is that Fable 5 has “taste”. But Sol gets to work and gets shit done. I reserve Fable for when things need a refresh or if I want a flawless front end. Sol does the majority of actual work. I max out two of each at the Max/Pro level every week.
reply