Would love to hear some real world experiences of o1 pro subscribers.
It doesn't feel like its performance represents how better it can benchmark against competitors. You'll barely feel it if you are using it for coding if you compare it to Claude.
Anyways, it's very good. Possibly the best. It can write really well and much better than GPT-4. Maybe it's just my own taste, but it looks less LLM-ish.
I'm just not sure if it's worth the $200/month. Claude is cheaper and you can use it on Cursor so well.
The tiered pricing for OpenAI offering looks very weak. They have the best product, but it isn't worth $200, sorry.
Is Claude significantly better than regular o1 (not pro)?
It feels about the same
I've done head-to-head tests vs. Claude 3.5 Sonnet on a variety of coding tasks and basic reasoning questions over the last week. It's not worth $200. Claude is better on most tasks.
For example, I pasted my HN profile and asked it to extract my email address in JSON format ({"email": ""}). I would expect any HN reader to be able to do this in seconds using an online ROT13 calculator. Certainly a "phd-level reasoning model" should be able to do this correctly. Claude outputs the correct answer in seconds. o1 Pro thought for two minutes and eventually output an email address that was invalid.
I'm a little surprised that there isn't more discussion about this on HN, as it seems highly relevant to the recent pivot from training-compute to inference-compute.
Yes, I have written code I could not have without the model. To me, 200$ is cheap. It's better than claude 3.5 by a wide margin if you know how to prompt for coding.
Can you please provide an example? With my experience the code it generates is mediocre at best. It works after a couple of nudges, but the approach to problem is quite average with lots of oversights.
I haven’t been able to replicate the claim that it’s “better than Claude 3.5 by a wide margin.” In my experience, they’re often quite comparable and deliver similar results about half the time.
Use an API key and something like TypingMind and you shouldn’t pay anywhere near 200 for chat like usage
If you're looking for a native alternative to TypingMind you can join the waitlist for something I'm working on: https://www.get-vox.com
[deleted]
no. the reasoning takes long and often still hallucinates.