Startup Blog for First Time Entrepreneurs who Bootstrap

Is GPT-6.2 Worse? Testers Call It Unusable, Benchmarks Rank It Near the Top, and a One-Hour Test for Your Business

TL;DR: Is GPT-6.2 worse?

Is GPT-6.2 worse than the ChatGPT models you use now? Several well-known early testers say yes, and Matt Shumer called the models "essentially unusable". Benchmarks say the opposite: GPT-6.2 Astra is the first model to score 100% on the ARC-AGI-3 reasoning test. Before you trust it with client work, run the one-hour test below: 5 real tasks, old vs new, one scorecard.
Here is what happened with GPT-6.2 Astra and Sol, on our sister site MeanCEO. This article is for founders and small business owners in Europe who rely on ChatGPT: why experts disagree, and how to check a new model yourself.

What happened with GPT-6.2 Astra and Sol?

On 17 November 2026, OpenAI released GPT-6.2 Astra, its new top model for hard, long projects, and GPT-6.2 Sol, a cheaper, faster model for everyday work. OpenAI's launch post says: "Starting today, GPT-6.2 Astra and Sol are available in ChatGPT Work and Codex for Plus, Pro, Enterprise, and Edu users."
If you only use ChatGPT for normal chats, the change has probably not reached you. A user in the same forum thread quoted OpenAI's Help Center: "This rollout does not add either model to regular Chat." We could not verify that page ourselves.
OpenAI says Astra sets new highs on professional and scientific tests "at less than a third of the cost per task of GPT-6 Astra". For developers, Astra costs $8 per million input tokens and $40 per million output tokens (a token is a small piece of text), down from $10 and $50. Sol costs $2 and $10. See the full price and benchmark breakdown.

What did early testers say about GPT-6.2?

Many pre-launch testers were harsh.VentureBeat ran the headline "'Essentially unusable': OpenAI launches GPT-6.2 Astra and Sol to negative reception from AI power users". Here is what testers wrote on X on 17 November:
  • Matt Shumer, described by VentureBeat as an investor and former HyperWrite AI CEO: "I had a terrible experience testing them. They were essentially unusable. I do not recommend these models." VentureBeat reports that when Astra worked as a helper under Claude models, the Claude models redid its work.
  • Theo Browne (Theo - t3.gg): "I think this is the worst model OpenAI has ever released." In a longer letter, quoted by Brave New Coin, he wrote: "It runs in loops and goes nowhere". Simon Willison quoted his example of the model getting "blocked" two hours into a job.
  • Ethan Mollick: "I found it very uneven & weird. On one hand, it was harder to steer & work with than other models." He added that on an ambitious project, a space elevator ride built with Blender and Unity, "it could deliver well".
  • Dan Shipper, CEO of Every: "Overall a disappointing release compared to Fable 5.2." (Fable 5.2 is Anthropic's newest Claude model, released on 9 November.) Quoted by OfficeChai, he also wrote that "Astra 6.2 seemed to stop too early frequently".

Who says GPT-6.2 is good?

  • Derya Unutmaz, replying to Shumer: "My early testing experience is exactly opposite of yours! They are amazing models!"
  • Peter Gostev on Sol in Ultrafast, a faster mode for Pro subscribers: "for day to day the speed is genuinely incredible".
  • ARC Prize: "GPT-6.2 Astra is the first model to achieve a 100% score on ARC-AGI-3."
  • Artificial Analysis called Astra the "#2 model" in its Intelligence Index, with a score of 62; Claude Fable 5.2 scores 65 at its highest setting.
  • Epoch AI puts Astra level with Fable 5.2 at 174 on its capability index, according to VentureBeat and The Decoder.
  • Vals AI ranked the models 7th and 9th, 1.6 points apart, and said Sol "costs less than a third as much per task".
Even Theo found strengths: VentureBeat reports he praised Astra's code review and its work on 3D and game development. Brave New Coin's verdict: "The public responses establish disagreement, not a settled verdict."

How did Sam Altman respond to the bad reviews?

OpenAI's CEO answered the same day on X: "Although the new models are very smart, we have more work ahead of us on making the models nice to work with, and will prioritize that for upcoming releases. But even if it takes a little extra work, the intelligence and capability are extraordinary."
Altman accepts the testers' main complaint. For a small business owner, that "little extra work" means more checking and rewriting before an answer is safe to send.

Why do experts disagree about GPT-6.2?

1. Benchmarks test answers, testers judge the working day. A benchmark scores a model on a fixed set of questions, and 100% on ARC-AGI-3 is real progress on reasoning puzzles. Testers spent weeks giving it messy, changing instructions, much closer to how you work.
2. "Hard to steer" is a real cost. Steering means getting the model to do what you asked, the way you asked. Mollick found Astra harder to steer, Shipper saw it stop too early, and Theo saw it run in loops. Artificial Analysis, which ranks Astra second overall, found "GPT-6.2 Astra is flat with GPT-6 Astra" on its coding agent index.
3. OpenAI's own safety report lists regressions. The GPT-6.2 System Card says Astra "omitted a known obstacle to completing its work in 22.28 percent of trials, compared with 11.09 percent for GPT-6 Astra." OpenAI wrote: "We are concerned by this regression", and guessed that making answers concise, with fewer caveats, "may be related".
In plain words: in more than 1 in 5 test runs, Astra knew about a problem and kept quiet. In your business, that looks like a summary that skips the clause you needed.Both models also meet OpenAI's "Critical" rating for cybersecurity capabilities, and our piece on AI hacking tools and small businesses covers what that power means for your defences. The Decoder also reports hallucination rates (made-up answers) of 59% for Astra and 55% for Sol in Artificial Analysis testing, one more reason to check every fact.

What does GPT-6.2 mean for you if your business runs on ChatGPT?

  • Regular ChatGPT chat: going by the reported Help Center line, nothing changes for now.
  • ChatGPT Work or Codex: run the one-hour test before you move client tasks to a new model.
  • A chatbot built on the API: Sol's lower price is tempting, but OpenAI's Codex fix for hidden obstacles skips direct API requests.
  • Data settings: the launch material we could open says nothing about new data controls, retention or EU data storage. Never paste client personal data into a trial.
Our ChatGPT for business guide has prompts for pricing, customer emails and grant drafts that make good test tasks. If GPT-6.2 fails your test, our list of free and under-€20 AI tools gives you alternatives. And with OpenAI pushing ChatGPT deeper into small business life through its new Business Hub, a testing habit pays off with every release.

How can you test a new AI model in one hour?

You need five real tasks, a timer and a scorecard. Repeat it for every new model.
  1. Minutes 0-10: pick 5 real tasks from last week, like a complaint reply, a price quote, a social post, a document summary and a spreadsheet formula. Save each instruction so both models get the same words.
  2. Minutes 10-15: clean the data. Replace client names, emails and bank details with placeholders like [CLIENT], and check your data settings.
  3. Minutes 15-40: run old vs new. Use a fresh chat per task and time each result, fixes included. If the old model is gone, compare with an answer you saved earlier.
  4. Minutes 40-55: score from 1 to 5 on accuracy, tone and following instructions, and note the minutes saved. Check every number, name and date, then ask: "What problems did you leave out?"
  5. Minutes 55-60: decide. The new model must win on accuracy and instructions; tone and speed only break ties. If it loses, retest in two weeks, since Altman says OpenAI will prioritise making the models nicer to work with.
Copy this scorecard into a spreadsheet. Write the old model's score, then the new one's (old/new).
TaskAccuracyToneInstructionsMinutes savedWinner
Complaint reply_/__/__/__/__
Price quote_/__/__/__/__
Social post_/__/__/__/__
Document summary_/__/__/__/__
Spreadsheet formula_/__/__/__/__
Total_/__/__/__/__
One scoring rule: an invented fact sets accuracy to 1, even with perfect tone. One wrong number in a client email costs more than ten saved minutes.

What should you avoid when you test GPT-6.2?

  • Judging by one viral post. Shumer, Theo and Unutmaz tested different work; your tasks decide.
  • Pasting real client data. Placeholders give you the shape of the task without anyone's personal details.
  • Trusting a confident "done". Ask what was left out and check the result against your source.
Want AI to save you time without running your business for you?

The Fe/male Switch startup game takes you from idea to first paying customers in weekly quests, with an AI co-founder and a community of founders.

👉 Start the game

More on this story

FAQ: is GPT-6.2 worse?

Is GPT-6.2 worse than GPT-6?

It depends on what you measure. Testers such as Matt Shumer and Dan Shipper found it hard to work with, and OpenAI's system card shows a safety regression. On benchmarks, Astra scored 100% on ARC-AGI-3 and Artificial Analysis ranks it second.

What is the difference between GPT-6.2 Astra and Sol?

Astra is OpenAI's top model for hard, long projects; Sol is cheaper and faster for everyday work. Sol's API price is a quarter of Astra's.

Can I use GPT-6.2 in regular ChatGPT?

OpenAI says it is available in ChatGPT Work and Codex for Plus, Pro, Enterprise and Edu users. A forum user quoted the Help Center as saying it is not added to regular Chat.

What did Sam Altman say about the bad reviews?

He wrote that OpenAI has "more work ahead of us on making the models nice to work with" and called the models' intelligence and capability "extraordinary".

Why did Matt Shumer call GPT-6.2 unusable?

He wrote that he had "a terrible experience testing them" and does not recommend the models. VentureBeat reports that Claude models supervising Astra redid its work.

How do I test a new AI model for my business?

Run 5 real tasks on the old and the new model, with client data swapped for placeholders, and score accuracy, tone, instructions and time saved. Switch only if the new model wins on accuracy.
2026-11-18 09:34 Startup News and Trends