Introducing Governor, a context-driven model router for coding agents Bito Ai
Your coding agent bill, cut in half
Technical design in hours, not days 

DeepSeek V4.1 Flash 1017 vs V4.1 Flash: the cheap model reached Sonnet’s level

DeepSeek V4.1 Flash 1017 vs V4.1 Flash coding benchmark scores

Table of Contents

Cheap models tend to fall short on long coding work, and DeepSeek’s V4.1 Flash was no exception: 73% on our long sessions, a few points behind Claude Sonnet 5.5. When DeepSeek released V4.1 Flash 1017, at $0.15 per million input tokens and $0.60 output, we wanted to know whether the update closed that gap. We ran it through the same long agentic coding sessions we use for every model, next to a fresh rerun of the V4.1 Flash it replaces, with default settings and nothing tuned for either model.

On our long coding sessions the new DeepSeek scores 77%, the same as Sonnet 5.5 and Haiku 5.5 at their defaults. A session costs $0.14. Sonnet costs $1.28 for the same score, and Opus 5.5 costs $5.42 for 87%. On quick one-shot asks it matched Opus outright.

DeepSeek V4.1 Flash vs V4.1 Flash 1017Old vs new
Long-session score
73%77%
One-shot score
72%90%
Cost per long session
$0.16$0.14
Minutes per long session
14.0 min15.4 min

DeepSeek V4.1 FlashDeepSeek V4.1 Flash 1017 (new)

Scores on a 0-100% scale; cost and minutes are each scaled to their own largest value.

The numbers

Scores are percentages of the maximum. Long build sessions, per session:

Long sessionsScoreCost per sessionTime per session
ClaudeOpus 5.5, medium (default)87%$5.4217.2 min
NewDeepSeek V4.1 Flash 101777%$0.1415.4 min
ClaudeSonnet 5.5, medium (default)77%$1.286.8 min
ClaudeHaiku 5.5, medium (default)76%$0.698.1 min
PreviousDeepSeek V4.1 Flash73%$0.1614.0 min

One-shot asks, per question:

One-shot asksScoreCost per questionTime
ClaudeOpus 5.5, medium (default)90%$0.320.9 min
NewDeepSeek V4.1 Flash 101790%$0.0132.0 min
PreviousDeepSeek V4.1 Flash72%about $0.011.4 min

Both DeepSeek versions are priced at the same per-token rates, so the cost gap between them is only the tokens each one used.

Where the new DeepSeek got better

Our long sessions are multi-turn agentic coding sessions on a real service that can run 15-20 minutes. The model traces the relevant code, works out why something fails, plans a fix, implements it and then audits its own change. The new model’s gains all sit in the middle of that sequence:

Turn in a long sessionScore by model
Trace the code
89%92%95%
Analyze the failure
66%75%90%
Plan the fix
73%81%81%
Implement
69%68%85%
Audit
66%70%82%

DeepSeek V4.1 FlashDeepSeek V4.1 Flash 1017 (new)Claude Opus 5.5

The analysis turn went up 9 points and the plan turn 8, and the new model’s plans now score level with Opus. Implementation didn’t move at all. Both DeepSeek versions write the fix about 17 points worse than Opus, and the audit is 12 points behind. Analysis still trails Opus by 15, so the new model reasons better than the old one without reasoning as well as Opus. Haiku 5.5 showed a similar shape earlier this month, closer to Opus while reading and planning, further behind once it had to write the change and check it.

On one-shot asks the bigger story is consistency. The old model was usually fine and occasionally fell apart: on one of the four one-shot tasks, a V4.1 Flash session replied to a stray notification from a background command instead of answering the question, and that task averaged 31% for it. V4.1 Flash 1017 didn’t have a single session like that in 16, and its scores across sessions sat within a few points of each other. For an agent you leave running unattended, a model that doesn’t have off days is worth more than a few points of average.

Slower than Claude, and it thinks a lot

V4.1 Flash 1017 is a talkative thinker. On the first turn of one of our longest tasks it made 78 requests and wrote about 3,600 output tokens per request, against 30 requests and 700-900 tokens for the old model. Almost all of that is reasoning. It was served at about 148 tokens a second, faster than we saw for the old model, so the slowness comes from the model itself. On the long sessions this mostly evens out (15 minutes against the old model’s 14), but on one-shot asks the new DeepSeek takes two minutes where Opus takes under one.

The extra thinking barely shows on the bill. At $0.60 per million output tokens, those 283,000 output tokens cost about 17 cents.

How we tested

The benchmark is a battery of agentic coding tasks taken from real work on our own services. Each model gets the same fixed prompts in Claude Code with the version pinned, and every turn is graded against an answer key that was checked against the code, by a judge scoring five dimensions. The judge is Opus 5.5. We ran 8 long sessions for each DeepSeek version and 16 one-shot sessions for the new one. The old model’s one-shot numbers come from 8 sessions on October 6, and its long-session score in this rerun (73%) matched that earlier run (72%). Claude numbers are from our earlier runs on the same tasks and keys. Session-to-session noise is about 3 points, so read 1-2 point gaps as ties.

What we’d tell a team

If you’re running V4.1 Flash, switch. It’s the same price for a better model, and the gain lands on the analysis and planning turns, where the old model was weakest.

If you’re paying for Sonnet 5.5 at its default to do agentic coding, V4.1 Flash 1017 gives you the same score on long sessions for about a ninth of the money, at roughly twice the wall-clock. Haiku 5.5 is the other cheap option and sits at the same score for about five times the new DeepSeek’s price, and it’s faster. For one-shot asks, V4.1 Flash 1017 is as good as Opus on our tasks at about 1/25th of the cost.

Keep a stronger model for writing and checking the actual change. Neither DeepSeek version has closed any ground on implementation, and that’s still a 17-point gap to Opus.

Picture of Amar Goel

Amar Goel

Bito’s Co-founder and CEO. Dedicated to helping developers innovate to lead the future. A serial entrepreneur, Amar previously founded PubMatic, a leading infrastructure provider for the digital advertising industry, in 2006, serving as the company’s first CEO. PubMatic went public in 2020 (NASDAQ: PUBM). He holds a master’s degree in Computer Science and a bachelor’s degree in Economics from Harvard University.

Picture of Amar Goel

Amar Goel

Amar is the Co-founder and CEO of Bito. With a background in software engineering and economics, Amar is a serial entrepreneur and has founded multiple companies including the publicly traded PubMatic and Komli Media.

Written by developers for developers red heart icon

This article is brought to you by the Bito team.

Latest posts

Best cheap model for coding: Inkling 2 Small vs Claude Haiku 5.5, Gemini 3.8 Flash, DeepSeek and GPT-6.1 Terra

Inkling 2 Small vs Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 for coding

Inkling 2 Small benchmark: the strongest U.S. open-weight coding model we tested, vs DeepSeek, GLM and Kimi

GPT-6.1 Terra vs GPT-6.1 Sol vs GPT-6 Luna for coding: which OpenAI model to use

GPT-6.1 Terra benchmark: how it compares to Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 for coding

Top posts

Best cheap model for coding: Inkling 2 Small vs Claude Haiku 5.5, Gemini 3.8 Flash, DeepSeek and GPT-6.1 Terra

Inkling 2 Small vs Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 for coding

Inkling 2 Small benchmark: the strongest U.S. open-weight coding model we tested, vs DeepSeek, GLM and Kimi

GPT-6.1 Terra vs GPT-6.1 Sol vs GPT-6 Luna for coding: which OpenAI model to use

GPT-6.1 Terra benchmark: how it compares to Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 for coding

From the blog

The latest industry news, interviews, technologies, and resources.

Best cheap model for coding: Inkling 2 Small, Claude Haiku 5.5, Gemini 3.8 Flash, DeepSeek and GPT-6.1 Terra overall scores

Best cheap model for coding: Inkling 2 Small vs Claude Haiku 5.5, Gemini 3.8 Flash, DeepSeek and GPT-6.1 Terra

arrow bito ai
Inkling 2 Small vs Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 coding benchmark overall scores

Inkling 2 Small vs Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 for coding

arrow bito ai
Inkling 2 Small vs DeepSeek, GLM, Kimi and MiniMax coding benchmark overall scores

Inkling 2 Small benchmark: the strongest U.S. open-weight coding model we tested, vs DeepSeek, GLM and Kimi

arrow bito ai