How to Cut Your OpenAI / LLM API Bill (Practical Guide)

TL;DR
Route simple tasks to mini or nano models, put stable prompt content first so it's cached at up to 90% off, send async work to the Batch API or Flex (50% off), and cap output tokens, since output costs 5x input. In our worked example, those moves take a $5,000/month assistant to about $381.
Quick answer: To cut your OpenAI or LLM API bill, send each task to the smallest model that passes your tests, put the stable part of every prompt first so it gets cached at up to 90% off, move anything that can wait to the Batch API or Flex (50% off), and cap output tokens. Stacked together, these levers routinely remove most of the bill.
Most teams overpay for the same reason. They picked one model during the prototype, wired it into every feature, and never looked again. The invoice grows with traffic, someone in finance asks why, and the answer is "we're using AI." That answer stops working around the third month.
This guide walks through the levers we actually reach for when an LLM API bill gets out of hand, in the order we reach for them. Prices quoted are OpenAI's published Standard-tier rates as of November 2026. They change often, so treat the ratios as the lesson and check the official pricing page before you plan a budget around them.
Where your LLM API money actually goes
Before cutting anything, know what you're paying for. Every request bills three things: input tokens, output tokens, and (on newer models) cache writes. Output is the expensive one. Across OpenAI's current lineup, output tokens cost 5x as much as input tokens, and reasoning tokens are billed as output even though you never see them.
Model (Standard tier) | Input / 1M | Cached input / 1M | Output / 1M |
|---|---|---|---|
GPT-6 Astra | $10.00 | $1.00 | $50.00 |
GPT-6.1 Sol | $2.00 | $0.10 | $10.00 |
GPT-6.1 Terra | $1.00 | $0.05 | $5.00 |
GPT-6 Luna | $0.10 | $0.01 | $0.50 |
GPT-5.4 mini | $0.75 | $0.075 | $4.50 |
Look at the spread. Astra input costs 100x what Luna input costs. If a feature is running on the top model by default, that one decision probably outweighs every prompt tweak you could make.
1. Measure cost per feature before you cut anything
You can't fix a number you only see as a monthly total. Log, for every call: the feature or endpoint, the model, input tokens, cached tokens, output tokens, and the user or tenant. The API already returns most of this in the usage object, including usage.prompt_tokens_details.cached_tokens.
Then sort by spend. In almost every audit, one or two features own the majority of the bill. Often it's something nobody thinks of as "the AI feature" (a background summarizer, a retry loop, an agent that re-reads the same document on every step). Start there.
2. Route each task to the smallest model that passes your tests
If you want to cut your OpenAI API bill fast, this is the biggest lever and the most ignored one. Classification, extraction, tagging, short summaries, routing decisions and formatting don't need a frontier model. A mini or nano tier handles them, and the price difference is an order of magnitude or more.
The trick is to make the decision with data instead of vibes:
- Pull 100 to 200 real requests for the feature, with the outputs you consider correct.
- Run them through the cheapest candidate model.
- Score the results the same way you'd score the expensive model.
- Move up one tier only if the cheap one fails your bar.
For mixed traffic, put a small router in front: the cheap model handles the request, and only escalates to the larger one when it's unsure or the task is flagged as hard. Most requests never escalate.
(One honest caveat: a cheaper model that needs three retries isn't cheaper. Count retries in your eval.)
3. Structure prompts so they get cached
Prompt caching is the closest thing to free money in the API. When the start of a request exactly matches a recent one, the matching tokens bill at the cached rate, which is 90% to 95% below the normal input price on current OpenAI models. OpenAI applies it automatically, but only if your prompts are built for it.
Rules that decide whether you get a cache hit:
- Stable content first, variable content last. System instructions, tool definitions, schemas and few-shot examples go at the top. The user's message and anything dynamic go at the bottom.
- No timestamps or request IDs near the top. One changed character early in the prompt breaks the match for everything after it. Move that data into the request's
metadatafield. - Keep tool lists identical. If you need to restrict tools per turn, use
allowed_toolsintool_choiceinstead of changing the tools array. - Append, don't edit. Rewriting earlier messages in a conversation throws away the cached prefix.
- Use
prompt_cache_key. It groups related requests onto the same machines. In OpenAI's Prompt Caching 201 cookbook, one coding customer raised their hit rate from 60% to 87% just by adding it.
On older models, caching only applies once the prompt reaches 1,024 tokens, so a short prompt never benefits.
Here's the part most guides skip. On the GPT-5.6 and GPT-6 families, OpenAI now lists a separate cache-write price: 1.25x the input rate for a 30-minute cache and 2.5x for 12 hours. That changes the math. A prefix you write once and never reuse costs more than not caching. On GPT-6.1 Sol, though, one reuse already wins: $2.50 to write plus $0.10 to read, against $4.00 for sending the same million tokens twice uncached. So caching pays off on any prefix that gets hit at least twice inside the window, which describes nearly every chatbot and agent.
4. Move anything that can wait to the Batch API
If the user isn't waiting for the answer, you shouldn't pay real-time prices. The Batch API takes a file of requests, returns results within 24 hours, and bills at 50% of the Standard rate on input, cached input and output alike.
Good candidates:
- Nightly summarization or tagging of tickets, reviews or emails
- Generating embeddings or metadata for a content backlog
- Running evals and regression tests on new prompts
- Bulk content generation, translation or data cleanup
Requests that don't finish in the window expire, and only completed requests get billed. Anthropic and Google sell the same 50%-off batch deal on their APIs, so this lever travels with you if you switch providers.
5. Use Flex processing for traffic in between
Some work is too slow for real time but can't wait a day either. Flex processing sits there. It bills at the same 50% discount as Batch, but you call it synchronously: set service_tier: "flex" on the request.
The trade-off is availability. When capacity is tight, Flex can return a 429 error instead of an answer (those failed calls aren't charged). Wrap it in a retry with backoff, and fall back to Standard if a job is getting close to its deadline.
The opposite setting deserves a warning too. "Fast" mode (the priority tier) costs 2x Standard on most models, and Ultrafast costs 6x. Turn those on only for the endpoints where latency clearly earns money, never as a global default.
6. Cap output and reasoning tokens
Since output costs 5x input, a chatty model is an expensive model. Three fixes:
- Set
max_output_tokenson every call. It includes reasoning tokens, so it also stops a reasoning model from thinking for 20,000 tokens about a yes/no question. - Lower reasoning effort for simple tasks. Most extraction and classification work runs fine on low effort.
- Ask for structured output. A JSON schema with the exact fields you need cuts the preamble, the restated question and the "I hope this helps" at the end.
A plain instruction like "Answer in under 80 words" also works better than people expect. Test it against your eval set before trusting it.
7. Send less context
Input tokens are cheaper than output, but you send a lot more of them. Common leaks:
- Full chat history on every turn. After 20 turns you're paying for the whole transcript each time.
- Too many RAG chunks. Retrieving 20 chunks when the answer lives in 3 multiplies input cost with no quality gain. Tune top-k and add a reranker. (Our RAG chatbot tutorial covers this setup.)
- Whole documents where a section would do.
There's a tension here worth knowing about. We used to summarize old conversation turns aggressively to save input tokens. Then we noticed summarization rewrites the start of the prompt, which kills the cache. Now the default is: keep the prefix stable as long as the conversation is short, and compact only when the history gets long enough that the savings clearly beat the lost cache hits.
One more threshold: on current flagship models, requests over 272K input tokens switch to long-context pricing, which doubles the input rate for the entire request. If you're near that line, trim to stay under it.
8. Watch the hidden line items
Token counts aren't the whole bill. Check your usage dashboard for these:
- Web search tool: $10 per 1,000 calls, plus the search content tokens at model rates.
- File search tool: $2.50 per 1,000 calls, plus storage at $0.10 per GB per day after the first free GB.
- Containers (Code Interpreter and hosted shell): billed per 20-minute session, from $0.03 to $1.92 depending on memory size.
- Regional data residency: a 10% uplift on newer models.
An agent that runs a web search on every step can quietly spend more on tool calls than on tokens.
9. Put hard limits on agents
Agents are where bills explode, because one bad loop can make hundreds of calls before anyone notices. Build the brakes into your own code:
- A maximum number of steps per task
- A per-task and per-user token budget, checked before each call
- A stop rule when the agent repeats the same error or the same tool call twice
- Separate API projects per environment, so a runaway test in staging can't touch production spend
Project budgets in the OpenAI dashboard are useful for alerts, but don't count on them to stop traffic. The hard stop belongs in your application. If you're scoping an agent project, our breakdown of AI agent pricing in 2026 covers how running costs fit into the total.
How much can you cut your OpenAI API bill? A worked example
Here's the math for a hypothetical support assistant, using OpenAI's list prices. It handles 100,000 requests a month. Each request sends 3,000 input tokens (2,500 of them a fixed system prompt and knowledge snippet) and gets back 400 output tokens.
Step | Monthly cost |
|---|---|
Everything on GPT-6 Astra | $5,000 |
Route to GPT-6.1 Sol after it passes evals | $1,000 |
Fixed 2,500-token prefix served from cache | ~$531 |
Output capped at 250 tokens | ~$381 |
That's about a 92% cut, before touching Batch or Flex at all. Your numbers will differ. The order of the savings usually won't: model choice first, caching second, output discipline third.
Where to start this week
Lever | Typical saving | Effort |
|---|---|---|
Cost logging per feature | Shows where to act | Low |
Smaller model for simple tasks | Often 80%+ on that feature | Medium (needs evals) |
Cache-friendly prompt order | Up to 90% on repeated input | Low |
Batch API for async jobs | 50% | Low to medium |
Flex for in-between traffic | 50% | Low (add retries) |
Output and reasoning caps | Varies, often large | Low |
Agent step and budget limits | Prevents blowups | Medium |
If you only do two things to cut your OpenAI API bill, add per-feature cost logging today and reorder your prompts for caching tomorrow. Both take hours, not weeks, and they tell you exactly where the remaining money is going.
And if you'd rather have someone do the audit for you, Craxinno builds and tunes production AI features on the OpenAI and Claude APIs. Tell us what you're running and we'll show you where the bill can come down.
Frequently Asked Questions
How can I reduce my OpenAI API costs quickly?+
Start with two changes that take hours, not weeks: log token usage per feature so you can see which endpoint owns the bill, and reorder your prompts so stable content (instructions, tool definitions, examples) comes first and gets cached. Then move simple tasks to a mini or nano model and send non-urgent jobs to the Batch API at 50% off.
Is the OpenAI Batch API really 50% cheaper?+
Yes. Batch requests bill at half the Standard rate on input, cached input and output. The trade-off is time: results come back within a 24-hour window, and requests that don't finish in that window expire without being billed. It suits summarization, tagging, evals and bulk generation, not live chat.
How does OpenAI prompt caching work?+
When the beginning of a request exactly matches a recent request, those matching tokens bill at the cached input rate, which is 90% to 95% cheaper on current models. Put fixed content at the top of the prompt, keep timestamps and IDs out of it, keep tool definitions identical, and use prompt_cache_key to group related requests.
What is the cheapest OpenAI model for simple tasks?+
As of November 2026, GPT-6 Luna is the lowest-priced model on OpenAI's Standard tier at $0.10 per million input tokens and $0.50 per million output tokens, about 100x cheaper than GPT-6 Astra. Test it on 100 to 200 real requests before switching, and move up a tier only if it fails your quality bar.
What is the difference between Flex processing and the Batch API?+
Both cost 50% of the Standard rate. Batch takes a file of requests and returns results within 24 hours. Flex is called synchronously with service_tier set to flex, so you get answers in normal request time, but it can return a 429 error when capacity is tight. Use Flex for work that can tolerate retries but can't wait a day.
Do these cost-cutting tips work for Claude and Gemini too?+
Mostly, yes. Anthropic and Google both offer 50%-off batch processing and discounted cache reads, and the habits that matter most (picking the smallest model that passes your tests, keeping prompt prefixes stable, capping output and limiting agent loops) apply to any LLM API. Check each provider's rate card for exact numbers.
Building something with AI?
We ship production AI — agents, RAG pipelines and LLM integrations that survive real users, not demos.
Start a projectKeep ReadingMore case studies like this
Engineering retros, product launches, and brand systems from our studio — updated monthly.
All case studiesTechnology Used
Tags & Keywords
Continue with Blogs.
View all blogs
AI AgentsAI Agent Pricing: What Should You Actually Pay in 2026?
AI Agent Pricing: What Should You Actually Pay in 2026? AI agent pricing in 2026 runs from about $0.10 per interaction for off-the-shelf support bots to $200,000+ for enterprise custom builds. Most businesses should expect to pay $0.99 to $2.00 per resolved conversation on a SaaS agent, or a one-time build fee plus $500 to $3,000 a month in model and infrastructure costs for a custom agent. That range is useless on its own, so this guide breaks it apart. You'll see what the big vendors actually charge, how custom-build quotes are put together, the token math most quotes leave out, and the point where building your own agent becomes cheaper than renting one. We build AI agents for clients at Craxinno, and the most common question we get isn't "can you build this?" It's "why did the last agency quote me $90,000 for this?" Let's answer that properly. AI Agent Pricing at a Glance (2026) Option Typical 2026 price Best for No-code agent platforms $29 to $200 per month to start Internal automations, small teams testing ideas Off-the-shelf support agents $0.10 per session to $2.00 per conversation or resolution Customer support on an existing helpdesk Enterprise agent platforms $125+ per user per month, or $30,000 to $300,000+ per year Large orgs already on Salesforce, Zendesk and similar Custom-built agent (agency quotes) $20,000 to $200,000+ one-time Agents tied to your own data, tools and workflows Running costs for a custom agent Model tokens plus roughly $500 to $3,000+ per month in infrastructure Anyone who owns the agent The no-code figure comes from Toolradar's survey of 319 agent tools, where the median cheapest paid plan was $29 a month. The custom-build range matches what agencies such as Cleveroad publish. Vendor prices below are list prices as reported in late 2026, so check the vendor's page before you sign. The 5 AI Agent Pricing Models You'll Run Into Vendors stopped agreeing on how to charge for agents somewhere around 2024. Salesforce alone now sells Agentforce three different ways at the same time. Here's how the models break down. 1. Per conversation You pay every time the agent talks to someone, whether it solves the problem or not. Salesforce launched Agentforce at $2 per conversation in October 2024. Ada reportedly lands between $1.00 and $3.50 per interaction. Simple to understand, but you pay for the failures too. 2. Per resolution (outcome-based) You pay only when the agent closes the ticket without a human. Intercom's Fin charges $0.99 per outcome. Zendesk charges $1.50 per automated resolution on committed volume and $2.00 pay-as-you-go. Gorgias sits around $0.60 to $1.27 depending on plan. This sounds like the fairest deal, and often it is. The catch: the vendor defines "resolved." If a customer gives up and leaves, some systems count that as a win. Read the definition in the contract, not the landing page. 3. Credits or per action Salesforce's Flex Credits (May 2025) bill $0.10 per action, sold in packs of 100,000 credits for $500. Microsoft Copilot Studio sells 25,000 credits for $200 a month. Lindy starts at $29.99 per user per month with 3,000 credits. Credits are flexible, but forecasting them is hard until you've watched real usage for a month or two. 4. Per seat The old SaaS model, bolted onto agents. Agentforce per-user licenses start at $125 per user per month. Microsoft 365 Copilot is $30 per user per month. Fine for copilots that help employees. Strange for an agent that's supposed to replace the work of seats. 5. Platform fee plus usage Common at the enterprise end. Decagon reportedly charges around $50,000 a year as a platform fee, then adds per-conversation or per-resolution fees on top. Sierra quotes custom enterprise pricing only. What Off-the-Shelf AI Agents Cost in 2026 If you want a support agent and already use one of these helpdesks, here's the real comparison. Vendor Pricing model Reported price Intercom Fin Per outcome $0.99 per outcome (helpdesk seats from $29/month) Zendesk AI Per resolution $1.50 committed, $2.00 pay-as-you-go Salesforce Agentforce Conversation, credits or seat $2/conversation, $0.10/action, or $125/user/month Freshdesk Freddy AI Per session $0.10 per session ($100 per 1,000) Gorgias Per resolution About $0.60 to $1.27 Ada Custom About $30,000+ yearly minimum (reported) Decagon Platform fee plus usage About $50,000/year plus usage (reported) Don't stop at the per-resolution number. Agentforce implementations reportedly run $50,000 to $150,000, with ongoing consulting at $10,000 to $25,000 a month. A cheap unit price on a platform that needs a six-figure rollout isn't cheap. How Much Does It Cost to Build a Custom AI Agent? Published agency ranges look like this, using Cleveroad's 2026 breakdown: Reactive agent (answers questions, no memory): $20,000 to $35,000+ Contextual agent (remembers the conversation, uses your data): $40,000 to $70,000+ Autonomous agent (plans steps, calls tools, takes actions): $80,000 to $120,000+ Domain-specific enterprise agent (compliance, deep integrations): $100,000 to $200,000+ Those numbers are real quotes from real agencies. But they mostly reflect agency rates, not the difficulty of the work. Every custom quote comes down to one formula: Build cost = hours × hourly rate A US or UK agency billing $120 to $200 an hour gets to $80,000 fast. The same scope at a $25 an hour studio (our standard rate at Craxinno) costs a fraction of that. So the first question to ask about any quote is simple: how many hours is this, and what's the breakdown? Where do the hours actually go? In the agents we've shipped, the LLM call itself is the easy part. The time disappears into four places: Integrations. Connecting the agent to your CRM, helpdesk, database or payment gateway. Every API has its own auth, rate limits and weird edge cases. Knowledge setup. Cleaning your docs, chunking them and building retrieval (RAG) so the agent answers from your data instead of guessing. We wrote a full walkthrough in our guide to building a RAG chatbot with the OpenAI API . Guardrails and fallbacks. What happens when the agent isn't sure? When a tool call fails? When a user tries to make it say something it shouldn't? Testing against real conversations. This is the step that's skipped in cheap builds and the reason many agents look great in a demo and fall apart in week two. If a quote doesn't show hours against these items, you can't compare it with anything. If you're weighing offshore options, our post on how to hire AI developers in India covers what to check before you sign. The Token Math Most AI Agent Quotes Leave Out Here's the cost that keeps running after the build invoice is paid. Every time your agent thinks, it spends tokens. Cost per task = (input tokens × input price) + (output tokens × output price) Take a simple inbox-triage agent that reads 20,000 tokens and writes 2,000 per task. On a mid-tier model priced at $2 per million input tokens and $10 per million output tokens (OpenAI list pricing as of October 2026, per Dupple's breakdown): Input: 20,000 × $2 ÷ 1,000,000 = $0.04 Output: 2,000 × $10 ÷ 1,000,000 = $0.02 Total: $0.06 per task Now look at what model choice does to the same workload, per 1,000 tasks: Agent type Budget model Mid-tier model Flagship model Inbox triage (20k in, 2k out) $3 $60 $300 Research, 5 lookups (60k in, 6k out) $9 $180 $900 Coding/ops with cached context $11.70 $222 $1,170 That's a 100x spread for the same job. And this is where most of the waste we see comes from: teams pick the flagship model for everything because it won the demo, then get a shock at month-end. Route the easy 80% of requests to a cheap model and save the expensive one for the hard cases. It's usually the single biggest saving available. Two more things on tokens. Prices move. Dupple reports Google's Gemini Argon doubling on December 1, 2026, and Gemini 3.8 Flash doubling on January 1, 2027, so lock your budget to a model you can swap out. And if a job can wait, OpenAI's Batch processing halves standard prices. Hidden Monthly Costs of Running an AI Agent The model bill is rarely the biggest line once you're in production. Budget for these too: Vector storage and memory: roughly $500 to $2,500+ a month depending on traffic and data size Logging pipeline: $300 to $800 a month Monitoring integrations: $300 to $1,000 a month, plus $100 to $500 for error tracking Prompt tuning and testing: $1,000 to $5,000 a month if you keep improving the agent (and you should) Search grounding: on Gemini, 5,000 requests a month are free, then $14 per 1,000 Hosting: your servers or serverless functions. If you're on AWS, plan this before launch, not after Those ranges come from Cleveroad's and Dupple's 2026 breakdowns. A small agent with modest traffic will land at the low end of each. You won't need every line on day one, but you'll need most of them by month three. Buy vs. Build: When Does a Custom AI Agent Pay for Itself? This is the question that actually decides your budget, so let's do the math with real list prices. Say your support agent resolves 5,000 tickets a month. On Intercom Fin at $0.99 per outcome: about $4,950 a month, or roughly $59,400 a year. On Zendesk at $1.50 per resolution: about $7,500 a month, or $90,000 a year. On a custom agent: 5,000 tasks on a mid-tier model at a triage-sized prompt costs about $300 a month in tokens. Add $1,000 to $2,000 for infrastructure and monitoring, and you're at roughly $1,300 to $2,300 a month before the build cost. At that volume, the gap is roughly $2,600 to $6,200 a month. A custom build in the $15,000 to $30,000 range can pay for itself inside a year. At 300 tickets a month, it won't. Fin would cost you about $300 a month, and no custom build beats that. So our honest rule of thumb: Under about 1,000 resolutions a month: buy. Use the agent built into your helpdesk. Thousands of resolutions a month, or the agent needs to take actions in your own systems: build, or at least get a quote. The agent is the product you sell: build. You can't rent your core feature from a vendor who sets the price. So, What Should You Actually Pay for an AI Agent in 2026? Here's where we'd land for most small and mid-sized businesses: Testing an idea: $0 to $200 a month on a no-code platform. Spend a month proving people will use it. Customer support on a standard helpdesk: $0.99 to $2.00 per resolution. Negotiate committed-volume pricing once you know your numbers. A custom agent tied to your systems: a build fee based on clear hours at a fair rate, then a monthly running cost you can predict from token math plus infrastructure. Paying $90,000 for a first agent with no hour breakdown and no running-cost estimate is the thing to avoid. Not because the agency is dishonest, but because you can't tell whether the price matches the work. Red Flags in an AI Agent Quote No hour breakdown, just a single number. No estimate of monthly token and hosting costs. Locked to one model provider with no way to switch. No test plan using your real past conversations. No answer to "what happens when the agent is wrong?" A per-resolution contract that doesn't define "resolution." If a quote has two or more of these, ask for a revised one before you compare prices. For a wider view of how development pricing works, see our breakdown of what a dedicated development team costs in 2026 and our best tech stack picks for SaaS startups . Get a Clear AI Agent Quote from Craxinno Craxinno is an AI-first design and development studio. We've been building software since 2017, and we price AI agent work the way this article recommends: hours you can see, at $25 an hour, with a running-cost estimate before you commit. If you want a second opinion on a quote you've already received, send it over. We'll tell you if it's fair. Talk to us about your AI agent project → Sources Fin: AI Agent Pricing Comparison 2026 SaaStr: Salesforce now has 3 pricing models for Agentforce Dupple: How Much Does It Cost to Build an AI Agent in 2026? Cleveroad: AI Agent Development Cost in 2026
RAGHow to Build a RAG Chatbot with the OpenAI API (Tutorial)
How to Build a RAG Chatbot with the OpenAI API (Tutorial) A RAG chatbot answers questions from your documents instead of from whatever the model memorised during training. You split your content into chunks, turn each chunk into an embedding, store them, retrieve the closest chunks for every question, and pass them to the model as context. With the OpenAI API you can do this in two ways: the managed file_search tool, or a do-it-yourself pipeline with embeddings. This tutorial builds both in Python, then shows what changes when you take it to production. What is a RAG chatbot (and why not just fine-tune)? RAG (retrieval-augmented generation) is a pattern where the app looks up relevant passages from your own data at question time and gives them to the language model as context, so the answer is grounded in those passages rather than in the model's training data. Picture a support bot asked, "Do you offer refunds after 30 days?" A plain LLM will guess. It sounds confident, and it may be wrong. A RAG bot first pulls the refund section of your policy PDF, then answers from that text and can point to the source. Fine-tuning changes how a model writes. It is a poor way to teach it facts that change every month. If your pricing page, product docs or HR policy gets edited, RAG picks up the change the moment you re-index the file. A fine-tuned model needs another training run. For a chatbot that has to be correct about your business , retrieval is almost always the first thing to build. How a RAG chatbot works: the 5-step pipeline Ingest your sources (PDFs, Markdown, help-centre pages, database rows). Chunk them into passages of a few hundred tokens with some overlap. Embed each chunk into a vector with an OpenAI embedding model and store it. Retrieve the top-k chunks closest to the user's question. Generate an answer with the model, using only the retrieved chunks, and cite them. Everything else (re-ranking, hybrid search, query rewriting, evaluation) is a refinement of one of these five steps. Two ways to build it with the OpenAI API Option A: Managed file_search Option B: DIY embeddings Setup time Minutes A few hours Chunking & embedding Handled by OpenAI You control it Where vectors live OpenAI vector store Your database (pgvector, Qdrant, Pinecone, etc.) Metadata filters, custom ranking Limited Full control Best for Prototypes, internal tools, small to medium document sets Multi-tenant SaaS, strict data residency, large or fast-changing data Our view after shipping both: start with Option A to prove the bot is useful, then move to Option B only when you hit a concrete limit (tenant isolation, filtering, cost at scale, or data that cannot leave your infrastructure). Prerequisites Python 3.10 or newer An OpenAI API key stored as the OPENAI_API_KEY environment variable A folder of documents to test with (a product manual or FAQ export works well) 1 pip install openai numpy tiktoken In the code below, the chat model is read from an OPENAI_MODEL environment variable. Set it to a current model from OpenAI's models page, since model names change more often than this tutorial will. Option A: Build a RAG chatbot with the Responses API and file_search This is the fastest route. OpenAI's file search tool parses your files, chunks them, embeds them and runs the search for you. You get a working bot in roughly 40 lines. Step 1: Upload files and create a vector store 1 2 3 4 5 6 7 8 9 10 11 12 13 from pathlib import Path from openai import OpenAI client = OpenAI ( ) vector_store = client . vector_stores . create ( name = "company_knowledge_base" ) for path in Path ( "docs" ) . glob ( "*.pdf" ) : with open ( path , "rb" ) as f : uploaded = client . files . create ( file = f , purpose = "assistants" ) result = client . vector_stores . files . create_and_poll ( vector_store_id = vector_store . id , file_id = uploaded . id , ) print ( path . name , result . status ) # expect "completed" print ( "Vector store ID:" , vector_store . id ) What goes wrong: scanned PDFs with no text layer index as almost nothing. If a file "completes" but the bot never finds anything in it, run OCR on it first. Step 2: Ask questions with the file_search tool 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 import os MODEL = os . environ [ "OPENAI_MODEL" ] VECTOR_STORE_ID = "vs_..." # from step 1 SYSTEM = ( "You are the support assistant for our company. " "Answer only from the provided files. " "If the files do not contain the answer, say you don't know " "and suggest contacting support. Keep answers short." ) def ask ( question , previous_id = None ) : response = client . responses . create ( model = MODEL , instructions = SYSTEM , input = question , tools = [ { "type" : "file_search" , "vector_store_ids" : [ VECTOR_STORE_ID ] , "max_num_results" : 5 , } ] , include = [ "file_search_call.results" ] , previous_response_id = previous_id , ) return response r1 = ask ( "What is the refund window?" ) print ( r1 . output_text ) include=["file_search_call.results"] returns the matched chunks, so you can log them and show sources in your UI. max_num_results trades quality for cost: fewer chunks means fewer input tokens and faster replies, but a higher chance of missing the right passage. Five is a sensible starting point. Step 3: Keep the conversation going 1 2 r2 = ask ( "And does that apply to annual plans?" , previous_id = r1 . id ) print ( r2 . output_text ) Passing previous_response_id lets the API carry the chat history, so the follow-up "that" resolves to the refund window. No manual message list needed. That is a complete RAG chatbot. Wrap ask() in a FastAPI endpoint, put a chat widget on top, and you have an MVP. Option B: Build RAG from scratch with OpenAI embeddings Build it yourself when you need control: per-customer filtering, your own database, custom chunking for tables or code, or a hard rule that documents never leave your servers except as embedding requests. Step 1: Chunk your documents Chunking decides retrieval quality more than any other setting. Too large and each chunk mixes topics; too small and it loses the context needed to answer. 1 2 3 4 5 6 7 8 9 10 import tiktoken enc = tiktoken . get_encoding ( "cl100k_base" ) def chunk_text ( text , source , size = 400 , overlap = 60 ) : tokens = enc . encode ( text ) chunks = [ ] step = size - overlap for start in range ( 0 , len ( tokens ) , step ) : piece = enc . decode ( tokens [ start : start + size ] ) chunks . append ( { "text" : piece , "source" : source } ) return chunks Start around 300 to 500 tokens with 10 to 15% overlap. Better still, split on headings first (one H2 section per chunk), and only fall back to token windows for long sections. Structured splits keep a question and its answer in the same chunk. Step 2: Create embeddings 1 2 3 4 5 6 7 8 9 10 11 12 13 import numpy as np EMBED_MODEL = "text-embedding-3-small" def embed ( texts ) : resp = client . embeddings . create ( model = EMBED_MODEL , input = texts ) return np . array ( [ d . embedding for d in resp . data ] , dtype = np . float32 ) chunks = [ ] for path in Path ( "docs" ) . glob ( "*.md" ) : chunks += chunk_text ( path . read_text ( ) , source = path . name ) vectors = [ ] for i in range ( 0 , len ( chunks ) , 100 ) : # batch requests vectors . append ( embed ( [ c [ "text" ] for c in chunks [ i : i + 100 ] ] ) ) matrix = np . vstack ( vectors ) matrix /= np . linalg . norm ( matrix , axis = 1 , keepdims = True ) text-embedding-3-small returns 1,536-dimension vectors, accepts up to 8,192 input tokens, and is listed at $0.02 per million tokens ($0.01 with the Batch API) at the time of writing. Embedding a few thousand pages of docs usually costs cents, not dollars. Check OpenAI's pricing page before you quote a client. If retrieval quality on your data is weak, try text-embedding-3-large (3,072 dimensions) before rewriting anything else. Step 3: Retrieve the top-k chunks 1 2 3 4 5 6 def retrieve ( question , k = 5 ) : q = embed ( [ question ] ) [ 0 ] q /= np . linalg . norm ( q ) scores = matrix @ q # cosine similarity top = np . argsort ( - scores ) [ : k ] return [ ( chunks [ i ] , float ( scores [ i ] ) ) for i in top ] NumPy is fine for a few thousand chunks. Past that, move the vectors into a vector database: pgvector if you already run Postgres, or Qdrant, Weaviate or Pinecone if you want a dedicated service. The retrieval function stays the same shape; only the storage changes. Step 4: Generate a grounded answer with citations 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 def answer ( question , history = None ) : hits = retrieve ( question ) context = "\n\n" . join ( f "[{n}] ({c['source']})\n{c['text']}" for n , ( c , _ ) in enumerate ( hits , 1 ) ) prompt = ( "Answer the question using ONLY the context below. " "Cite sources like [1]. If the answer is not in the context, " "reply: \"I don't have that information.\"\n\n" f "Context:\n{context}\n\nQuestion: {question}" ) response = client . responses . create ( model = MODEL , input = ( history or [ ] ) + [ { "role" : "user" , "content" : prompt } ] , ) return response . output_text , hits text , sources = answer ( "How do I reset my password?" ) print ( text ) The "I don't have that information" line is the single most useful instruction in the prompt. Without it, the model fills gaps with plausible guesses, which is exactly the failure RAG is meant to stop. Step 5: Add memory for follow-up questions Follow-ups like "what about on mobile?" retrieve badly because the question alone has no subject. Fix it by rewriting the question before retrieval: 1 2 3 4 5 6 7 def standalone ( question , history ) : r = client . responses . create ( model = MODEL , input = history + [ { "role" : "user" , "content" : f "Rewrite this as a standalone question: {question}" } ] , ) return r . output_text Retrieve with the rewritten question, answer with the original one. This small step fixes a large share of "the bot forgot what we were talking about" complaints. Taking your RAG chatbot to production The demo works on day one. The work is in the next four weeks. These are the points we check on every RAG build at Craxinno: Build an evaluation set first. Write 30 to 50 real questions with known correct answers and the source they should come from. Re-run them after every change to chunk size, model or prompt. Without this you are tuning by vibes. Use hybrid search for names and codes. Embeddings are weak on exact strings like SKU numbers, error codes and people's names. Combine vector search with keyword (BM25) search and merge the results. Re-rank before you generate. Retrieve 20 candidates, re-rank them, and pass the best 5. Answers get sharper and you send fewer tokens. Filter by tenant and permission. In a multi-customer product, every chunk needs metadata (customer ID, role, document date) and every query must filter on it. One leaked chunk from another customer is a serious incident. Re-index on change. Hook ingestion to your CMS or storage events so edited documents are re-embedded automatically. Stale answers erode trust faster than "I don't know". Log retrieved chunks with every answer. When a user reports a wrong answer, you need to see whether retrieval or generation failed. They need different fixes. Stream responses. Use stream=True so text appears as it is generated. Perceived speed matters more than total latency in a chat UI. Common RAG mistakes (and quick fixes) Symptom Likely cause Fix Bot says "I don't know" when the answer exists Chunks too small or question phrased differently from the docs Larger chunks, heading-based splits, query rewriting Answers mix two unrelated topics Chunks too large Split by section, reduce chunk size Wrong product or SKU in the answer Pure vector search on exact identifiers Add keyword search (hybrid) Confident but invented answers Prompt allows answering outside context Explicit "only from context" rule and a fallback reply Costs climbing Too many chunks per query, long histories Lower top-k, re-rank, trim history How much does a RAG chatbot cost to run? Two meters run. Embeddings are cheap and mostly one-off: you pay once to index and again only for changed documents. The bigger, recurring cost is the chat model, because every question sends the retrieved chunks as input tokens. Five chunks of 400 tokens add about 2,000 input tokens per question before history. Cut top_k , trim history and pick a smaller model for simple FAQ traffic, and costs stay predictable. With managed file_search , also budget for vector storage and tool calls, which OpenAI prices separately on its built-in tools pricing page. Need a production-ready RAG chatbot? A tutorial gets you a working prototype. A chatbot your customers trust needs evaluation, permissions, monitoring and a UI people actually use. Craxinno designs and ships AI products end to end, from RAG pipelines and agentic workflows in Python (FastAPI, Django) to the web and mobile apps that sit on top. If you want a RAG chatbot built on your data, talk to our team .
SaaS DevelopmentBest Tech Stack for a SaaS Startup in 2026 (Our Picks)
Best Tech Stack for a SaaS Startup in 2026 (Our Picks) Short answer: the best tech stack for a SaaS startup in 2026 is TypeScript end to end. Next.js and React on the frontend, PostgreSQL for data, a managed auth provider, Stripe for billing, and Vercel or AWS for hosting. Add Python (FastAPI) only when your product does real AI or data work. Pick boring, popular tools, because your team and your AI coding assistant already know them. That's the whole recommendation. The rest of this guide explains why, where each piece breaks, and what we'd swap for different kinds of SaaS products. We've been building web apps , payment integrations and AI products at Craxinno since 2017, so the opinions below come from projects we've shipped, not from a vendor pitch. The Best SaaS Tech Stack for 2026 at a Glance Layer Our default pick Good alternative Switch when Language TypeScript Python (for AI/data services) You run models, RAG or heavy data pipelines Frontend Next.js + React SvelteKit, Remix Your team already ships fast in something else UI Tailwind CSS + shadcn/ui Material UI, Mantine You need a heavy enterprise data-grid look Backend / API Next.js route handlers or NestJS FastAPI, Hono A mobile app or third parties need a public API Database PostgreSQL (Supabase, Neon or AWS RDS) MySQL Almost never ORM Drizzle Prisma Your team prefers a schema-file workflow Auth Supabase Auth or Better Auth Clerk, Auth0 You need enterprise SSO out of the box Payments Stripe Paddle, Polar, Razorpay You want a Merchant of Record, or you bill in INR Hosting Vercel AWS, Railway, Coolify on a VPS Bills climb, or compliance needs a private cloud Email / analytics / errors Resend, PostHog, Sentry Postmark, Mixpanel, Better Stack Personal preference Why a "Boring" Tech Stack Wins in 2026 Two years ago the argument for popular tools was hiring. That still holds. But in 2026 there's a bigger reason: most of your code will be written or reviewed with an AI assistant, whether that's Cursor, Claude Code or GitHub Copilot. AI tools are very good at Next.js, PostgreSQL and Stripe. There are millions of public examples to learn from. They are noticeably worse at a niche framework that shipped a breaking change last quarter. So a mainstream stack now buys you speed twice: from humans and from machines. We see this on client projects every week. A founder arrives with an MVP built on an exotic combination (a new edge database, a beta auth library, a custom state manager). Each piece looked clever on its own. Together, nobody can debug them, and the AI assistant keeps suggesting code for the wrong version. The rule we follow: spend your novelty budget on the product, not the plumbing. Frontend: Next.js, React, TypeScript and Tailwind Next.js is the default SaaS frontend in 2026 for practical reasons. It handles your marketing site, blog, docs and logged-in dashboard in one codebase. Server rendering keeps your public pages fast and indexable, which matters if you plan to get customers from Google instead of only paid ads. Pair it with: TypeScript everywhere. Shared types between frontend and backend remove a whole class of bugs. Tailwind CSS for styling, and shadcn/ui for components you own and can edit, instead of a locked library. TanStack Query for data fetching on client-heavy screens. When would we skip Next.js? If your SaaS is a pure internal dashboard with no public pages and no SEO needs, a plain React + Vite single-page app is simpler to host and reason about. And if your team is already fast in Vue or Svelte, stay there. Switching frameworks to follow a blog post is a bad trade. Backend: Keep It in One Codebase as Long as You Can For most early SaaS products, the backend can live inside Next.js as route handlers and server actions. One repo, one deploy, one language. That's the fastest setup for a team of one to five developers. Split out a separate API when one of these becomes true: You're shipping a mobile app or browser extension that needs the same endpoints. Customers or partners need a documented public API. Long-running jobs (imports, reports, video or AI processing) start timing out serverless functions. At that point, NestJS or Hono keep you in TypeScript. If the heavy lifting is AI or data, use Python with FastAPI . It's where the AI libraries live, and FastAPI gives you typed, documented endpoints with very little code. We often run a TypeScript product app next to a small FastAPI service that only handles model calls, embeddings and data pipelines. It's a clean split that keeps each side in the language it's best at. Database: PostgreSQL, and Stop Worrying About It Use PostgreSQL. It handles relational data, JSON columns, full-text search, and vector embeddings through the pgvector extension . That last one matters: in 2026 most SaaS products add some AI search or retrieval feature, and Postgres lets you do it without a second database. Managed options that work well: Supabase if you want database, auth, storage and realtime in one place . Neon if you want serverless Postgres with database branching for preview environments. AWS RDS or Aurora if you're already on AWS or a client's security team requires it. Add Redis for caching, rate limits and sessions once you actually need it. Not on day one. What about MongoDB? It's fine for some products. But SaaS data is almost always relational: users belong to organizations, organizations have subscriptions, subscriptions have invoices. Forcing that into documents usually ends in painful joins written by hand. Pick MongoDB only if your core data really is schemaless. For the ORM, Drizzle is our current default because it stays close to SQL and is light on serverless. Prisma is still a solid choice with a gentler learning curve. Authentication: Don't Build It Yourself Login looks like a weekend task. Then come password resets, email verification, Google sign-in, magic links, two-factor auth, team invites, roles, and session security. Every hour on that is an hour not spent on the thing customers pay for. Supabase Auth is the easy pick if you're on Supabase already. Better Auth is an open-source TypeScript library that runs in your own app and database, with organizations and roles built in. Good for B2B SaaS. Clerk or Auth0 give you polished hosted UI and enterprise SSO, at a per-user price that grows with you. Check their pricing pages against your expected user count before you commit. Whichever you choose, design for multi-tenancy from the start: every row that belongs to a customer carries an organization ID, and every query filters by it. Retrofitting this later is one of the most expensive rewrites we see. Payments and Billing: Stripe, or a Merchant of Record Stripe is still the default for SaaS billing . Subscriptions, usage-based pricing, trials, coupons, invoices and the customer portal are all there. In the US its standard card fee is 2.9% + 30¢ per transaction . The real decision is whether you need a Merchant of Record like Paddle, Polar or Lemon Squeezy. They charge a higher fee, but they become the legal seller and handle global sales tax and VAT for you. For a solo founder selling to customers in dozens of countries, that's often worth it. For founders in India billing Indian customers, Razorpay handles UPI, cards and recurring payments in INR . Plenty of our clients run Razorpay for domestic customers and Stripe for international ones. It's more setup, but it covers both markets properly. One rule regardless of provider: treat webhooks as the source of truth for subscription status. Never trust the redirect after checkout alone. The AI Layer: What a 2026 SaaS Actually Needs Almost every SaaS brief we received this year had an AI feature in it . Smart search, a chat assistant over the customer's own data, document extraction, content generation. Here's the minimal setup that works: LLM APIs from Anthropic, OpenAI or Google, called through a thin wrapper so you can switch models without touching product code. pgvector in your existing Postgres for embeddings and retrieval (RAG) . You probably don't need a dedicated vector database until you're storing many millions of vectors. A job queue for anything slower than a few seconds. Trigger.dev or Inngest are simple for TypeScript teams. RabbitMQ or Celery fit Python-heavy pipelines. GPU hosting such as RunPod only if you run open-source or fine-tuned models yourself, for example image generation. Start with hosted APIs. Self-hosting models makes sense later, once you know your volume and the API bill is a real line item . Hosting and DevOps: Vercel First, AWS When It Pays Off Deploy to Vercel at the start. Git push, preview URLs for every branch, zero server management. For a pre-revenue startup that's the right trade. Move parts of the system to AWS (or GCP or Azure) when: Your hosting bill grows faster than your revenue. Enterprise customers ask for data residency, a VPC or specific compliance controls. You run background workers, queues or GPU jobs that don't fit serverless. A middle path that's grown popular: Coolify or Railway on top of a cheap VPS, which gives you a Vercel-like workflow at server prices. What to skip early: Kubernetes and microservices. A modular monolith with clear folders for billing, auth and each product area will carry you far further than most founders expect. You can split services when a real scaling problem shows up, and you'll know exactly where the seams are by then. The Supporting Tools Every SaaS Needs Before Launch Email: Resend or Postmark for transactional email, with React Email for templates. Product analytics: PostHog. It covers events, funnels, session replay and feature flags in one tool. Error tracking: Sentry. You want to hear about a failed checkout before the customer emails you. File storage: any S3-compatible bucket, such as AWS S3 or Cloudflare R2. CI: GitHub Actions running type checks, tests and migrations on every pull request. Most of these have free tiers. Before launch, your monthly stack bill can be close to zero beyond a domain name. Best Tech Stack by SaaS Type Type of SaaS Recommended stack B2B dashboard / workflow tool Next.js, NestJS or route handlers, PostgreSQL, Better Auth with organizations, Stripe, Vercel AI-first SaaS Next.js frontend, FastAPI service, PostgreSQL + pgvector, job queue, LLM APIs, AWS or Railway for workers Mobile-first SaaS React Native (Expo) + Next.js web , shared TypeScript API, Supabase, RevenueCat or Stripe Marketplace Next.js, PostgreSQL, Stripe Connect, Redis, background jobs for payouts and notifications Data / analytics SaaS Next.js, Python ETL pipelines, PostgreSQL plus a warehouse or Parquet files for heavy queries India-focused SaaS Next.js, PostgreSQL, Razorpay subscriptions, AWS Mumbai region for data locality Six Tech Stack Mistakes We See SaaS Founders Make The Frankenstein stack. Auth from one vendor, database from another, a third tool syncing users between them. Pick tools that work together. Microservices at ten users. You'll spend months on infrastructure that a single app handles easily. Building auth and billing from scratch. Use the services above and ship the product. Skipping multi-tenancy. Adding organization IDs to every table after launch is slow and risky. Choosing by hype. A framework trending this month is not a reason to bet your company on it. No monitoring until something breaks. Sentry and PostHog take an afternoon to set up. Do it before launch. Need Help Picking or Building Your SaaS Stack? The best tech stack for a SaaS startup is the one your team can ship and maintain without a fight. If you're planning a SaaS product and want a second opinion on architecture, or a team to build the MVP , Craxinno designs and ships full-stack web apps, AI features and payment integrations for startups worldwide. Browse more engineering guides on the Craxinno blog .



