How to Build a Chatbot with the OpenAI API: Step-by-Step Python Tutorial

How to Build a Chatbot with the OpenAI API: Step-by-Step Python Tutorial

Quick answer: To build a chatbot with the OpenAI API, install the openai Python SDK, save your API key in an environment variable, and send each user message to client.responses.create() with your system prompt in instructions. Pass the previous reply's ID back as previous_response_id so the bot remembers the conversation, then stream the output to a web page through a small backend.

That's the whole idea. The rest of this tutorial turns it into code you can run in about 20 minutes.

One warning first. Search "OpenAI chatbot tutorial" and a good share of the top results still call openai.ChatCompletion.create(). That function was removed when the Python SDK hit version 1.0 back in November 2023, and the SDK is on version 3.29 as of October 2026. Copy that code today and you get an error before your bot says hello. Everything below is written against the current Responses API and tested on openai 3.29.0.

What you'll build

A support assistant called Ava for a made-up coffee subscription brand, Brewly. By the end you'll have:

1. A terminal chatbot that streams replies word by word and remembers earlier messages.
2. A FastAPI backend that keeps your API key off the browser.
3. A plain HTML chat page that talks to that backend.
4. A clear idea of what each conversation costs you.

Python 3.9 or newer and an OpenAI account with billing enabled are all you need. No frameworks like LangChain. You'll understand every line, which matters the first time something breaks in production.

Responses API vs Chat Completions vs Assistants: which one?

OpenAI has three APIs that can power a chatbot, and older tutorials mix them up.

OpenAI's own migration guide says Responses is "recommended for all new projects," and reports 40–80% better cache utilization than Chat Completions in internal tests. Better cache hits mean lower bills on long chats, which is reason enough for a chatbot.

So this tutorial uses the Responses API throughout.

Step 1: Set up the project and your API key

Create a folder, a virtual environment and install four packages:

1
2
3
4
mkdir openai-chatbot && cd openai-chatbot
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install openai python-dotenv fastapi uvicorn

Generate a key at platform.openai.com → API keys, then create a .env file in the project folder:

1
2
OPENAI_API_KEY=sk-proj-your-key-here
OPENAI_MODEL=gpt-6-luna

Add .env to .gitignore right now, before your first commit. Leaked keys get scraped from public GitHub repos within minutes, and the bill lands on your account.

What goes wrong here: if you see OpenAIError: Missing credentials. Please pass an api_key..., the SDK can't find the variable. Check that load_dotenv() runs before OpenAI() is created and that you started Python from the folder containing .env.

Step 2: Send your first request

Create hello.py:

1
2
3
4
5
6
7
8
9
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI()  # reads OPENAI_API_KEY from the environment
response = client.responses.create(
    model="gpt-6-luna",
    input="Explain what an API key is in one sentence.",
)
print(response.output_text)

Run python hello.py. If a sentence comes back, your account, key and billing all work.

response.output_text is a convenience property that joins all the text the model produced. The full response.output list also holds reasoning items and tool calls, which you'll need later for memory.

Which model should a chatbot use?

These are the current text models and their prices per 1 million tokens, from OpenAI's models page:

Our take: start every chatbot on Luna. Most customer-facing questions ("where's my order?", "can I change my plan?") don't need a frontier model, and Luna costs 1/20th of Sol. Upgrade only the conversations that need it. Because the model name lives in .env, switching is a one-line change.

Step 3: Give the bot a personality and rules with a system prompt

Without instructions, the model answers anything about anything. A support bot needs a job description. In the Responses API, that goes in the instructions parameter:

1
2
3
4
5
6
7
8
9
10
11
SYSTEM_PROMPT = """You are Ava, the support assistant for Brewly, an online coffee subscription.
- Only answer questions about orders, subscriptions, shipping and coffee.
- Keep replies under 80 words unless the user asks for more detail.
- If you are not sure, say so and point the user to help@brewly.example.
- Never invent order numbers, prices or delivery dates."""
response = client.responses.create(
    model="gpt-6-luna",
    instructions=SYSTEM_PROMPT,
    input="Can I pause my subscription for a month?",
)
print(response.output_text)

A few things we've learned writing system prompts for client bots:

• Rules beat adjectives. "Keep replies under 80 words" works. "Be concise" gets interpreted loosely.
• Tell it what to do when it doesn't know. The fallback line (email support) is what stops a bot from making up a refund policy.
• Ban the specific lies. Order numbers and delivery dates are what support bots most love to invent, so the prompt names them.

If you want to go deeper on wording, our guide to prompt engineering covers structure and testing.

Step 4: Add conversation memory (three ways)

Ask the bot from Step 3 "Can I pause my plan?", then follow up with "What about two months?" It has no idea what you mean. Every API call starts with a blank slate, because the model is stateless.

You have three ways to fix that.

One thing none of the three changes: you pay for the history every turn. OpenAI's docs say it directly: with previous_response_id, "all previous input tokens for responses in the chain are billed as input tokens." Server-side memory saves you code, not tokens.

Option A: manual history (stateless)

1
2
3
4
5
6
7
8
9
10
11
history = []
def ask(user_text):
    history.append({"role": "user", "content": user_text})
    response = client.responses.create(
        model="gpt-6-luna",
        instructions=SYSTEM_PROMPT,
        input=history,
        store=False,   # nothing kept on OpenAI's side
    )
    history.extend(response.output)  # keep ALL output items, not just the text
    return response.output_text

Note history.extend(response.output). OpenAI's conversation-state guide says to keep every output item so reasoning items stay intact. Appending only output_text as a string works on simple models and quietly degrades answers on reasoning ones.

Option B: previous_response_id (the terminal chatbot)

This is the full, runnable terminal bot. Save it as chat.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI()
MODEL = os.getenv("OPENAI_MODEL", "gpt-6-luna")
SYSTEM_PROMPT = """You are Ava, the support assistant for Brewly, an online coffee subscription.
- Only answer questions about orders, subscriptions, shipping and coffee.
- Keep replies under 80 words unless the user asks for more detail.
- If you are not sure, say so and point the user to help@brewly.example.
- Never invent order numbers, prices or delivery dates."""
def main():
    previous_id = None
    print("Ava is ready. Type 'quit' to exit.\n")
    while True:
        user_text = input("You: ").strip()
        if user_text.lower() in {"quit", "exit"}:
            break
        if not user_text:
            continue
        stream = client.responses.create(
            model=MODEL,
            instructions=SYSTEM_PROMPT,   # resend every turn
            input=user_text,
            previous_response_id=previous_id,
            reasoning={"effort": "low"},
            stream=True,
        )
        print("Ava: ", end="", flush=True)
        for event in stream:
            if event.type == "response.output_text.delta":
                print(event.delta, end="", flush=True)
            elif event.type == "response.completed":
                previous_id = event.response.id
        print("\n")
if __name__ == "__main__":
    main()

Run python chat.py, ask about pausing, then ask "what about two months?" Ava now knows what "that" refers to.

See the comment # resend every turn? That's the bug that catches almost everyone. The SDK documentation says that with previous_response_id, "the instructions from a previous response will not be carried over to the next response." Send instructions only on the first call and by turn three Ava has forgotten she's a coffee bot and will happily write you a cover letter.

Option C: the Conversations API

For a web app, a user might close the tab and come back tomorrow. A conversation object survives that, and unlike stored responses it isn't subject to the 30-day TTL:

1
2
3
4
5
6
7
conversation = client.conversations.create()
response = client.responses.create(
    model="gpt-6-luna",
    instructions=SYSTEM_PROMPT,
    input="Can I pause my plan?",
    conversation=conversation.id,
)

Save conversation.id against your user record and pass it back on every message. We'll use this in the web version.

Step 5: Stream replies instead of waiting

You already saw streaming in chat.py, but it deserves its own section because it changes how the bot feels more than any model upgrade.

Without streaming, the user types a question and stares at nothing for two, three, four seconds while the whole answer generates. Then a wall of text appears. With streaming, the first words show up almost immediately and the reply types itself out. Same total time. Completely different experience.

The mechanics are two lines: pass stream=True, then loop over events and print the ones of type response.output_text.delta. Other events you'll see include response.created, response.completed (which carries the final response object and its ID) and error.

Step 6: Put the chatbot on a web page with FastAPI

Here's where a lot of tutorials go wrong. They call the OpenAI API straight from browser JavaScript, which means shipping your API key to every visitor. Anyone who opens dev tools can copy it and run up your bill.

The fix is a thin backend. The browser talks to your server; only your server talks to OpenAI.

Create app.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
import os
from dotenv import load_dotenv
from fastapi import FastAPI
from fastapi.responses import FileResponse, StreamingResponse
from openai import OpenAI
from pydantic import BaseModel, Field
load_dotenv()
client = OpenAI(timeout=30, max_retries=3)
MODEL = os.getenv("OPENAI_MODEL", "gpt-6-luna")
SYSTEM_PROMPT = """You are Ava, the support assistant for Brewly, an online coffee subscription.
- Only answer questions about orders, subscriptions, shipping and coffee.
- Keep replies under 80 words unless the user asks for more detail.
- If you are not sure, say so and point the user to help@brewly.example.
- Never invent order numbers, prices or delivery dates."""
app = FastAPI()
class ChatIn(BaseModel):
    conversation_id: str
    message: str = Field(min_length=1, max_length=2000)
@app.get("/")
def home():
    return FileResponse("index.html")
@app.post("/conversations")
def new_conversation():
    conversation = client.conversations.create()
    return {"conversation_id": conversation.id}
@app.post("/chat")
def chat(body: ChatIn):
    def generate():
        stream = client.responses.create(
            model=MODEL,
            instructions=SYSTEM_PROMPT,
            input=body.message,
            conversation=body.conversation_id,
            reasoning={"effort": "low"},
            max_output_tokens=800,
            stream=True,
        )
        for event in stream:
            if event.type == "response.output_text.delta":
                yield event.delta
    return StreamingResponse(generate(), media_type="text/plain")

Three small details doing real work here:

• max_length=2000 on the message stops someone pasting a novel into your bot and charging you for it. FastAPI rejects it with a 422 before OpenAI is ever called.
• max_output_tokens=800 caps the reply cost. Reasoning tokens count toward this limit, so don't set it too tight or replies get cut off.
• OpenAI(timeout=30, max_retries=3) raises the SDK's default of 2 automatic retries, which covers most brief rate-limit and network blips.

Now the front end. Create index.html in the same folder:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Brewly Assistant</title></head>
<body>
  <div id="log"></div>
  <form id="chat-form">
    <input id="msg" autocomplete="off" placeholder="Ask about your order...">
    <button>Send</button>
  </form>
  <script>
    const log = document.getElementById("log");
    let conversationId = null;
    function addLine(who, text) {
      const p = document.createElement("p");
      p.textContent = `${who}: ${text}`;  // textContent, never innerHTML
      log.appendChild(p);
      return p;
    }
    document.getElementById("chat-form").addEventListener("submit", async (e) => {
      e.preventDefault();
      const input = document.getElementById("msg");
      const text = input.value.trim();
      if (!text) return;
      input.value = "";
      if (!conversationId) {
        const res = await fetch("/conversations", { method: "POST" });
        conversationId = (await res.json()).conversation_id;
      }
      addLine("You", text);
      const botLine = addLine("Ava", "");
      const res = await fetch("/chat", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ conversation_id: conversationId, message: text }),
      });
      const reader = res.body.getReader();
      const decoder = new TextDecoder();
      let reply = "";
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        reply += decoder.decode(value, { stream: true });
        botLine.textContent = `Ava: ${reply}`;
      }
    });
  </script>
</body>
</html>

Start the server:

1
uvicorn app:app --reload

Open http://127.0.0.1:8000 and chat. That's a working, streaming, memory-aware chatbot in roughly 130 lines across two files.

(The textContent comment isn't decoration. Model output rendered with innerHTML is an XSS hole: a user can talk the bot into returning a <script> tag. If you later want Markdown formatting, run it through a sanitizer such as DOMPurify first.)

For a production widget with a launcher bubble, mobile layout and analytics, see our walkthrough on how to add AI chat to your website.

How much does an OpenAI chatbot cost to run?

Less than most people expect, if you pick the model deliberately.

Take a typical support chat: a 300-token system prompt, 10 back-and-forth turns, user messages around 50 tokens and replies around 150. Because earlier turns are re-billed as input each time, the 10 turns add up to roughly 12,500 input tokens and 1,500 output tokens.

These are estimates from list prices. They leave out reasoning tokens (billed as output) and cached-input discounts, which push in opposite directions. Measure your real numbers from response.usage after a week of traffic.

The pattern that matters: cost grows with conversation length, not just count, because every turn re-sends the history. Two ways to keep long chats in check:

• Trim manual history. With Option A, keep the system prompt plus the last 10–20 items.
• Turn on server-side compaction. Pass context_management=[{"type": "compaction", "compact_threshold": 200000}] and OpenAI compresses older context once the conversation crosses that token count.

Our full playbook on reducing LLM API costs covers caching, routing and batching in more depth. If you're budgeting a whole project rather than API calls, see what an AI chatbot costs to build in 2026.

Common mistakes (and the fix for each)

AttributeError: module 'openai' has no attribute 'ChatCompletion' You copied pre-2023 code. Use client = OpenAI() and client.responses.create().

400 Bad Request mentioning temperature GPT-6 models don't accept temperature, top_p or top_logprobs while reasoning is switched on. Remove them. Control tone through the system prompt instead. (On Luna you can set reasoning={"effort": "none"} if you really need sampling controls.)

The bot forgets its role after a few messages You only sent instructions on the first call. With previous_response_id, resend them every turn.

Replies stop mid-sentence max_output_tokens is too low. Reasoning tokens eat into it before visible text does. Raise the cap or lower the reasoning effort.

RateLimitError (429) under load The SDK retries automatically, but sustained traffic needs a queue or per-user rate limit on your side. Check your usage tier in the OpenAI dashboard.

Before you go live: a short production checklist

The tutorial bot works. A public bot needs a bit more:

1. Authenticate users and rate-limit per user. An open /chat endpoint is a free LLM for the whole internet.
2. Send a safety_identifier. It's a stable per-user ID (OpenAI recommends hashing the email or username) that helps OpenAI detect abuse coming through your app without flagging your whole account.
3. Decide on data retention. Responses are stored for 30 days by default. For health, finance or other sensitive data, use manual history with store=False.
4. Log every conversation on your side. You can't improve a prompt you can't read. Review 20 real chats a week and you'll find the gaps fast.
5. Write a test set. Twenty questions with expected answers, run after every prompt change, catches regressions before customers do.
6. Give the bot real knowledge. A system prompt can't hold your entire help center. That's what retrieval is for.

Where to go next: from chatbot to something useful

Right now Ava is polite but knows nothing about Brewly's actual policies or orders. The next two upgrades fix that:

• Retrieval-augmented generation (RAG) lets the bot answer from your own documents. Upload them to a vector store and give the model the file_search tool, or build your own pipeline. Start with how RAG works, and if you're weighing it against training a custom model, read RAG vs fine-tuning.
• Function calling lets the bot do things: look up an order, pause a subscription, book a call. At that point it stops being a chatbot and starts being an agent. Here's the difference in practice: AI agents vs chatbots.

Need a chatbot built for your business?

This tutorial gets you a working prototype in an afternoon. Getting from prototype to a bot that knows your products, connects to your CRM or order system, and holds up under real traffic is where most of the work lives.

That's the work our AI development team does: RAG chatbots, support agents and LLM integrations built in Python and FastAPI and shipped to production. Tell us what you're building and we'll tell you honestly whether you need us or this tutorial is enough.