Partner With Us

← All lessons / Foundations / Lesson 03 of 12

Interactive lesson

Prompting and context: why the same AI gives different answers.

A model only knows what it learned in training and what you put in front of it. Learn how to write clear instructions, give useful context, and understand why outputs vary.

You’ll learn to
  • Write a clear brief with relevant facts and constraints.
  • Explain how context limits and temperature affect a response.

A prompt is a brief, not a magic spell

Think of a language model as a capable new volunteer on their first day. They know a lot in general, but nothing about your charity, your data or your audience unless you tell them. A good prompt is simply a good brief.

Everything the model can “see” at once, including your instructions, pasted documents and the conversation so far, is called the context. It has a size limit, called the context window. And even with the same context, settings like temperature change how predictable the answer is.

What is temperature? After the model has scored every possible next word, it still has to pick one. Temperature is a dial that controls how adventurous that pick is. Turned low, it almost always picks the top-scoring word: predictable, sometimes repetitive. Turned up, lower-scoring words get picked more often: more varied, sometimes odd. It changes variety, not accuracy: a low temperature can repeat the same wrong answer every time.

Interactive

Build a better prompt

Your task: get a useful summary of a volunteer survey for your trustees. Start with the vague prompt, then add ingredients one at a time and watch the response change.

Your prompt


            
Illustrative response

          

          
The responses were written by our team to show typical model behaviour. No AI model is called.

        

        

          
Interactive

          
Experiment: temperature and randomness

          
We have assigned example scores to possible next words for “Our volunteers are …”. Move the temperature slider: the probabilities below are calculated live with the same formula real models use (called softmax). Then generate ten completions to see how often each word is picked.

          

          

          

          

        

        

          
Interactive

          
Experiment: the context window

          
Below is a long conversation. This demo keeps the newest complete messages that fit its example token budget. Shrink the window and watch what drops out of view. Real applications may trim, summarise or retrieve older messages, or reject an oversized request.

          

          
    
          
    
        
    

        
    
          
    Key ideas to take away
    
          
      
            
    • Be specific about the task, audience and format. “Three bullet points for trustees” beats “summarise this”.
      • 
            
    • Never paste personal or sensitive data into public AI tools. Names, health details, case notes and donor records should only go into tools your organisation has approved for that data, and only when it is needed. Use synthetic or anonymised examples while you experiment.
      • 
            
    • Give the model the facts. Without your data in the context, it may guess, ask for the missing data or decline to answer. Check that the response uses the evidence you supplied.
      • 
            
    • Tell it what to do when unsure. Ask it to say “not in the data”, then test whether it follows that instruction. A prompt cannot guarantee that it will.
      • 
            
    • Temperature changes variety, not truth. Lower settings concentrate the next-token probabilities. They can still produce the same wrong answer repeatedly, so verify factual claims at any setting.
      • 
            
    • Context windows are finite. In long conversations, early instructions can drop out of view. Quality can also slip well before the limit, especially for details buried in the middle of a long input. Restate what matters, or start a fresh conversation.
      • 
          
    
        
    



        
    
          
    Check your understanding
    
          
    3 quick questions. Try as often as you like. Your progress stays on this device; answers clear when you reload.
    
          
    
          
    
        
    

        
    
          
    
            Want to go deeper?Equations, code and research for technical learners. Optional.
            
    
              
    Messages, roles and system prompts
    
              
    Chat models receive a list of messages, each with a role. The system message sets standing instructions, and an application can preserve it while trimming older conversation turns. This lab demonstrates one simple trimming policy; it does not model how every chat application handles context.
    

    messages = [
  {"role": "system", "content": "You summarise survey data for charity trustees. "
                                "Only use the data provided. If something is not in the data, say so."},
  {"role": "user",   "content": "Survey data:\n" + survey_text + "\n\nWrite three bullet points."},
]
    
              
    Sampling settings
    
              
      
                
    • Temperature rescales scores before softmax: P(i) ∝ exp(z_i / T).
      • 
                
    • Top-p (nucleus sampling) only samples from the smallest set of tokens whose probabilities add up to p.
      • 
                
    • Greedy decoding always picks the top token, which is roughly what temperature 0 does. Results are more consistent, but the wording can be repetitive, and many services still vary slightly from run to run.
      • 
              
    
              
    Techniques worth knowing
    
              
    
                
    Few-shot prompting
    Including worked examples of the input and desired output in the prompt.
    
                
    Prompt chaining
    Splitting a task into steps, each with its own prompt, passing outputs forward (Gullí, Chapter 1).
    
                
    Structured output
    Asking for JSON or a fixed template so results can be checked automatically.
    
                
    Context engineering
    Deciding what information goes into the context, in what order, and what stays out. You will use this approach in RAG (Lesson 05).
    
              
    
            
    
          
    
        
    



        
    
          
    Continue exploring
    
          
    We've picked resources that explain these ideas from different angles. They are independent of Code With Impact.
    
          
        
    

        
    
          
    Sources for this lesson
    
          
    The survey example, illustrative responses and interactive labs are original Code With Impact material; the survey data is synthetic. Temperature probabilities are calculated with the standard softmax formula.
    
          
          
    How we reference: we link and credit, we never copy other educators' text, diagrams or code. See our full Sources & credits register.