Advanced Prompt Engineering

Basic prompting gets you basic results. Once you understand how LLMs actually process your instructions, you can design prompts that consistently produce better, more reliable output. This article covers three of the most effective advanced techniques: chain-of-thought prompting, few-shot learning, and structured output control.

Chain-of-Thought Prompting

LLMs are surprisingly bad at direct reasoning. Ask a model “A bat and a ball cost $1.10. The bat costs $1.00 more than the ball. How much does the ball cost?” and many models will confidently answer $0.10 (it's actually $0.05). The problem isn't that the model can't do math — it's that it shortcuts to a plausible-sounding answer without showing its work.

Chain-of-thought (CoT) prompting fixes this by asking the model to reason step by step before giving its final answer. The results are dramatic: on many benchmark tasks, CoT improves accuracy by 20-30% compared to direct answering.

How to Apply It

The simplest form is just adding “Let's think step by step” at the end of your prompt. This works surprisingly well, but more structured approaches are even better:

Bad:
"Calculate the total cost of 3 items priced at $12, $8, and $15 with 9% tax."

Good:
"Calculate the total cost step by step:
1. First, add the item prices.
2. Then calculate the tax amount.
3. Finally, add the tax to the subtotal.

Items: $12, $8, and $15. Tax rate: 9%."

For complex reasoning tasks, you can go further with “chain-of-thought with self-consistency” — run the same prompt multiple times with a temperature setting above 0, then take the majority answer. This effectively lets the model “vote” on the most likely correct answer across multiple reasoning paths.

Few-Shot Prompting

Instead of describing what you want, show it. Few-shot prompting works by giving the model a small number of input-output examples before asking it to handle your actual input. This leverages the model's pattern-matching ability to infer the task structure from examples rather than from abstract instructions.

When Few-Shot Shines

Few-shot is particularly effective for:

• Format conversion. Show 2-3 examples of unstructured data mapped to JSON, and the model will generalize the format.
• Classification. Show examples with labels, and the model handles edge cases better than a rule-based classifier.
• Tone/style transfer. Show examples of formal vs. casual rewriting, and the model internalizes the difference.

Choosing Your Shots

The quality of your examples matters more than the quantity. A single well-chosen example that covers a tricky edge case is worth five generic ones. Best practices:

• Pick examples that represent the diversity of inputs you'll encounter.
• Include one “tricky” example that shows how to handle a corner case.
• Keep examples concise — long few-shot examples consume context window space and can dilute the model's focus.
• Order matters: put your closest example to the target last (recency bias is real in LLMs).

One underrated trick: use “zero-shot” as your baseline first, then add examples only if the output quality isn't good enough. Many modern models (GPT-4, Claude 3.5+) are surprisingly good from instructions alone, and over-engineering with too many examples can actually hurt performance by biasing the model toward patterns that don't generalize.

Structured Output Control

One of the most practical advanced techniques is getting the model to output structured data reliably — JSON, XML, markdown tables, or custom formats. Without explicit structure, you get free-form text that's hard to parse programmatically.

The JSON Prompt Pattern

You are a data extraction assistant. Extract the following fields from the text and return ONLY valid JSON with no additional text:

{
  "product_name": "string",
  "price": "number",
  "in_stock": "boolean",
  "categories": ["string"],
  "description_summary": "string (max 50 words)"
}

Text: [input text here]

JSON:

Key tips for structured output:

• Be explicit about types. “Return a JSON object with fields …” with sample schemas works far better than “Extract the data.”
• Constrain the output. Tell it “ONLY valid JSON, no markdown formatting, no code fences.” Many models default to wrapping JSON in markdown code blocks.
• Use system-level constraints. Put format instructions in the system prompt where available, and the specific data in the user message.
• Validate after the fact. Parse the output before trusting it. A malformed JSON response is still a real possibility even with careful prompting.

Schema-Driven Generation

For complex structured outputs, provide a full JSON schema rather than a single example. This gives the model a precise specification to follow:

Generate a meeting summary in the following JSON Schema format:
{
  "type": "object",
  "properties": {
    "meeting_title": {"type": "string"},
    "date": {"type": "string", "format": "date"},
    "attendees": {"type": "array", "items": {"type": "string"}},
    "key_decisions": {"type": "array", "items": {
      "type": "object",
      "properties": {
        "decision": {"type": "string"},
        "owner": {"type": "string"},
        "deadline": {"type": "string", "format": "date"}
      },
      "required": ["decision", "owner"]
    }},
    "action_items": {"type": "array", "items": {"type": "string"}}
  },
  "required": ["meeting_title", "date", "key_decisions", "action_items"]
}

Putting It All Together

The real power comes from combining these techniques. A production workflow might look like:

1. System prompt defines the model's role and output format constraints.
2. Few-shot examples in the user message show the pattern for the specific task.
3. Chain-of-thought instructions guide the reasoning for complex parts.
4. Schema-driven format spec ensures machine-parseable output.
5. Post-processing validation catches format errors and retries if needed.

A Note on Limits

No prompting technique can fix a model that doesn't have the right knowledge, or that was asked something fundamentally outside its capabilities. Chain-of-thought makes reasoning better, not perfect. Few-shot examples help with format, not with factual accuracy. Structured output reduces parsing failures, not content errors. The best prompt engineer in the world still needs to verify the output — these techniques make the output more reliable, not reliable.