Interview Prep Guide
Generative AI and LLM Interview Questions with Practical Answers
Prepare for modern AI interviews with practical questions on LLM application design, RAG, evaluation, prompt injection, agents, reliability, latency, and cost.
Core Generative AI Interview Questions
How is an LLM application different from a traditional deterministic application?
An LLM application has probabilistic outputs, so it needs explicit context control, evaluation datasets, guardrails, observability, and fallback behavior in addition to normal software testing.
What is RAG, and when would you choose it instead of fine-tuning?
Retrieval-augmented generation supplies relevant external knowledge at request time. Choose it when information changes often, must be attributable, or belongs to a private knowledge base; fine-tuning is better suited to behavior, style, or stable task patterns.
What does temperature control, and why is setting it to zero not a complete reliability strategy?
Temperature influences sampling randomness. A low value can make outputs more consistent, but model updates, context changes, ambiguous prompts, and nondeterministic infrastructure still require validation and evaluation.
Applied LLM Engineering Questions
How would you evaluate an LLM feature before releasing it?
Build a representative versioned evaluation set, define task-specific quality and safety criteria, compare against a baseline, include human review for subjective cases, and monitor the same signals after release.
How do you design an LLM workflow to resist prompt injection?
Treat user and retrieved content as untrusted data, keep instructions separated by trust level, minimize tool permissions, validate every tool input and output, and require confirmation for sensitive actions.
How would you reduce latency and cost in a production LLM feature without damaging quality?
Measure the request path, reduce unnecessary context and calls, route simple tasks to smaller models, cache safe repeatable work, stream when useful, and verify every optimization against quality evaluations.
Advanced AI System Questions
How would you design a reliable AI agent that can call external tools?
Give the agent a narrow goal, typed tool contracts, least-privilege credentials, bounded steps and spend, idempotent actions, observable traces, and explicit approval before consequential operations.
How would you monitor an LLM application in production?
Monitor task success, groundedness or validity, safety events, refusal quality, latency, token use, cost, retries, tool failures, and user feedback while protecting sensitive prompt and response data.
Scenario-Based AI Interview Questions
Users report that your RAG assistant gives confident answers that are not supported by retrieved documents. What do you do?
Reproduce and classify failures, measure retrieval and generation separately, require evidence-aware responses, improve chunking or ranking, add abstention behavior, and block release changes that regress groundedness.
A team wants to let AI-generated code merge automatically. How would you evaluate and control that proposal?
Treat AI code like untrusted contributor code: require tests, static and security checks, scoped permissions, provenance, review thresholds, and stricter controls for sensitive repositories or infrastructure changes.
Frequently Tested Generative AI Engineering Questions
How would you evaluate an LLM feature before release?
Create a representative, versioned evaluation set with task-specific quality, safety, latency, and cost metrics, then compare changes against a baseline and inspect failures.
What is prompt injection and how should an application defend against it?
Prompt injection is untrusted content attempting to change the model’s instructions or trigger unsafe actions; defend with privilege separation, constrained tools, validation, and explicit trust boundaries.
When should you use retrieval-augmented generation?
Use retrieval when answers must be grounded in changing or private knowledge and the application can fetch relevant evidence at request time.
How do temperature and structured output affect reliability?
Temperature changes sampling variability, while structured output constrains the response shape; neither guarantees that the content is factually correct.
How do you control LLM latency and cost?
Measure tokens and stage latency, choose the smallest model that meets quality, reduce irrelevant context, cache safe repeated work, and route only difficult requests to stronger models.