Interview Prep Guide

Applied Generative AI and LLM System Interview Questions

Applied AI questions on embeddings, retrieval, structured output, model routing, privacy, evaluation, agents, multimodal input, and operational incidents.

Applied LLM Questions

  1. What are embeddings, and what do they not guarantee?

    Embeddings map content into vectors useful for similarity, but similarity does not guarantee truth, relevance, authorization, or task success.

  2. How would you choose a chunking strategy for RAG?

    Start from document structure and user questions, preserve meaningful context, include metadata, and tune size and overlap through retrieval evaluation.

  3. Why are structured model outputs useful?

    Schemas make downstream handling more predictable and validate required types, ranges, and fields before application logic uses a response.

  4. How should conversation memory be designed?

    Store only useful consented context, distinguish durable facts from temporary turns, retrieve selectively, and give users visibility and deletion controls.

  5. When would you route requests between multiple models?

    Route when tasks have measurably different quality, latency, privacy, modality, or cost needs, with a safe fallback and evaluation per route.

Advanced LLM System Questions

  1. How do you evaluate a RAG system end to end and by component?

    Measure retrieval relevance or recall separately, then groundedness, answer correctness, citation quality, abstention, latency, and cost end to end.

  2. What security risks arise when an agent can browse or read documents?

    Untrusted content can inject instructions, exfiltrate context, misdirect tools, or trigger unsafe actions, so content never grants authority.

  3. How would you handle model-provider outages or rate limits?

    Use timeouts, bounded retries with jitter, quotas, circuit breaking, graceful fallback, request preservation, and transparent user messaging.

  4. What should be considered before sending personal data to an AI provider?

    Minimize data, establish purpose and consent, enforce retention and regional requirements, redact where possible, and verify contracts and access controls.

  5. How do multimodal AI inputs change system design?

    Images, audio, and documents add parsing, size, malware, privacy, accessibility, latency, and cross-modal evaluation requirements.

AI Production Scenarios

  1. A model upgrade improves average quality but worsens safety refusals. Do you release it?

    Do not use the average alone; compare segmented release gates, investigate the safety regression, and release only with an acceptable bounded mitigation.

  2. An AI summarizer invents action items that were never discussed. How do you fix it?

    Reproduce the cases, require evidence-linked extraction, distinguish facts from suggestions, add abstention, and evaluate on a durable incident set.

  3. Users paste secrets into an AI assistant. How should the product respond?

    Detect and minimize likely secrets, warn users, avoid unnecessary provider transmission or storage, redact logs, and support incident handling and deletion.

  4. An agent repeats a paid external action after a timeout. How do you prevent it?

    Assign an idempotency key before execution, persist action state, reconcile uncertain outcomes, and never retry consequential tools blindly.

  5. AI costs double without traffic growth. How do you investigate?

    Break cost down by route, model, tokens, retries, context, tools, and user cohort, then correlate changes with deployments and quality metrics.