AI-SOL-001. A public agency needs an assistant to answer questions from approved policy documents that change weekly and to identify the supporting passages. Which architecture is the best starting point?

• Fine-tune the model once on a snapshot of the policies and never update it, even though the documents change every week
• ✓ Use a retrieval pipeline over versioned, approved policy documents and provide retrieved passages as context, then evaluate the responses (CORRECT)
• Store the policies only in the system prompt and manually rewrite that prompt by hand after each weekly change to the documents
• Ask the model to answer from its pretrained memory, without retrieving the current documents or identifying any supporting passages


AI-SOL-002. An assistant retrieves internal documents for users with different access permissions. Where should document-level access control be enforced?

• ✓ Filter retrieval results using the authenticated user's permissions before retrieved content is passed to the model (CORRECT)
• Put a prompt instruction telling the model not to reveal documents that the user is not allowed to access
• Give every user the same index and rely on the model to infer authorization from the wording of the question
• Apply permissions only after the final answer has already been generated, by checking it against the user's role


AI-SOL-003. An AI agent can create support tickets and issue refunds. Which design best limits harm from an incorrect or manipulated model output?

• Give the agent unrestricted access to every tool and system so that it can complete tasks without any delays
• Let the model decide for itself whether its own refund action is authorized, based on the conversation context
• ✓ Expose only allowlisted tools, enforce authorization in the service, and require human confirmation for high-impact actions (CORRECT)
• Put the administrator password in the prompt and ask the model to keep it private from the users it talks to


AI-SOL-004. A RAG assistant sometimes follows malicious instructions embedded in retrieved documents. Which mitigation is strongest?

• ✓ Treat retrieved text as untrusted input, constrain available tools and permissions, and validate actions outside the model (CORRECT)
• Increase the model temperature so that it is less likely to follow the embedded instructions found in retrieved documents
• Assume that internal documents cannot contain prompt injection, because only trusted employees are able to edit them
• Remove authentication from the retrieval service so that it can respond faster and pass documents to the model sooner


AI-SOL-005. A team is comparing two RAG pipelines before release. Which evaluation plan gives the most useful evidence?

• Compare only the average response length of both pipelines on a few examples written by the developers themselves
• ✓ Use a representative, held-out set to assess retrieval quality and answer relevance/grounding, and review safety failures (CORRECT)
• Choose the pipeline whose underlying model has the largest parameter count, since larger models always perform better
• Measure only whether the API returned HTTP 200 for each request, and treat every successful response as correct


AI-SOL-006. A user-facing AI endpoint has a 500 ms latency objective. Which measurement best tests whether the objective is met under realistic use?

• The model's training time on a developer laptop, measured before the model is deployed
• The average latency of one request sent while the service is idle and no other users are connected
• The median time for token generation only, excluding retrieval and network time
• ✓ End-to-end latency percentiles, such as p95, under representative concurrency and input sizes (CORRECT)


AI-SOL-007. A new model version may improve quality but could increase errors. Which release approach best limits production risk?

• Replace the current version globally in a single step, and retain no copy of the previous version that could be used to roll back
• ✓ Deploy the new version to a small canary share, compare defined quality and reliability measures, and expand or roll back based on results (CORRECT)
• Test the new version only on its own training data and, if the scores look good, route all production traffic to it immediately
• Disable monitoring during the rollout to avoid alert noise, and review the quality of the new version only after the release is complete


AI-SOL-008. A model must return a JSON object used by a downstream service. What should the integration do before acting on the response?

• Assume the model output is valid, because JSON was explicitly requested in the prompt instructions
• Execute any code or command contained in the output, since the model was asked to produce it
• ✓ Parse and validate the output against a strict schema and reject or safely handle invalid values (CORRECT)
• Remove all validation so there are no delays and let the downstream service cope with bad values


AI-SOL-009. A document-analysis request can take several minutes and does not need an immediate result. Which API pattern is most appropriate?

• ✓ Accept the request, enqueue a background job, return a job identifier, and provide a status/result endpoint (CORRECT)
• Keep one HTTP connection open indefinitely and retry the entire job from the start if it times out
• Return a fabricated answer immediately so the user is not kept waiting while the real job runs
• Start a new synchronous request for every internal processing step and wait for each one to finish


AI-SOL-010. A transient timeout occurs after a request that may have created a payment record. What design makes retrying safer?

• Retry forever with no delay and no request identifier
• Disable timeouts so the client waits indefinitely
• Repeat the operation with a new identifier each time, even if the first attempt may have succeeded
• ✓ Use bounded retries with backoff and an idempotency key or deduplication check for the side effect (CORRECT)

Last modified: Tuesday, 20 October 2026, 10:50 AM