CCAO-F practice questions: ten worked scenarios across the seven Associate domains

Answer: C. Choose the prompting strategy for the task type.

A. Demanding citations for statistics about the retailer's own customers, which nobody has collected, invites fabricated references. The framing still asks for one conclusion when the team needs a wide field.

B. Revenue projections for concepts that have never run are invented numbers with a decimal point. The prompt narrows to three ideas before the workshop has explored the space at all.

C. Correct. Brainstorming across named angles gives the workshop breadth, banning evidence claims removes the pressure that produced the invented survey figures, and evaluation is deferred to a step with real criteria.

D. Research mode finds published sources; it does not create the team's own customer data, and the team's problem is idea generation, not market statistics. It keeps the single-idea framing that narrowed the field.

The objective is adapting prompting strategy to task type. The team's need on Wednesday is breadth, which is a brainstorming task, and the prompt they used asked for a single conclusion with evidence, which is an analysis task. Analysis prompts push the model to justify, and when no data exists the justification gets invented, which is exactly what the survey statistics were. Reframing as brainstorming with named angles produces a wide field, explicitly forbids evidence claims so there is nothing to fabricate, and separates the later evaluation step, where the team's own criteria and any research it commissions can be applied. The distractors keep the analysis framing and try to make the evidence problem go away: by demanding citations for figures that do not exist, by asking for revenue projections that would also be invented, or by researching public statistics before the team has even decided what ideas to test.

Answer: A. Inconsistency is evidence about the source, not the format.

A. Correct. The disagreement between table and narrative shows the figures cannot be trusted as produced, so the check runs against the supplier master data first and editing follows the verified numbers.

B. The model that produced two conflicting figures has no independent access to the truth. Asking it to choose returns a confident answer with the same evidential basis as the original, which is none.

C. Tables are not more reliable than prose when the same unverified process generated both. Reconciling the text to the table tidies the paper while the count itself remains unchecked against any source.

D. A caveat does not stop the board acting on a wrong count, and it leaves a visible contradiction in a governance document. Validation before the deadline is possible; polish before validation is the trap.

Identifying inconsistencies in a response is the objective, and the point of identifying them is what they tell you. When a table and its narrative disagree and a supplier is placed in two countries, the output is showing that at least one figure is wrong and that the process producing the figures is unreliable, so neither the table nor the prose can be trusted by default. The first step is therefore to go back to the authoritative record, the supplier master data, and establish the true count and countries; only then is editing meaningful. The distractors each pick a version without evidence: the model's own adjudication, a preference for tables over prose, or a caveat that lets both versions go to the board. A tidy paper with the wrong single-source count is worse than a late one, and the director's deadline does not change which figures are true.

Answer: A and D. Format follows how each audience will use it.

A. Correct. Structured rows with the importer's field names are what the tracking system can load, and an artifact is the right container for a file the payments team will reuse rather than read.

B. Prose cannot be imported without someone re-keying 40 rows, and a long reply is the opposite of a two-minute read. It serves the analyst's convenience, not either audience's next action.

C. A deck cannot be loaded into a system that imports rows with fixed field names, and a briefing built to double as a data source is neither a two-minute read nor a record. One format for two next actions serves neither.

D. Correct. The chief operating officer's next action is a short call, so the summary gives her the total, the material exceptions and the plan, which is the information she needs in the time she has.

E. Working from the same record is not the same as needing all of it before a call. Forty rows take longer than two minutes to read and bury the total exposure and the three exceptions that matter; the summary carries those.

Organising information and selecting the output format means asking what happens to the output next. The payments team's next action is a system import, so the format is structured data with the field names the importer expects, and an artifact is the right container for a file that will be reused. The chief operating officer's next action is a two-minute read before a call, so the format is a short inline summary with the totals and the exceptions that matter. Two audiences with two next actions need two formats, and thirty minutes is enough for both. The distractors force one format on both: prose that cannot be imported and is too long to read, a slide deck that suits neither a system nor a hurried executive, and the raw table for an audience whose need is the shape of the data, not its volume.

Answer: D. Long context is a design problem, not a model problem.

A. Re-pasting decisions adds to the context that is already too long to be attended to reliably. The history staying in one place is the problem, not the solution, and the colleagues still cannot pick it up.

B. Restarting is right, but the draft is not the state. The agreed exclusions and settled questions live in the conversation, not in the checklist, so they are lost, and a single new conversation still gives the two colleagues nothing to continue from.

C. The failure is accumulated context, not insufficient capability. A stronger model in the same conversation inherits the same three hours and the same contradictions, at higher cost and with no persistence for the team.

D. Correct. A verified decision log turns three hours of history into a bounded, checkable state, and a Project gives the six-week matter and the colleagues a shared place to continue from.

The objective covers context limitations and knowing when to restart, summarise or persist. The symptom, re-asking settled questions and contradicting an agreed exclusion, is the signature of a conversation that has outgrown what the model can attend to reliably. The remedy is to compress the state into a verified summary and move it into a container built for persistence: a Project, because the work runs six weeks and two colleagues must pick it up. Verifying the summary against the draft matters, since a wrong decision log would persist an error. The alternatives each fail on one of the stated constraints. Pasting decisions back grows the very context that is failing. A fresh start with only the draft pasted in discards the decisions, which live in the conversation rather than the checklist, and does nothing for the colleagues. A more capable model in the same conversation still inherits the accumulated history, and the constraint is length and continuity, not reasoning power.

Answer: B. Analyse the use case by segment, not as one problem.

A. Automating refund decisions and safety incident replies puts a payment action and a safety judgement in the model's hands, with sampling after the fact. Volume justifies analysis, not blanket automation.

B. Correct. Each segment is matched to a role Claude can hold safely: drafting under review, augmenting a human decision, or triage only. That is what a requirements analysis is meant to produce.

C. Confining Claude to information requests leaves the refund queue, the backlog the head named, untouched, and fails a brief that asked for all three categories. Claude can prepare a refund case or triage an incident without deciding either.

D. One prompt for three categories with different consequences cannot hold the right controls for each, and launching to the whole team before measuring is the scale-first error. Segmentation is the analysis, not an afterthought.

Applying Claude to analyse requirements and use cases means recognising that 'complaint handling' is three different tasks with different structure, error detectability, consequence and decision authority. Information requests are stable and low consequence, so drafting under agent review is a good fit. Refund claims involve a payment action inside a regulated timescale, so the decision stays with a person and Claude augments the preparation. Safety incidents carry the highest consequence and need human judgement, so Claude's role is confined to triage that a person acts on. The segmented proposal captures value in every category while placing the human where authority cannot be delegated. Limiting Claude to information requests ignores the refund backlog the head named and the brief to cover all three categories, and the other two options treat the categories as identical, which is how a refund gets decided or a safety report gets answered by a template.

Answer: B and E. Communicate the benefit and the control together.

A. Draft time is not headcount. The saving has to be netted against review and the office's other work before anyone can claim budget from it, and promising savings that then fail costs credibility.

B. Correct. Stating the benefit and the correction rate together, with the review step that catches errors, gives leadership the true net value and the resourcing the design depends on.

C. Staff satisfaction and the office's approval of the pilot are not evidence of accuracy. The pilot's own data shows that 12 percent of drafts were wrong on facts that matter to students.

D. Better sources lower the correction rate; they do not eliminate confident errors, and a promise of zero removes the review that protects students from a wrong deadline. Overstated certainty is a limitation of its own.

E. Correct. Leadership needs the specific failure mode for this use, wrong fees and deadlines stated fluently, and the controls that address it, so the design is understood as part of the proposal.

Communicating what Claude offers, and where it fails, to stakeholders means giving leadership a picture they can make a decision on: what was measured, what it cost to get, and what the failure mode looks like. The pilot measured a large time saving and a 12 percent factual correction rate, and those two numbers belong together, because the review step that catches the corrections is part of the cost of the benefit. The limitation that matters to a college is confident wrong statements about fees and deadlines, which is why controlling sources and a named reviewer are design features rather than optional extras. The distractors overstate the value or understate the limitation: converting minutes into headcount before any process redesign, treating staff satisfaction as evidence of accuracy, and promising that uploading documents removes the need for review, which no configuration can honestly guarantee.

Answer: C. An effective instruction states sources, limits and format.

A. Telling a model to be accurate is not a rule it can act on, and 'never quote an incorrect price' does not say which source is correct. The failures came from missing information, not missing emphasis.

B. Examples convey tone but not authority. Twenty past replies do not tell Claude that the uploaded list overrides its general knowledge or that refunds are outside its remit, so both failures continue.

C. Correct. Each observed failure is closed by a specific rule: a named pricing source, a prohibition on commitments, a defined audience and format, and an escalation path for anything the material does not cover.

D. Consulting the documents is not the same as answering only from them, so general knowledge still competes with the price list. Nothing says what Claude may not commit to, so refunds keep being promised, and no format or escalation rule is set.

Creating effective system-level instructions means writing the rules the outputs must follow, not exhorting the model to be good. Each failure maps to a missing rule. Tone varied because no audience or format was defined. Refunds were promised because nothing said what Claude may not commit to on the team's behalf. Wrong prices were quoted because the instruction never named the uploaded list as the authority, so the model's general knowledge competed with the source. The rewrite supplies each of those: role and audience, a named pricing source, an explicit prohibition on commitments, a format, and an escalation rule for questions outside the material. Emphasis in capitals adds pressure without information. Examples without stated rules teach style but not the boundaries that were crossed. A line about tone and consulting the documents softens the symptom without naming a source, a limit or a format, so the two costly failures continue.

Answer: D. Keep the consequential decision with an accountable person.

A. A disclosure line does not make an automated rejection a documented human decision, and it does nothing about undetected bias in the ranking. Informing applicants is necessary but does not make the use appropriate.

B. This complies with the policy but leaves the screening time, the director's stated aim, untouched: recruiters still read all 600 applications by hand when structured summaries against published criteria would speed that without deciding anything.

C. One approval for 600 outcomes is a rubber stamp. No recruiter can review a full ranking in one step, so the decision is the model's in substance, which the policy and equality law do not permit.

D. Correct. Summaries against published criteria are an aid recruiters can check, bias testing on a sample addresses the hidden failure mode, and every consequential decision remains a documented human one.

Identifying appropriate and inappropriate use cases turns on consequence, detectability of error and where decision authority sits. Rejecting an applicant is a consequential decision with legal exposure, bias risk that is hard to detect after the fact, and an internal policy that requires a documented human decision for each one. Automated rejection is therefore inappropriate regardless of the disclaimer. Summarising applications against published criteria, tested for bias on a sample, is an appropriate augmentation: it makes recruiters faster and more consistent while every shortlist and rejection remains a recruiter's documented decision. Confining Claude to rejection letters satisfies the policy and misses the director's aim, since screening time is untouched. Approving a 600-name ranked list in one step is a human signature on a machine decision, which satisfies the letter of the policy and none of its purpose.

Answer: C and E. Follow the policy's list and its route for additions.

A. A tool built on Claude is not the tool the policy approved, and driver rosters and manifests without names are still classified data. The dispatcher's own filtering of tabs is not the policy's classification step.

B. A vendor's assurance about retention is not an approval, and the policy names a governance owner for exactly that assessment. Continuing while the email is unanswered keeps the data flowing meanwhile.

C. Correct. The approved workspace is the tool the policy sanctions, and classifying the manifest data before entry is the step the policy requires, so the dispatch work continues within the rules.

D. A personal trial is the decision the governance owner is meant to make, taken without them, and a week of customer manifests reaches a third party while it runs. Experience is not authorisation.

E. Correct. The policy provides a route for new tools, and using it puts the extension in front of the person with authority to assess the vendor and the data flow, while nothing is used before approval.

Following organisational AI policies means using the tools the policy approves and using its stated route when something new looks useful. The extension is not on the list, it ships whole tabs including customer manifests and driver rosters to a third party, and being 'powered by Claude' says nothing about who receives the data or under what terms. The two consistent actions are to do the work in the approved workspace with the data classified as the policy requires, and to put the extension through the governance owner's approval route, treating it as unapproved until listed. The distractors each substitute a private judgement for the policy's mechanism: the dispatcher's own view of which tabs are safe, a vendor's assurance about retention, or a personal trial. None of those is the governance owner's decision, and the roster and manifest data are already flowing while they happen.

Answer: B. Diagnose the source before touching prompt or model.

A. 'Use the correct excess' is the goal, not a rule. Without a way to identify which wording applies to a policyholder, the instruction gives the model nothing new to act on and the conflict remains.

B. Correct. Tracing the failing letters identifies the source conflict, the renamed documents and the inception-date rule give the model a basis for choosing the right wording, and re-testing on the same cases verifies the fix.

C. This restores the old acceptance rate by making letters for 2027 policyholders wrong instead, which the stem rules out: both wordings are needed. It also skips the diagnosis in favour of a rollback.

D. Whatever its capability, the model cannot tell which wording applies to a policyholder; the missing input is a selection rule, not reasoning power. Switching models adds cost and leaves the ambiguity untouched.

Diagnosing an underperforming output starts by locating where the failure enters: wrong or missing fact, wrong reasoning, wrong form, or inconsistency. A wrong excess amount that appeared the week a second policy wording was uploaded points to a source conflict, and tracing failing letters to the document each figure came from confirms or refutes that in minutes. The fix then follows the diagnosis: because both wordings are legitimately in force, the Project needs clearly named documents and a rule that selects the wording by inception date, and re-testing on the failing cases shows whether the rule works. The distractors change something before knowing what is wrong. Prompt emphasis cannot tell the model which document applies. Removing the new wording restores the old numbers and breaks letters for new policyholders, ignoring the stated constraint. A stronger model still has two indistinguishable documents and no rule for choosing between them.