# Team CPX assessor guide v0.1.0 · 2026-10-16 · Framework 0.1.0 Prepare raters, record evidence and rehearse scoring with fictional examples for all eight criteria, connected to KSEC/OBE and baseline, post-training, month 6 and month 12 follow-up. This guide and illustrative answers are project proposals for piloting framework 0.1.0, without validity/reliability testing or endorsement. Each variant is an alternative fictional event, not a real team's result, standardized test or passing standard. It grants no authority over real agencies. https://hadr-cpx.pk-research.work/en/assessor-guide/ ## Framework rating rules - 0 = an observable, planned opportunity existed but the criterion behavior was not demonstrated. It does not stand for missing data, unseen behavior or absent opportunity. - NO = not observed, insufficient evidence or no opportunity: record why and the missing opportunity. NA = inapplicable to the role/scope as declared before the round: record the justification owner. Neither is zero or enters scored denominators. - Assign the highest anchor fully evidenced; level 3 includes level 2. Levels 2/3 require no solution coaching by controllers. Log all prompting. If the task omits a necessary opportunity, use NO with partial evidence rather than automatically awarding a low score. - Report eight criteria grouped as K/S/E/C without a total or cross-criterion mean. Show scored/8, NO/8 and NA/8 and each criterion’s observed opportunities. - Compare phases only for scored, comparable criterion pairs, showing pairs available/expected, roster and method changes. Do not impute missing values, turn NO/NA into zero or calculate percentage retention from ordinal 0–3 ratings. - Report pre-consensus exact agreement by criterion, separating paired 0–3 ratings from NO/NA pairs, with counts and disagreements. During piloting, prespecify ordinal agreement statistics and uncertainty estimates with methodological expertise. ## Before assessment: prepare together - The assessment lead records team code, T0/T1/M6/M12 wave, rubric/scenario versions, raters A/B and a disagreement reviewer. Separate controllers who deliver injects and simulate external agencies from raters where feasible; disclose overlapping roles and conflicts of interest. - Check all eight criterion–OBE–inject–artifact–observation-window links. C1 needs two opportunities; C2 needs a follow-up/recheck window within a comparable simulated interval at every wave. Organizers verify applicable local roles/orders before translating them into exercise documents. - Provide the same accessible criterion briefing to every team. Keep this rater-practice set separate from assessment scenarios A–D. Mark materials exercise only, identify who stops the exercise for danger/real incidents, and verify authorized evidence storage. ## During the exercise: separate observation from interpretation - A/B independently record time, inject/criterion ID, what the team said or did, artifact/version, information recipients, opportunity provided and controller prompting. Replace labels such as weak leader with behavior that can be checked against evidence. - Usable note: 09:12, TC-S1, COP-v2 flags conflicting R1/R2 reports and assigns V to check by 09:18; at 09:17 R3 arrives and COP-v1 is retained. Record this before assigning a score. Sending a message does not establish that its recipient read it. - Record actual prompts, distinguishing rule clarification from solution coaching. Levels 2/3 require no controller solution coaching. If an event was unseen or an artifact is absent, document the gap rather than filling it from impression or memory. ## After observation: rate independently before the AAR - Check opportunity first, then compare evidence with each current anchor. Choose the highest fully evidenced level; level 3 includes level 2. If an essential condition had no opportunity, use NO with partial evidence, not an automatic lower rating. - Preserve dated A/B forms before either rater sees the other's scores or the AAR begins. Mask prior scores and wave where feasible and disclose what could not be masked. Every rating needs evidence references and a rationale separate from participant confidence/satisfaction. - Report the eight-criterion K/S/E/C profile with scored, NO and NA counts. Do not create a total, calculate percentage retention from ordinal 0–3 levels, or derive a passing cutoff from these examples. ## Calibrate raters and recheck - Read each context and evidence variant with its answer closed. Independently note score, rationale and missing conditions before comparing the teaching answer. These answers are not a validated gold standard. - Compare A/B criterion by criterion, separating unequal evidence access, anchor interpretation and missing opportunity. Retain originals and add the reasoned resolution. If unresolved, preserve dissent, refer to the designated reviewer and leave the agreed-score field blank pending resolution. Do not average ratings or settle by rank. - After discussion, use a fresh unseen set spanning 0–3 and NO/NA to recheck agreement before field use. Report scored pairs and disagreements by criterion using independent pre-consensus ratings. Do not claim reliability from a few examples or from post-consensus scores. - Prespecify ordinal analysis, uncertainty estimates and rubric revision rules with methodological expertise before the pilot. Recheck scoring drift with new examples across T0/T1/M6/M12. A change to anchor meaning requires a new version and comparability review before pooling results. ## Longitudinal follow-up and real incidents - Use the existing four-wave forms. Record actual dates, elapsed days, roster/role changes, additional training and scenario versions. Compare only scored criterion pairs with justified comparability; show missing pairs and reasons without replacing missingness with zero. - Keep CPX ratings, organizational improvement actions and real-incident records separate. Real incidents do not follow the month 6/12 schedule; no incident is not failed transfer. Never manufacture an incident for data collection or interrupt life-saving work. - Record incident transfer afterwards when authorized: behavior used, corroboration, adaptations from training, other influences and limits. Distinguish accounts from checked artifacts. Use codes and necessary data only in authorized institutional storage; this website accepts no person-level or incident records. ## 0 / NO / NA ### 0 At 09:10 the simulated receiver is present and hears the request. The team sends a message and marks completion without checking receipt. Both raters observe the full event. TC-S2 = 0: opportunity and observation were complete, but transmission was treated as completion; this is not missing data. ### NO The controller omits TC-C1's second challenge opportunity. One well-handled challenge is documented. TC-C1 = NO, retaining the first observation and naming the missing opportunity. Do not award 1 for a missing task opportunity or 2 from one event. ### NO A complete handover occurs, but rater B loses connection and the recording fails, so B cannot verify read-back. A has separate notes. B uses NO for insufficient evidence; A rates from A's evidence. Preserve the independent difference. If authorized corroboration later becomes available, record a separate review without overwriting B's original. ### NA Before a scoped practice round, the organizer's scope map excludes external communication and TC-E2, recording the justification owner's role. TC-E2 = NA under the predeclared scope. This subset is not a complete eight-criterion assessment and must not be compared as one. A forgotten inject in a full round is NO, not NA. ## Worked disagreement: TC-S1, A rates 3 and B rates 2 - The task provides full opportunity to send corrections and obtain acknowledgment. Without solution coaching, the team retains conflicting reports, provenance, time, status, verifier and timely COP-v2, fully evidencing level 2. - A rates 3 from SENT-09:20 and SITREP-v2; B rates 2 because the log shows sending but no acknowledgment by users of the old version, despite time to obtain it. Preserve A=3/B=2 and their original evidence before discussion. - Review of the designated channel log and observation window finds no acknowledgment, so the agreed score is 2: the last level-3 condition is missing. Record agreed_score=2 and resolution_reason citing SENT-09:20/window. Do not average to 2.5 or replace A's original 3. - This is one disagreeing independent TC-S1 pair, not an instrument reliability estimate. If the task had offered no acknowledgment opportunity, the NO rule would replace the score-2 reasoning in this example. ## TC-K1 · Explain roles and authority boundaries K / EOC-K · The team explains proposal, coordination and approval roles and the authority to verify before a decision. https://hadr-cpx.pk-research.work/en/team-assessment/#TC-K1 Fictional ‘Learning Canal’ CPX, 09:00–09:20 exercise time (ICT): explain the role table, then receive a changed condition at 09:10. Exercise card R, edition 1, clauses 2–4, is organizer-checked for this task only and names proposal P, coordination C, approval A and scope-verification V roles; it grants no real authority. These four project-synthesized examples are alternatives, not a team's progression. ### Example level 0 At 09:04, note K1-0 records ‘desk C can approve because it is in the EOC.’ The team enters C as both coordinator and approver although card R is on the table. Asked to check the roles at 09:10, it repeats the claim and does not consult the card before the window ends. Anchor: Given an opportunity, the team does not distinguish coordinator from approver or claims authority from the EOC label without checking. Rationale: An opportunity and reference were available, but the team claims authority from the EOC label and does not distinguish coordinator from approver: level 0. Boundary: There is not even a partly correct explanation of proposal and approval roles to support level 1. This is not NO: the response to the planned opportunity was observed. ### Example level 1 At 09:05, table K1-1 says ‘P proposes; A approves.’ The team says C forwards the request to A. Asked which card edition supports this and what A may approve, it answers ‘as we remember’ and leaves reference and limits blank through 09:20. Anchor: Identifies some proposal and approval roles but not the plan/order to check or role limits. Rationale: Some proposal and approval roles are identified, but the document to check and role limits are missing: level 1. Boundary: Level 2 needs the complete task explanation, organizer-checked edition/clause and remaining unknowns. Recalling role labels does not fill these gaps. ### Example level 2 At 09:06, table K1-2 cites card R edition 1, clauses 2–4: P proposes options, C coordinates, and A considers approval only within the card's scope. The team explains that cross-area conditions remain unknown and need V's check. At 09:10 the controller moves the task beyond that scope; the team writes ‘pending check’ but neither explains the decision implications nor revises the table by 09:20. Anchor: Explains all proposal, coordination and approval roles in the task, citing organizer-checked edition/clause and remaining unknowns. Rationale: All roles, checked edition/clause and remaining unknowns are explained, meeting level 2. Boundary: The changed-authority opportunity was provided, but decision implications and a justified revised table are absent, so level 3 is incomplete. ### Example level 3 At 09:06, table K1-3 v1 explains P proposing, C coordinating and A approving within card R edition 1, clauses 2–4, and marks cross-area scope as unknown. After the 09:10 out-of-scope inject, the team explains that A cannot yet confirm this decision. At 09:13, v2 adds V's scope check and C's controller-mediated referral to the card's escalation role, with the reason ‘confirm scope before seeking a new decision.’ Anchor: Meets level 2 and, when authority conditions change, explains decision implications, identifies verification/escalation roles and justifies a revised table. Rationale: Level 2 is fully evidenced, followed by decision implications, verification/referral roles and a reasoned table revision under changed conditions: level 3. Boundary: The card, table versions and explanation record were checked. The score covers this fictional explanation task only; it establishes neither legal authority nor real-incident readiness. ## TC-K2 · Explain what an evidence item establishes K / CPX-K · The team links outcome, inject and evidence, distinguishing exercise ratings, self-report and incident evidence. https://hadr-cpx.pk-research.work/en/team-assessment/#TC-K2 Fictional ‘Three Folders’ CPX, 10:00–10:20 ICT: link the acknowledged-handover outcome to O (exercise read-back log), S (confidence response) and I (an entirely invented incident-record facsimile). A conflicting-evidence card arrives at 10:12. All materials are project fiction; the four levels are independent alternatives. ### Example level 0 At 10:05, map K2-0 draws an arrow from S ‘confidence 5/5’ to ‘handover already succeeds in real incidents.’ The team repeats this conclusion when asked to explain O and I and makes no outcome links for those folders. Anchor: Given examples, makes no evidence–outcome link or claims confidence/exercise scores prove incident effectiveness. Rationale: Examples were provided, but confidence is treated as proof of incident effectiveness: level 0. Boundary: No appropriate evidence-type/outcome link supports level 1; an arrow making an unsupported inference is not a valid link. ### Example level 1 At 10:07, map K2-1 links the handover inject and O's 08:42 receiver read-back to acknowledged receipt in the task. It places S ‘I can hand over’ under direct observation, and all three limits fields remain blank at 10:20. Anchor: Links one evidence type to an outcome but still confuses observation with report or omits limits. Rationale: One observation type is linked, but report is confused with observation and limits are omitted: level 1. Boundary: The complete map and three-way distinction with limits required for level 2 are missing. ### Example level 2 At 10:10, map K2-2 links the outcome to each event and artifact: O's CPX read-back supports behavior in that task only; S's post-training response reports confidence, not observation; I's fictional incident handover log supports recorded actions, not training causality. At 10:12 a card says I lacks acknowledgment; the team attaches it without identifying which conclusion must be withheld. Anchor: Completes the task’s outcome–inject–artifact links and distinguishes observation, self-report and incident evidence with limits for each. Rationale: The task links and distinctions among observation, self-report and incident evidence, each with limits, meet level 2. Boundary: After conflicting evidence, neither the unsupported conclusion nor question-specific additional evidence is identified, so level 3 is incomplete. ### Example level 3 At 10:10, map K2-3 links O to the CPX read-back event, limited to that task; S to post-training confidence, not observed behavior; and I to recorded handover in the fictional incident, without establishing training causality. When the 10:12 card reveals a blank receipt field, the team writes at 10:15 ‘receiver acceptance cannot yet be concluded’ and requests a contemporaneous acknowledgment record from the controller's receiver role to establish which task was accepted and when. Anchor: Meets level 2 and, with conflicting or missing evidence, identifies unsupported conclusions and proposes additional evidence for the specific question. Rationale: Level 2 is complete; the evidence gap leads to a withheld conclusion and a specific evidence request answering the outstanding question: level 3. Boundary: The map, stated limits and additional request were checked. This score assesses reasoning about fictional evidence, not actual transfer or causal effects. ## TC-S1 · Update the situation picture with a verification trail S / EOC-S · The team updates its COP while preserving provenance, time, versions and uncertainty. https://hadr-cpx.pk-research.work/en/team-assessment/#TC-S1 Fictional ‘Duplicate Count’ CPX, 09:00–09:20 ICT: COP v1 has 40 people waiting; a 09:02 report gives 55 at the same site. An evidence-based update is due by 09:12 and notification to old SITREP users by 09:18. Controllers simulate available verification sources. These four synthesized examples are separate alternatives. ### Example level 0 At 09:04, COP S1-0 changes the total to 95 by adding 40 and 55. The team deletes the original report row and says it has combined both reports. No check is requested although both reports name the same site and verification remains available until 09:12. Anchor: Given conflicting reports, does not update/check, double-counts or deletes prior information without a trail. Rationale: Conflicting reports were available, but the team double-counts and removes prior data without a trail: level 0. Boundary: Changing the total to 95 does not support level 1: it is the double-counting behavior explicitly described at level 0. ### Example level 1 At 09:06 the team keeps both sheets and writes ‘40 or 55, uncertain’ in COP S1-1. The row has no reporting source, data time, verification status or verification owner, and no check is requested by 09:12. Anchor: Updates figures or notes doubt but omits provenance, time, verification status or verification owner. Rationale: Doubt is recorded, but provenance, time, status and verification ownership are missing: level 1. Boundary: Level 2 requires the complete trail, assigned verification and a timely evidence-based version. Keeping the sheets alone does not establish these. ### Example level 2 At 09:05, file S1-2 preserves A's 08:55 count of 40 and B's 09:02 count of 55 with sources and ‘conflicting, pending’ status, assigning V1 to verify. At 09:09 the controller confirms that 55 includes the original 40. COP v2 at 09:11 gives 55, links V1's reply and retains v1. The earlier SITREP still says 40 and its users are not notified by 09:18. Anchor: Preserves conflicting reports, records provenance/time/status, assigns verification and versions the update on evidence within the specified time. Rationale: Conflicts, provenance, time, status, verification owner and an evidence-based version before the deadline meet level 2. Boundary: The correction is not traced into the SITREP/decision or notified and acknowledged by old-version users, so level 3 is incomplete. ### Example level 3 At 09:05, S1-3 retains A (08:55, 40) and B (09:02, 55), their sources, conflicting status and verifier V1. After the controller confirms at 09:09 that 55 includes the earlier count, COP v2 at 09:11 records 55, links the reply and preserves v1. At 09:14 the team corrects the SITREP and decision-support table from 40 to 55, sends the change to both v1 user roles and records both receipts by 09:17. Anchor: Meets level 2 and traces the correction into the SITREP/decision, notifies users of the old version and confirms receipt of the correction. Rationale: Level 2 is complete, and the correction reaches decision documents with notification and acknowledgment from earlier users: level 3. Boundary: COP versions, verification reply, SITREP and timed receipts were checked. The score covers this dataset's handling, not other COPs or real decision quality. ## TC-S2 · Close the loop on task coordination S / EOC-S · The team coordinates requests and handovers with acknowledged ownership, status, deadlines and follow-up. https://hadr-cpx.pk-research.work/en/team-assessment/#TC-S2 Fictional ‘Coordination Handover’ CPX, 11:00–11:25 ICT: send a simulated transport-plan request to L and follow up at 11:10. At 11:12 L lacks resources; the exercise agreement permits controller-mediated referral to L2, with handover at 11:20. No real agency is contacted. The four synthesized examples are independent alternatives. ### Example level 0 At 11:03 the team posts ‘transport plan requested’ to a general message channel without naming a receiver, then marks S2-0 ‘done’ because the message was sent. It does not check receipt at 11:10, and the 11:20 handover omits the task. Anchor: Given a handover task, fails to send/identify a receiver or treats transmission as task completion without checking receipt. Rationale: A handover task was available, but no receiver is identified and transmission is treated as completion without checking receipt: level 0. Boundary: Level 1 at least requires sending to an identified role. A ‘done’ label is not evidence of acceptance. ### Example level 1 At 11:03, request S2-1 goes to role L asking for a transport plan. Status, response deadline and constraints are blank. There is no acknowledgment, and at 11:10 the team does not revisit the request despite the offered follow-up opportunity. Anchor: Sends to an identified role but lacks acknowledged receipt or status/follow-up time. Rationale: The request reaches an identified role, but acknowledgment and follow-up information are absent: level 1. Boundary: Owner, status, deadline, constraints, accurate read-back and scheduled follow-up required for level 2 are not all evidenced. ### Example level 2 At 11:03, S2-2 names L as owner, status ‘draft pending,’ response time 11:10 and the constraint to use simulated vehicle data only. L accurately reads these back at 11:04. The team follows up at 11:10 and records the draft still open. After L reports insufficient resources at 11:12, it retains the open status at 11:20 handover but does not use the agreed L2 route. Anchor: Sends owner, status, deadline and constraints; receiver accurately reads back/acknowledges and the team follows up as scheduled. Rationale: The complete request, accurate read-back and scheduled follow-up meet level 2. Boundary: Resource failure has not led to rerouting and confirmed replacement ownership, so level 3 remains incomplete despite retaining open work at handover. ### Example level 3 At 11:03, S2-3 gives owner L, ‘draft pending’ status, a 11:10 deadline and the simulated-vehicle-data constraint. L reads back correctly at 11:04; the team follows up at 11:10 and records the draft open. Following the 11:12 resource failure, it uses the agreed controller-mediated route to L2. L2 reads back acceptance of ownership at 11:15 with a 11:25 deadline. The 11:20 handover retains the open task, L2 and the new deadline, accurately read back by the incoming desk. Anchor: Meets level 2 and, when contact/resources fail, reroutes coordination under the exercise agreement, confirms replacement ownership and retains open work at handover. Rationale: Level 2 is complete; resource failure triggers agreed rerouting, confirmed replacement ownership and retained open work at handover: level 3. Boundary: The request, read-backs, follow-up and handover were checked. The score covers this simulated coordination loop; it does not mean the draft was completed or transport occurred. ## TC-E1 · Justify allocation with attention to those still waiting E / EOC-E · The team compares options and consequences for hard-to-reach groups without treating convenience as need. https://hadr-cpx.pk-research.work/en/team-assessment/#TC-E1 Fictional ‘Two Waiting Sites’ CPX, 13:00–13:20 ICT: 60 simulated water kits; accessible site A has 60 waiting, hard-to-reach B has 40 and reports dwindling reserves, with needs still to verify. The controller's B representative challenges the choice at 13:10. The four project alternatives teach reasoning, not real allocation orders. ### Example level 0 At 13:04, table E1-0 assigns all 60 kits to A ‘because it is nearer,’ although the team received B's card. Waiting-group and adverse-consequence fields are blank. At the 13:10 challenge, the team only says the delivery plan is complete and never discusses B before the window ends. Anchor: Given evidence of an unreached group, omits those waiting and the adverse consequences of allocation. Rationale: An unreached group is documented, but those waiting and adverse allocation consequences are omitted: level 0. Boundary: Level 1 at least requires mentioning those waiting; saying the delivery plan is complete does not provide that evidence. ### Example level 1 At 13:05, E1-1 records ‘40 at B still waiting’ and proposes all 60 kits for A, explaining ‘everyone will receive them in queue order, so it is fair.’ It gives no second option or comparison of the consequences of B's dwindling water, including after the 13:10 challenge. Anchor: Mentions those waiting but offers one option or claims fairness without comparing consequences. Rationale: Those waiting are mentioned, but only one option is offered and fairness is claimed without comparing consequences: level 1. Boundary: Two options assessed by need/access, reasons, uncertainty and assigned follow-up are not complete for level 2. ### Example level 2 At 13:07, E1-2 compares A60/B0, faster but leaving B40 waiting, with A20/B40, reducing B's water-depletion risk but leaving A40 waiting and using B's slower route. It selects the latter, flags B's reserves as unverified and assigns F1 to follow up both waiting groups at 13:15. When B's representative reports the access route closed at 13:10, the team records the challenge but does not check it or revisit the choice within the window. Anchor: Compares at least two options by need/access, identifies recipients and those waiting, explains reasons/uncertainty and assigns follow-up. Rationale: Two need/access options, recipients and waiting groups, reasons, uncertainty and assigned follow-up meet level 2. Boundary: The challenge was received, but evidence checking, a revised or justified choice, mitigation and a review time are missing for level 3. ### Example level 3 At 13:07, E1-3 compares A60/B0 with A20/B40, identifies B40 or A40 left waiting, explains B's dwindling water versus slower access, flags unverified reserves and assigns F1 to follow up both sites at 13:15. After B challenges the closed route at 13:10, the team requests the controller's map and verifies an alternate exercise route at 13:12. It retains A20/B40 with reasons, adds waiting-time updates and a simulated-role check of A's temporary reserves as mitigation, and schedules route/waiting-group review at 13:18. Anchor: Meets level 2 and, after a group representative challenges it, checks evidence and revises or justifies the choice with mitigation and a review time. Rationale: Level 2 is complete, followed by checking the challenge, justifying the retained choice, mitigation and a review time: level 3. Boundary: The option table, fictional map and review schedule were checked. The score covers demonstrated reasoning; it does not establish optimal allocation or actual beneficial outcomes. ## TC-E2 · Communicate within evidence and data limits E / EOC-E · The team routes simulated messages through agreed review without overstating success or disclosing unnecessary data. https://hadr-cpx.pk-research.work/en/team-assessment/#TC-E2 Fictional ‘Reviewed Bulletin’ CPX, 14:00–14:20 ICT: evidence confirms 30 kits at the receiving point, with 20 pending. Simulated review and approval must precede release to two simulated recipient roles. An excessive request for names arrives at 14:12. All artifacts are exercise-only, with no real names. The four synthesized levels are separate alternatives. ### Example level 0 At 14:05, E2-0 releases ‘all 50 kits delivered; everyone has received them’ to the exercise inbox without checking the receipt for 30, despite a 14:00–14:05 review window and the other 20 explicitly marked pending. Anchor: Given a review opportunity, releases unsupported success claims or unnecessary identifying data. Rationale: A review opportunity existed, but unsupported success claims were released: level 0. Boundary: There is no partial correction or data reduction supporting level 1; an exercise-only label does not substantiate the claim. ### Example level 1 At 14:06 the team changes draft E2-1 from ‘all 50’ to ‘30 kits reached the receiving point’ and removes fictional household codes. The issued message omits the data time and the 20 still pending, with no record of agreed review/approval before release. Anchor: Reduces data or corrects some claims but omits uncertainty/time or the agreed review step. Rationale: Some claims and data are corrected, but uncertainty, time and agreed review are omitted: level 1. Boundary: Level 2 requires verified/pending separation, timestamp, necessary data only and recorded review/approval before release, all together. ### Example level 2 At 14:07, E2-2 states ‘as of 14:00, 30 kits confirmed at the receiving point; 20 pending; not a household-delivery count,’ with no identifying data. R records a receipt check at 14:05 and A approves at 14:06 before release at 14:07. After the 14:12 names request, the team holds it but provides no aggregate alternative, recorded reply reason or receipt. Anchor: Separates verified/pending facts, timestamps the message, limits data and records simulated review/approval before release. Rationale: Verified/pending facts, time, limited data and review/approval before release meet level 2. Boundary: Holding the request alone is insufficient for level 3; this opportunity needs an aggregate alternative with reasons and receipt. ### Example level 3 At 14:07, E2-3 releases the 14:00 position: 30 kits confirmed at the receiving point, 20 pending, no identifying data or claim about all households, after R's 14:05 receipt check and A's 14:06 approval. After the 14:12 names request, the team sends an aggregate count at 14:15 through the same review process, recording ‘names exceed this recipient's scope and are unnecessary to answer the count question.’ The requester acknowledges the summary at 14:17. Anchor: Meets level 2 and, on error or an excessive-access request, sends a correction to prior recipients or offers an aggregate alternative, recording reasons and receipt. Rationale: Level 2 is complete; an excessive-access request receives an aggregate alternative, recorded reasons and acknowledged receipt: level 3. Boundary: The issued message, review/approval trail, aggregate reply and acknowledgment were checked. The score covers this simulated message, not legal data compliance or system-wide real communications. ## TC-C1 · Hear challenges and explain revisions C / EOC-C · The team invites and hears dissent, checks evidence and preserves reasons when revising decisions. https://hadr-cpx.pk-research.work/en/team-assessment/#TC-C1 Fictional ‘Listen Twice’ CPX, 15:00–15:30 ICT, with two planned opportunities: at 15:05 controller role Q challenges a duplicate report count; at 15:15 role R requests decision review after a route change. Evidence cards are available for both. These four project alternatives describe neither personality nor real personal histories. ### Example level 0 Note C1-0 records the team cutting Q off at 15:05 with ‘already decided, no changes,’ without opening the duplicate-count card. At 15:15, when R offers the route card, the team moves to the next agenda item without hearing R or inspecting it. Both observation records and the unopened cards are retained. Anchor: Given a challenge, blocks or ignores it and does not examine its reasoning/evidence. Rationale: Challenges were available at both opportunities, but the team blocks or ignores them without examining reasoning/evidence: level 0. Boundary: No listening, even after prompting, or revision supports level 1. This concerns two observed windows, not personal worth. ### Example level 1 At 15:05 the team does not respond to Q until the controller prompts at 15:07 to hear the information. It then reads the card and changes the count in C1-1 without recording why. At 15:15 it hears R and switches to an alternate route without checking the map or recording a reason. Anchor: Listens after controller prompting or changes without recording reasons; consistent behavior across two opportunities is not shown. Rationale: Listening follows prompting and revisions lack recorded reasons; complete, consistent behavior across both opportunities is not shown: level 1. Boundary: Level 2 requires inviting challenge, checking evidence and recording reasons without blame at both opportunities. Changing a decision alone is insufficient. ### Example level 2 At 15:05 the team invites Q's explanation, compares both report cards and records in C1-2 a change from 70 to 50 because batch codes overlap, without blaming anyone. At 15:15 it invites R's evidence, checks the new map and records why route B is retained: it avoids the closure, again without personal blame. By the end, it has neither applied the learning to related work tables nor invited Q/R to check shared understanding. Anchor: At both opportunities, invites challenge, checks evidence and records why it changes or retains the decision without personal blame. Rationale: Both opportunities show invitation, evidence checks and recorded reasons for revising/retaining decisions without blame: level 2. Boundary: Related-work checks, challenger confirmation of shared understanding and confirmed follow-up of unresolved concerns required for level 3 are absent. ### Example level 3 At 15:05, C1-3 records the team inviting Q, checking both cards and changing 70 to 50 for duplicate batch codes without blame. At 15:15 it invites R, checks the map and records retaining B because it avoids the closure, without blame. At 15:20 it applies the learning to related packing and route tables, correcting one table still using 70. Q/R are invited to read back the reasoning and both confirm understanding at 15:24. B's capacity remains unresolved; V2 accepts the check, and a 15:28 follow-up records a pending reply with review at 15:35. Anchor: Meets level 2 and applies the learning to related work, invites the challenger to check shared understanding and confirms unresolved concerns are followed up. Rationale: Level 2 is met at both opportunities, followed by related-work checks, challenger verification of understanding and evidence that unresolved concerns are followed up: level 3. Boundary: Both opportunity notes, corrected tables, read-backs and follow-up were checked. The score concerns this window's behavior, not enduring personality, and does not mean the capacity question is resolved. ## TC-C2 · Own improvements through recheck C / CPX-C · The team converts feedback into owned actions with evidence and rechecks, disclosing unfinished work. https://hadr-cpx.pk-research.work/en/team-assessment/#TC-C2 Fictional ‘Correct and Recheck’ CPX uses a standardized 16:00–16:30 ICT simulated interval: at 16:00 feedback identifies a missing response-deadline field on the handover form; follow-up is scheduled at 16:15 and a recheck task offered at 16:20. Every level has the same opportunities. This is CPX work, not actual month 6/12 action closure; the four synthesized examples are separate alternatives. ### Example level 0 At 16:03 the team writes ‘fixed, closed’ in C2-0 but attaches only the unchanged form without a deadline field. At the 16:15 review there is no corrective action or new evidence, and the team repeats that it is closed without checking. Anchor: Given feedback and a follow-up window, creates no action or claims closure without evidence. Rationale: Feedback and a follow-up window existed, but closure is claimed without evidence: level 0. Boundary: A closure label does not establish a usable improvement entry for level 1 and cannot replace evidence of correction. ### Example level 1 At 16:04, C2-1 lists ‘add response-time field to handover form’ but leaves owner, due time and verification evidence blank. At 16:15 the team continues another task and does not open the register despite the provided review window. Anchor: Lists improvements but omits owner, due date, verification evidence or the offered follow-up. Rationale: An improvement is listed, but owner, deadline, evidence and offered follow-up are missing: level 1. Boundary: The complete action specification and scheduled, accurate closed/open/blocked status review required for level 2 are missing. ### Example level 2 At 16:04, C2-2 assigns D1 to add the deadline field by 16:12, evidenced by form v2, and D2 to distribute it with receipts by 16:15. At the 16:15 review, the team checks v2, closes the evidenced field-addition action, records distribution as open and identifies one unreachable simulated desk as blocked, matching the logs. It does not use the fresh 16:20 handover task to test whether the correction works. Anchor: Assigns owner, deadline and evidence, follows up on schedule and accurately distinguishes closed, open and blocked actions. Rationale: Owners, deadlines and evidence, scheduled follow-up and accurate distinctions among statuses meet level 2. Boundary: Checking that a new field exists is not testing effectiveness. The recheck task and retained result needed for level 3 are absent; this closure is not between-month follow-up. ### Example level 3 At 16:04, C2-3 assigns D1 to add a deadline field by 16:12, evidenced by form v2, and D2 to distribute it with receipts by 16:15. At the 16:15 review it checks v2, closes the field-addition action, and accurately records distribution as open and the unreachable desk as blocked. At 16:20 the team uses the fresh handover task and finds the new field still blank. It retains test artifact T2 and reopens the correction at 16:23, explaining ‘field exists but is not completed,’ assigning D1 to add instructions and an example, and scheduling another review at 16:30. Anchor: Meets level 2 and tests the correction with a recheck task, keeps evidence and reopens ineffective actions with reasons, owner and a new review time. Rationale: Level 2 is complete; the correction is tested with retained evidence, and failure leads to reopening with reasons, owner and a new review time. This meets level 3 even though the correction is not yet effective. Boundary: The register, scheduled review, form v2 and T2 were checked. The score measures ownership through recheck in the simulated interval, not effective closure or month 6/12 retention. ## Rater practice record Team/wave/scenario version/rubric version/criterion/rater code: __________ Provided and missing opportunities/time/artifact IDs/behavior/prompting: __________ Independent A rating and reason/B rating and reason/form lock times: __________ Disagreement/additional evidence/resolution or unresolved/reviewer/date-time: __________ Improvement/owner/due date/recheck evidence/fresh set used: __________ ## Evidence and limits The guide uses the framework's existing rationale: WHO SimEx supports role practice and gap identification; WHO AAR supports review for learning. Their Overview pages were rechecked on 16 October 2026, without claiming additional PDF review in this round. Research and Thai-plan locators/access limits remain in the framework. Guide procedures and example scores are project syntheses, not tested findings from those sources. ### swiss-pharmacy-ttx-followup-2026 · Le Bloc’h et al. (2026). Tabletop Exercise for Training and Improving Business Continuity Plan Development in Hospital Pharmacies https://doi.org/10.1017/dmp.2026.10424 Methods: Hypothesis and Outcomes; Data Collection and Analysis; Results Table 3; Limitations. An example of later organizational team observation: five hospital pharmacies underwent FSX 7.4–10.2 months after TTX. No baseline FSX: not proof of retention or causality, and not Thai teams, real incidents or a 12-month measurement. ### pediatric-virtual-ttx-followup-2019 · So et al. (2019). Extending the Reach of Pediatric Emergency Preparedness: A Virtual Tabletop Exercise Targeting Children’s Needs https://journals.sagepub.com/doi/pdf/10.1177/0033354919849880 Methods: Data Sources and Data Collection / Data Analysis; Results: Follow-up Actions; Table 3; Limitations and Future Directions. Fourteen matched respondents reported state progress at months 1 and 6, distinguishing intentions from later reported action. Participant reports, not observed six-month team behavior; no 12-month or real-incident outcome. ### manchester-exercise-incident-transfer-2020 · Skryabina et al. (2020). The role of emergency preparedness exercises in the response to a mass casualty terrorist incident: A mixed methods study https://pmc.ncbi.nlm.nih.gov/articles/PMC7709486/ §3.1.1–3.1.2; §4.1.2/Table 2; §4.2; §5.2 Limitations. Accounts of using rehearsed plans, roles and coordination during a real incident within two months of exercises. Retrospective self-report, not independent live-team scores or 6–12-month retention evidence. ### california-clinic-exercise-transfer-2010 · Fowkes et al. (2010). Exercises in Emergency Preparedness for Health Professionals in Community Clinics https://pmc.ncbi.nlm.nih.gov/articles/PMC2952103/ Methods: Evaluation; Table 2; Results: Effects on CHCs Emergency Plans / Post Disaster Interviews; Discussion. Three-month plan follow-up and seven post-wildfire interviews motivate separating organizational improvement from incident-use evidence. Exercise-to-wildfire interval unspecified, with attrition and concurrent assistance; not a 6/12-month outcome. ### who-simex-2017 · WHO (2017). WHO Simulation Exercise Manual, WHO/WHE/CPI/2017.10 https://www.who.int/publications/i/item/WHO-WHE-CPI-2017.10 WHO publication landing page, Overview paragraphs 1–4 (the portion inspected in this pass). Supports exercises as a way to practice roles and examine gaps in systems, procedures and mechanisms. Does not endorse this KSEC rubric, follow-up schedule or cutoffs; PDF access failed in this pass. ### who-aar-2019 · WHO (2019). Guidance for after action review (AAR), WHO/WHE/CPI/2019.4 https://www.who.int/publications/i/item/WHO-WHE-CPI-2019.4 WHO publication landing page, Overview paragraphs 1–3 (the portion inspected in this pass). Supports reviewing response for learning and improvement, with several review formats including interviews. AAR alone is not causal evidence; this incident form is a project proposal. PDF access failed in this pass. ### ddpm-plan · กรมป้องกันและบรรเทาสาธารณภัย. แผนการป้องกันและบรรเทาสาธารณภัยแห่งชาติ พ.ศ. 2564–2570 https://www.disaster.go.th/upload/download/file_attach/3753867497001f8e1303362092706df1.pdf Printed p.43, Figure 3–2 (PDF p.54); printed p.124, items 5.1–5.5 (PDF p.135). Provides Thai mechanism context and information, option-development, coordination and operational-follow-up functions. Inspected saved original-page images, not current local orders. Responsible officials must verify applicable plans and actual authority before exercise use. ## Continue with recording forms https://hadr-cpx.pk-research.work/en/team-assessment.md https://hadr-cpx.pk-research.work/en/team-assessment.csv ## Calibration and recheck kit Practise fresh cases, rate independently using forms A/B, then check agreement before discussion. Uses the existing framework rating rules. https://hadr-cpx.pk-research.work/en/assessor-calibration/ https://hadr-cpx.pk-research.work/en/calibration-questions.md https://hadr-cpx.pk-research.work/en/calibration-key.md https://hadr-cpx.pk-research.work/en/calibration-a.csv https://hadr-cpx.pk-research.work/en/calibration-b.csv https://hadr-cpx.pk-research.work/en/calibration-review.csv https://hadr-cpx.pk-research.work/data/assessor-calibration.json ## Plan a team pilot Define readiness, teams and baseline, post, 6- and 12-month follow-up. Use independent rater forms to preserve evidence before review. This plan is not validated. https://hadr-cpx.pk-research.work/en/team-pilot/ https://hadr-cpx.pk-research.work/en/team-pilot.md Copyright 2026 PK-Research. Original source rights remain with their owners.