Skip to main content

Question bank — operator guide

Goal: Manage the full question bank catalogue — create, edit, review, import, and retire questions.

Who can do this: Platform administrators and Directory administrators (full operator access). Company administrators and staff can browse and insert questions into assessments but cannot push questions through the review pipeline or use bulk operations.

/question-bank

Before you start​

  • Confirm you can see Question Bank in the sidebar. If it's missing, you don't have question bank access — contact a Platform admin.
  • Operators see four tabs: Question List, Review Queue, Usage Dashboard, and Item Analysis. Browse-only users see the same tabs but cannot perform mutations.

A. Browse and filter the question list​

  1. Sidebar → Question Bank to land on the Question List tab.
  2. Use the search bar to find questions by keyword.
  3. Open the Filters panel to narrow by job family, parent skill, primary subskill, skill type, format, cognitive level, status, tooling tags, or source. Parent skill and primary subskill are dropdowns of the distinct values already in the bank (with counts), not free text.
  4. Adjust the sort order (default: most recently updated first).
  5. Select rows using checkboxes when you need to perform a bulk action.

The Question List table itself now also shows Parent Skill and Subskill as columns, next to Skill Type, so you can scan taxonomy without opening each row.

B. Create a new question​

  1. Click Create or Add (top of the page) to open the creation form.
  2. Fill in the required fields:
    • Parent skill labels — attach to the taxonomy
    • Format — multiple choice (MCQ), scenario, short answer, or open-ended (OEQ)
    • Level — difficulty tier (L0–L6, or N/A)
    • Source — where this question originated
    • Industry context and tooling tags — optional but recommended
  3. Enter the question text and answer options or rubric depending on the format.
  4. Save — the new row becomes draft status by default unless you select another status before saving.
  5. Fix any inline validation errors before retrying if saving fails.

C. Edit a question or change its status​

  1. Click Edit on a question row to open the edit form.
  2. Update the content or change the status (available transitions: draft → review → live unvalidated → live validated → retired). The flagged status is set automatically by the system.
  3. Click Save.

To delete a question permanently, use the Delete action on the row and confirm in the dialog. This cannot be undone.

Bulk actions: Select multiple rows → choose a target status or bulk retire in the action bar → submit. Watch for row-level errors if the operation partially fails.

D. Work the Review Queue​

The Review Queue surfaces questions that are ready for human approval before going live.

  1. Click the Review Queue tab (or go to /question-bank/review-queue).
  2. Page through the list — up to 50 items per page.
  3. For each item, click Approve or Reject to progress or remove it from the queue.
  4. If the queue is empty, No items in queue is shown — nothing to action.

E. Read the Usage Dashboard​

  1. Click the Usage Dashboard tab.
  2. Review the cards showing question distribution by job family and level:
    • Live unvalidated — by job family
    • Live unvalidated — by level
    • Flagged — by job family
    • Flagged — by level
  3. Use these counts to schedule content clean-up or validation sprints.

F. Import questions from CSV​

For the full import walkthrough including the complete column reference, accepted values, and common error fixes, see the dedicated guide:

→ Import questions into the Question Bank (CSV)

G. Item Analysis tab​

Psychometric quality data appears here after live assessments have real candidate response data. Full walkthrough: Question bank — Item Analysis.

  1. Click the Item Analysis tab.
  2. Select an assessment from the dropdown and click Run Analysis to compute metrics.
  3. Use verdict filter chips (Bank It, Monitor, Review, Remove) to triage the results.
  4. Click any row for a detail modal showing p-value, D-index, and distractor breakdown.

Common failure modes​

ProblemCheck
Forbidden when trying to saveYour session role is browse-only — MCA is not an operator role
Filters return empty resultsYou have an over-restrictive filter combination — click Reset filters
Import partially succeededDownload the error report from the summary — fix the flagged rows and re-import
Item Analysis tab is emptyNo CTT analysis has been run yet — select an assessment and click Run Analysis