How we check every question

Before a question goes live

New questions are made by our generator, which runs every hour. A draft has to clear every step below; one miss and it's thrown out.

1. 1. Real source text

   Questions are written from source text we store: current federal regulations pulled from the official eCFR, and official U.S. government documents such as OSHA, NIOSH and FEMA publications. The writer is given one excerpt and told to use nothing else.

2. 2. A word-for-word quote

   Each question must carry a short quote that decides the answer. Code checks that the quote appears in one paragraph of the stored source, word for word (ignoring only spacing and curly quotes). If it doesn't, the draft is thrown out. The citation and link you see come from the paragraph where code found the quote, never from what the writer claimed.

3. 3. An independent blind solve

   A separate AI model, which is not told the intended answer, solves the question from the source. It must pick the keyed answer with high confidence, find exactly one defensible option and report no problems. Grounded questions are then solved a second time without the source, and that answer must agree too.

4. 4. An explanation check

   Another pass reads each AI-written explanation against the source. It must support the keyed answer, say briefly why each wrong option is wrong, and contain no claim the source or question doesn't support.

5. 5. Math done by code

   Calculation questions come from templates in code: the numbers are drawn and the answer is computed, not written by an AI. When a scenario is reworded, every number must survive unchanged. The blind solver also writes out its own arithmetic, and code checks it lands on the keyed answer (within 2%, or the option's rounding).

6. 6. No near-duplicates

   Each draft is compared with every question already in that exam's bank. A written question that is 80% or more similar to an existing one is rejected, and so is a calculation variant that matches an existing one almost exactly.

7. 7. Format rules

   Four distinct options, one key, no "all of the above" or "none of the above", sensible lengths, no references to option letters, and no mention of BCSP. Anything else is rejected.

8. 8. Tagged to the exam outline

   Every question is tagged with the BCSP exam outline it was written for (today ASP11, CSP11 and CHST5). If two outlines are ever live at once, your exam date decides which one you practice.

Since the hourly generator started, it has made 209 drafts and rejected 52 of them (25%). Rejected drafts are kept with their reasons and never shown to anyone.

After it goes live

• Nightly regulation check

  Each night we compare the regulation sections our questions cite with the eCFR. If a section changed, every affected question's quote is found again: a moved paragraph gets its citation updated, a quote that no longer exists pulls the question, and a changed paragraph sends the question back for a full review.

• Weekly BCSP blueprint watch

  Every week we fetch BCSP's exam blueprints and reference lists, the Current Changes Index, the ASP, CSP and CHST pages, exam development panels and BCSP news, plus public news searches (20 sources in all). Blueprint PDFs are compared field by field. Any change produces a plain-English report of the questions and pages it affects, and those go into a review queue. Nothing is rewritten without a check.

• Your problem reports

  Every question has a Report a problem link. Three reports from different people pull a question from practice, and a single report that it might be real exam content pulls it at once. Pulled questions stay out of practice until they're checked, then fixed or retired. Nothing is deleted, so every change is traceable.

The first questions

The questions written before the hourly generator went through an earlier version of the same idea: the word-for-word quote check, then a second AI answering from the quote alone, with any disagreement thrown out. Calculation answers were computed by code. On Oct 16, 2026, a reviewer AI re-read 2,583 questions against the full surrounding regulation text, looking for wrong keys, second defensible answers and explanations the rule doesn't support. Questions that failed were pulled.

What this can't promise

Checks catch most problems, not all. If an answer looks wrong to you, use Report a problem on the question and we'll look at it. Our questions are original, written from public regulations and government documents, and never from real exam content. No one can guarantee you a pass, and we don't.

The tutor is an AI that explains a missed question from that question's cited source.

Terms: how questions are made