check-ride.ai vs MockDPE: 98 vs 65 and the full trial transcript

In the sessions we reviewed, check-ride.ai scored 98/100 and the MockDPE trial excerpt converted to 65/100. Both used synthetic Instrument Rating candidates. The numbers describe these observed runs; the scoring inputs and session lengths are listed below.

How we ran the evaluation

Our voice harness sends a fixed synthetic pilot through the real check-ride.ai oral pipeline, saves the conversation and ACS assessment events, and has an LLM judge inspect examiner behavior. We used the same gpt-5.4 judge for both session reviews. The score is calculated from weighted checks for factual errors, missed or unsupported grading, repeated questions, premature completion, tool or reasoning leaks, and turn taking. Serious failures can cap the score. It is a quality score for the examiner's behavior in that run, not the candidate's test grade.

The 98 came from a completed Instrument Rating run on August 17, 2026: Gemini 3.7 Flash as examiner, a synthetic C172RG/KCWI candidate, 29 examiner turns and 29 candidate answers. The configured oral scope completed with 38 ACS elements evaluated. The judge found one isolated error about lost-communications approach timing; the other scored conduct checks were clean. Run ID: 6d981f7f-1322-4f87-a503-c7e9d350c4d0.

For MockDPE, we used a fixed synthetic C172/G1000 Instrument candidate answering in its text trial. The scenario ran from KLEX to KSDF, with KCVG as the alternate. The candidate's answers were generated with gpt-4o-mini from that profile and the visible scenario. We captured 10 confirmed answers and 11 examiner turns. The brand-blind transcript review rated question quality 4/5, responsiveness 3/5, assessment quality 3/5, factual accuracy 4/5, and continuity 4/5. Converting each rating with (rating − 1) × 25 and averaging gives 65/100. The competitor harness treated this as a diagnostic excerpt and did not issue an official ranking score.

What the transcript showed

The clearest miss came in the weather scenario. The synthetic candidate attributed a 3-mile visibility and light-drizzle forecast to destination KSDF, but that forecast belonged to alternate KCVG. MockDPE probed a separate ceiling point and moved on without correcting the airport mix-up. It also did not probe the candidate's blanket claim that every lapse in instrument currency requires an instrument proficiency check. The check-ride.ai run's confirmed factual issue was the single lost-communications timing error noted above.

Access note: We only had MockDPE's anonymous trial, so that is what we evaluated. It stopped at the account gate after ten confirmed candidate answers. We did not see a full paid oral or its debrief.

Full captured MockDPE transcript and scenario

Departure METAR:

SPECI KLEX 220039Z 02013KT 10SM OVC013 21/18 A3000 RMK AO2 T02060178 $

Destination TAF:

TAF KSDF 212326Z 2200/2306 01010G17KT P6SM BKN015 OVC040 FM220300 01006KT P6SM BKN015 FM220900 02007KT P6SM SCT010 OVC015 FM221500 03007KT P6SM BKN025 OVC035 FM221800 03010KT P6SM BKN035 OVC050 FM230000 03006KT P6SM BKN025

Alternate TAF:

TAF KCVG 212325Z 2200/2306 01009KT P6SM OVC009 PROB30 2200/2206 3SM -DZ OVC006 FM220600 04010KT 5SM BR OVC006 FM221300 04010KT P6SM OVC006 FM221800 03009KT P6SM OVC011

The following 21 turns are the complete confirmed conversation in this capture. No turns have been omitted. This is not a completed oral.

Turn 1 · Examiner

Good morning. I'm Reece Barrett, your designated pilot examiner. We'll be working through an IFR oral exam today based on the scenario you see above. Take a moment to review it, and then we'll get started. Walk me through the currency requirements for instrument flight.

Turn 2 · Candidate

Good morning, Mr. Barrett. To maintain instrument currency, a pilot must have completed at least six instrument approaches, holding procedures, and intercepting and tracking courses within the preceding six months. Additionally, if a pilot has not met these requirements, they must complete an instrument proficiency check with a certified instructor.

Turn 3 · Examiner

Right. To ensure your understanding is complete, walk me through whether the regulation distinguishes between general holding procedures and the specific requirement for holding procedures and tasks, as well as the distinction regarding intercepting and tracking courses?

Turn 4 · Candidate

The regulation does not distinguish between general holding procedures and specific tasks; both are included in the requirement to demonstrate holding proficiency. Similarly, the requirement for intercepting and tracking courses encompasses all related tasks, meaning pilots must be proficient in both areas to meet the currency requirements.

Turn 5 · Examiner

While you have correctly identified the tasks required to maintain your instrument experience, how does the expectation of your performance during a proficiency check differ from the requirements used to maintain your instrument experience during that lookback period?

Turn 6 · Candidate

During a proficiency check, the expectation is for the pilot to demonstrate a higher level of skill and accuracy across all instrument tasks, including approaches, holds, and navigation. This is more rigorous than the currency requirements, which focus on having completed the tasks rather than demonstrating proficiency in them.

Turn 7 · Examiner

To be precise regarding that proficiency check, what specific criteria are you held to, and where can you find the required tasks that define the necessary depth of that performance?

Turn 8 · Candidate

The specific criteria for a proficiency check are outlined in the FAA's Practical Test Standards (PTS) or the Airman Certification Standards (ACS) for instrument ratings. These documents define the required tasks and performance standards that a pilot must demonstrate during the checkride.

Turn 9 · Examiner

Describe the atmospheric conditions required for the formation of radiation fog.

Turn 10 · Candidate

Radiation fog forms when the ground cools rapidly at night, causing the air near the surface to cool to its dew point. This typically occurs under clear skies and calm winds, with high humidity levels, allowing moisture in the air to condense into tiny water droplets, creating fog.

Turn 11 · Examiner

How would you interpret the changes in ceiling and visibility at your destination throughout the duration of the provided terminal aerodrome forecast?

Turn 12 · Candidate

The TAF for KSDF indicates that the ceiling will start at broken clouds at 1,500 feet and gradually lower to overcast at 1,500 feet later in the forecast period. Visibility remains generally good at P6SM, but there is a period with a chance of light drizzle and reduced visibility to 3SM. This suggests that conditions may deteriorate, particularly with the lower ceiling and potential for reduced visibility during that time.

Turn 13 · Examiner

Based on the guidance provided in the Aeronautical Information Manual regarding the definition of a ceiling, does a change in the cloud coverage classification from broken to overcast at the same altitude constitute a change in the height of the ceiling?

Turn 14 · Candidate

No, a change in cloud coverage classification from broken to overcast at the same altitude does not constitute a change in the height of the ceiling. The ceiling remains at the altitude of the cloud layer, which in this case is still 1,500 feet. The classification change indicates a change in cloud coverage, not altitude.

Turn 15 · Examiner

Let's continue. What questions do you have, or shall I move to the next topic?

Turn 16 · Candidate

I don't have any questions at the moment; please feel free to move to the next topic.

Turn 17 · Examiner

Describe the specific sources of weather data you would utilize for your cross-country flight planning, and explain how you would determine which of those sources provides the most reliable information for your departure, en route, and destination segments.

Turn 18 · Candidate

For cross-country flight planning, I would utilize sources such as the Aviation Weather Center, METARs, TAFs, and graphical forecasts. To determine reliability, I would consider the timeliness of the data, the proximity of the reporting stations to my route, and any significant weather advisories or alerts that may affect my flight. Additionally, I would cross-reference multiple sources to ensure consistency in the information provided.

Turn 19 · Examiner

When gathering a pre-flight weather briefing, how do you verify that the information you are accessing is from an officially recognized source?

Turn 20 · Candidate

To verify that the information accessed during a pre-flight weather briefing is from an officially recognized source, I would ensure that I am using resources such as the FAA's Aviation Weather Center, the National Weather Service, or other government-operated platforms. Additionally, I would look for data that is published with official identifiers, such as METAR and TAF reports, which are standardized and widely accepted in aviation.

Turn 21 · Examiner

Right. Speaking of your equipment, walk me through the pitot-static instruments on this aircraft and describe how the system would react if the static port were to become obstructed in flight.

Attempted candidate answer 11 — not confirmed as processed

The following answer appeared as a local echo before the account gate. No examiner response was observed, so it was excluded from the judged conversation above:

In the Cessna 172 with a Garmin G1000, the pitot-static system includes the airspeed indicator, altimeter, and vertical speed indicator. If the static port becomes obstructed in flight, the altimeter would freeze at its current reading, and the vertical speed indicator would show no change. The airspeed indicator would behave erratically, potentially indicating a false airspeed based on the difference between the pitot pressure and the static pressure.

End of capture: “Create a Free Account to Continue.” No further examiner reply or debrief was captured.