Better Performance Test Results? Check the Comparison Conditions First

Differences in numbers and the effects of design are not the same thing

In development meetings, the results of previous and current tests are displayed side by side.

It is explained that since this number is higher, the design has improved.

At that point, one more question can be asked.

Were there no differences in the test conditions aside from the design?

This article covers the questions to use when reviewing non-clinical bench performance test data for medical devices and provides examples for self-writing.

It does not assume that larger numbers always indicate better performance.

The meaning of the numbers changes depending on which characteristics are being measured.

This is not guidance on determining the test methods or regulatory compliance for specific items.

The focus is on examining whether the questions this comparison seeks to answer connect to the actual test records.

First, write down the questions you want to answer through comparison

Is the 'new product better?' a broad question for setting a test plan.

MedQraft recommends narrowing the purpose of the comparison in the first line of meeting materials.

For example, this can be expressed as, 'Confirm the impact of changes to the connector shape on separation load under specified conditions' by attaching the subject and measured items.

The NIST experimental design guide discusses connecting purpose setting, variable selection, and design choices.

The perspective of distinguishing the elements you want to compare from other influencing factors becomes the starting point for interpreting the results.

NIST, Choosing an experimental design

Also distinguish whether the purpose is to explore the properties of candidate designs or to confirm compliance with established requirements.

If the meeting title reads 'Validation,' but the actual stage is exploring methods, the person reading the report's conclusion may misunderstand the nature of the test.

Hypothetical Case: Design and Fixture Methods Have Changed Together

The following is a scenario for explanatory purposes, not based on actual products or customer cases.

A team changed the shape of the connector.

When trying to compare with previous results, they noted that the fixture used for the test is also different.

Comparison Item	Previous Test	Current Test
Connection Shape	Existing Design	Changed Design
Test Fixture	Fixture A	Fixture B
Measurement Item	Separation Load	Separation Load
Explanation for Results	Observed value under previous conditions	Higher observed value under current conditions

This table includes both the intended design changes and changes in the testing methods.

Even if the fixture changes, there may be no influence on the results, but this table alone does not provide grounds to conclude that there is no effect.

Conversely, there is also no basis to definitively claim that the results have changed because of the fixture.

Therefore, it is better to separate what is confirmed with the current data from additional interpretations.

The statement 'A higher value has been observed under these conditions' retains the observational content, and further reviews what additional evidence is necessary for the conclusion that 'the design change is the cause of that difference.'

Whether the necessary follow-up work is a record check, method comparison, or additional testing should be determined based on the actual situation.

Place a condition comparison table next to the results table

The table below is a review note suggested by MedQraft.

It is not a list of items from established test specifications, and only relevant and meaningful rows for that test should be used.

Review Item	What to Find in the Report	Questions to Ask if There are Differences
Test Subject	Model, specifications, design version, specimen condition	Can they be grouped together as subjects with the same conclusion?
Preparation Process	Related pre-processing and storage history	How does this relate to the purpose of this comparison?
Test Method	Method version, fixture method, major settings	Is the difference intentional or are the conditions changed together?
Measured Value	Measurement items, units, result derivation methods	Do numbers with the same name have the same meaning?
Record Connection	Identification information from the report and specimen/raw materials	Can anyone find the related results?
Conclusion	Observed facts, interpretations, unverified parts	Is the information within the explained range?

The goal is not to make every cell 'identical.'

You may deliberately change several conditions depending on the purpose of the test.

The important thing is to explain the differences and confirm whether there is a design and interpretation that takes those differences into account.

They should be careful not to interpret materials with several elements that have changed simply by the effect of one element alone.

Reference Reporting Perspectives from FDA Material

The FDA's 2019 guidance on non-clinical bench performance testing reports recommends clearly describing the test purpose, methods, samples, applicable pre-approval criteria, and results.

For tests that do not have acceptance criteria, it is also treated to explain the evaluation criteria used for interpretation.

FDA Guidance, II.A

This is a non-binding recommendation regarding the reporting of premarket submission data for medical devices in the United States.

This document does not set testing standards for all medical devices or explain domestic submission requirements directly.

Clinical trials, biological safety assessments, software validation, etc., should be distinguished from the application scope of this document.

Practically, instead of simply copying the 'pass' indication from the report, it is suggested to convey what judgment was made under what conditions.

Others should be able to read the starting point and limits of the judgment without having to perform the test again.

How to Rewrite the Conclusion Sentence

The review note from the earlier hypothetical case could be written as follows:

A higher separation load was observed under the current test conditions compared to previous results.

Since both the design and the fixture method have changed, we do not separate the assessment of the effect of design change based on the current comparison alone.

After confirming the fixed conditions and related records from both reports, define the additional review range.

This is not an example of writing sentences passively.

It delivers the confirmed facts clearly and allows the next reviewer to proceed with the yet-verified parts.

Instead of arbitrarily changing the original report, it leaves the interpretation range and basis in internal comparison notes.

If the review has progressed to secure the basis for the impact of method differences, the conclusion can be updated with that basis included.

Rather than removing previous sentences, it should be processed according to the company's record management protocol to reflect why the judgment changed.

One Page for the Next Meeting

Select two recent test reports and write down the comparison purpose, intended changes, other condition differences, current conclusions, and remaining checks on a single page.

Do not arbitrarily decide the number of specimens or acceptance criteria in this note, but return to the actual plan and applicable criteria for confirmation.

At the end of the note, write down the person responsible for subsequent judgments and the materials to be checked.

Instead of ending with 'further review needed,' being specific like 'check the fixture methods and test setting records from both reports' clarifies what happens after the meeting.

References and Writing Scope

• FDA, Recommended Content and Format of Non-Clinical Bench Performance Testing Information in Premarket Submissions.

Published on December 20, 2019.

References were made regarding reporting information and the applicability scope.

• NIST/SEMATECH, Choosing an experimental design.

References were made regarding comparison purpose, variables, and connection to experimental design.

Verification Date: September 15, 2026.

The tables and hypothetical cases, and comparison notes were created by MedQraft and do not replicate the original document's tables.

Testing and judgment criteria needed will vary depending on the items, development stages, and applicable countries.