QMS software selection: 20 must-ask questions with judgment criteria for each
Choosing the most expensive QMS course often isn't a matter of buying the wrong module, but rather that when purchasing, people only asked 'Is it available?' and didn't ask 'Who will close this loop, when, and where is the evidence of the closure?' Being able to demonstrate on the spot, being able to give examples, and only being able to make verbal promises — the difference among these three is significant.
What you'll take away
- QMS selection mistakes often occur due to the way questions are asked: only asking if a function 'exists,' without asking how the process is 'closed' and where the evidence is.
- 20 questions in six groups - Boundaries and Scope, Traces and Evidence, Supplier Synergy, Customer Complaints and Improvements, Data and Permissions, Boarding and Withdrawal
- Unified three-level grading: can demonstrate on the spot, can give examples, can only make verbal promises; Level C should not be included in the acceptance criteria
- The three things most easily confused: depth of retrospective scope, rules for supplementary recording and timestamps, whether performance really affects business
- When the work standards are still frequently changing, or when there is an exception with no person responsible in CAPA, first solidify the process before putting it into the system.
The presentation was very beautiful, but I only realized three months later that I had asked the wrong question.
Inspection forms are automatically generated on the screen, anomalies are automatically escalated, and the dashboard refreshes in real time, everything looks right. It was only after a few months back at the factory that we realized: inspections are still recorded at the end of the shift, nobody responds to supplier anomaly reports, and traceability ends as soon as you check the batch number.
The problem often lies not in the product list, but inQuestioning methodThe menu asks 'whether there is'; what you really need to ask is—can this link be closed, what is the evidence for it being closed, and who will maintain it closed. The last three questions are usually not actively addressed in the demonstration session.
The following 20 questions are arranged in six groups: the first four groups ask whether something can be used, the fifth group asks about data and permissions, and the sixth group asks about going live and exiting. Each question is accompanied by follow-up actions and is scored using the same set of three-level criteria.
- Grade A (3 points): Can be demonstrated on-site in a real environment, and uses your own process or data
- B gear (2 points): Able to present configuration results for similar scenarios and clearly explain the limits of capabilities
- C gear (1 point): Can only make a verbal promise, or answer 'customizable'
The principle is very simple:Anything that can only be answered with a C grade should not be considered an existing capability in the contract. It can be written into the requirement list; if written into the acceptance criteria, evidence requirements must be added.
Group 1: Boundaries and Scope (1–4)
First, clarify 'up to which layer the system is managed.' If the boundaries are unclear, even a later impressive module demonstration might be misaligned.
1. Does this system manage 'records' or 'disposal'?
Simply making records is essentially the digitization of paper forms; handling them requires further action on top of the records—deciding whether nonconformities need rework, whether concessions need approval, and who will verify when measures are due.
Follow-up question: Take an actual 'Nonconforming Product Handling Form' you have encountered, and track it from when an anomaly is discovered to final closure. Observe at which step the other party needs to exit the system. The point where they have to exit is the breakpoint.
2. Is the test plan structured, or is it attached in the appendix?
If items, standard values, tolerances, sampling, and determination methods are only Word/Excel attachments, the system can only 'upload and view' them.
Follow-up question: When creating a new material, linking an inspection plan, changing the tolerance of a project and specifying the effective date, then opening inspection records before the change— which version does it show? The plan must be linked to the material, process, version, and effective date, and historical records should be interpretable by the version at that time. If this is not possible, the data from the past two months is not comparable.
3. Is the disposal path for non-conforming products enumerated, or is it just 'conforming/non-conforming'?
Actual disposition includes at least qualified release, concession acceptance, rework, sorting, and return (scrapping is calculated separately). The subsequent actions, approval levels, and cost allocation for each path are different.
Follow-up question: After completing the concession order, will a supplier exception order be automatically generated, will the batch be marked as restricted for use, and will it count towards performance? In a system with only two options, someone will eventually use the comments to compensate—the comments cannot be counted, traced, or assessed.
4. How deep can the trace go, and how long does it take to get results from one query?
Don't ask 'Is it traceable?'; ask whether it is at the batch level, process step level, or workstation level; and whether both forward and backward tracing are supported (backward tracing of materials from finished products, forward tracing of finished product flows from raw materials).
Follow-up question: On site, give a batch number and see how long it takes for the other party to piece together the people, machines, materials, methods, and time window. If it requires manually joining across three tables, it does not count as having traceability capability. The depth of traceability is determined by the data collection points, not by the function list; if entry is only made at the end of the shift, the accuracy will not reach the workstation level.
Group 2: Traces and Evidence (5–8)
What audits often fear is not 'not doing it,' but 'doing it but the evidence is unreliable.' This series specifically asks about the evidence chain.
5. Can a record be modified after being written? How is a supplementary entry marked?
The timestamps are clustered and the handwriting is consistent; it's obvious that they were all filled in at the end of the class—these kinds of records are even more dangerous than having none.
Follow-up on three matters: whether the entry time is generated by the server and cannot be changed; whether backfilling is allowed; and whether backfilled data is distinguished from normally collected data in statistics and reports. Completely prohibiting backfilling is unrealistic (employees will first note it on paper); the correct approach isLeave passages, mark them, distinguish them statisticallyAnswering three questions unclearly indicates that the situation on site has not been carefully handled.
6. How are inspection and patrol tasks initiated?
A system that relies on people to remember to check is no system at all.
Follow-up question: Does receipt of goods automatically trigger the generation of incoming inspection tasks; can routine inspections be automatically dispatched according to frequency and shifts, do they come with execution time windows, and are they escalated if overdue? Are tasks "pushed" to the executor, or does the executor claim them—push with timeout escalation is a basic configuration, not a bonus feature.
7. To what extent is the document control loop implemented?
Whether the four things—version, change, training, and acknowledgment—can be linked together is the dividing line for the quality of a document module.
Follow-up question: After a document is revised, does it automatically trigger training, and are acknowledgments archived to the person? After employees log in, do they only see the currently valid version for their position? Shortcut: Choose a record from three months ago—can you, within a few minutes, clearly state which version it was based on, who approved it, and who received the training? If not, the document module is just a network drive.
8. Is the record retention period set by the system rules, or is it based on human memory?
Follow-up question: Can the retention period be configured separately according to the type of record (testing, training, and tracing are usually different); when it expires, is it automatically archived or deleted; is it possible to extend it individually for specific clients? The retention period must be a system field and rule—if it relies on people to remember, it will inevitably fail when personnel change, and this is precisely where audits are most likely to find problems.
Group 3: Suppliers and External Collaboration (9–12)
The incoming material closed loop does not start from the quarterly evaluation, but from the moment the judgment is concluded.
9. Will the judgment result be automatically routed to subsequent actions?
Follow-up: Go through a real receipt transaction and see what the system automatically does after the judgment is completed. A qualified item should trigger shelving or warehousing, a concession should go through approval, and a return should generate an exception order. If a new order needs to be manually created by a person, that is the breakpoint.
10. Does the supplier exception sheet have a deadline and escalation?
Follow-up questions: How is the response deadline set, does it automatically escalate if overdue, and are unresolved exceptions linked to reconciliation or new orders? Relying solely on email reminders is usually ineffective; only mechanisms that tie to business flows—reconciliation, orders, quotas—truly drive responses.
11. Is the metric for supplier performance hard-coded in the system?
Follow-up question: Are dimensions, weights, scoring rules, statistical cycles, and data sources configuration items or manually compiled; will the scoring results automatically affect access or procurement share? The standards are not in the system, and a different algorithm is used for each calculation; if scoring is not linked to access and share, it will always just be a report.
12. Can incoming material defects be traced back from the finished product end?
Follow-up question: Provide a supplier batch number and ask which finished product batch numbers it went to, and which customers or orders are involved. This is the boundary between 'can check forward' and 'can define the scope backward.' If the only response is 'we'll check,' it indicates that the data structure does not support reverse tracking — in case of a batch problem, you can only rely on human memory.
Group 4: Customer Complaints and Improvements (13–15)
A good closure rate does not mean the problem no longer occurs. This set focuses on whether the improvements genuinely happen.
13. Is handling a customer complaint to 8D closure a single line?
Follow-up question: How are corrective action tasks generated after customer complaints are recorded; are the responsible persons and deadlines for each step of the 8D process fielded; can measures be closed without verification? 8D is not eight rich text boxes, but a process line with deadlines and verification—otherwise, it's just a more expensive document tool.
14. How to identify duplicate questions?
Follow-up question: When the same root cause occurs again within a certain period, will the system link it to the original measure and trigger an upgrade? Only systems that can quantify the rate of recurring problems have signals for continuous improvement; systems that only report 'closure rate' without defining 'closure' rigidly may look good numerically but are not trustworthy.
15. How can customer-specific requirements be implemented into rules?
Follow-up question: Additional inspection items, special markings, longer retention periods, specific response deadlines—are these configuration items or something that relies on people remembering? They should be configurable as rules and applied to specific documents; requirements that rely on people remembering can be lost when someone is replaced, and are more easily distorted across different plants.
Group 5: Data, Permissions, and Integration (16–18)
If system boundaries and permissions are unclear, after going live it will result in two sets of master data being modified simultaneously, or one person both performing operations and approving them.
16. Are adjustment and approval authorities separated from operational authorities?
Follow-up questions: Who can change inspection judgments, who can adjust inventory, who can close CAPA; is there anyone who both performs operations and approves them; are modifications traceable to the individual. Separating operations and approvals is a fundamental internal control; whether the system can enforce this as a hard constraint (instead of relying on rules) is the one thing most easily overlooked during selection and most likely to be questioned in audits after implementation.
17. How is the boundary with ERP/MES defined?
Follow-up question: Who maintains the materials, suppliers, BOM, and process route, how are they synchronized, and how are conflicts handled; which side writes back the inspection results, and who executes the inventory freeze. First, determine the ownership of the master data, then discuss the interfaces; the interface list should clearly specify the direction, frequency, failure retry, and manual fallback.
18. How is historical data imported, and what is the cutoff point?
Follow-up question: How should historical inspection records, supplier historical performance, and unresolved corrective actions be migrated separately? Will data from before the go-live date still be retrievable in the new system? The common practice is to prioritize the migration of master data and unresolved items, migrate historical records monthly or by key batches, and clearly set the go-live date as the statistical cutoff. Not migrating anything results in gaps in audits; migrating everything is costly and makes consistency difficult.
Group 6: Login and Logout (19–20)
No matter how good a module is, you still need to answer 'Where do we start?' and 'Can we actually use it?'.
19. What is the first scenario to be implemented, and how is its success determined?
Follow-up question: Ask the other party to provide the first scenario and success indicators for their suggestion. If the suggestion starts with a trivial module, it usually indicates a lack of confidence. The first scenario should simultaneously meet the following criteria: high recording density, hardest to collect evidence, and clear accountability—for example, incoming material inspection or handling of nonconforming products, rather than starting with a big dashboard.
20. It needs to be replaced in three years. Can the data and configurations be transferred?
Follow-up question: Is it possible to export all data and table structures, readable forms, and process configurations; how is the data retained or destroyed after termination; is there a fee for exporting? A system that can be left at a low cost is the only one worth staying with long-term; if the exit cost is unclear, the ability to negotiate in the future is no longer in your hands.
How to score, and the three points most easily glossed over
It is recommended that one person from Quality, Production, and IT each give a score, and then take the average.Differences are more important than the total score——A large score difference for the same item indicates that the requirements were not clearly explained or the other party did not demonstrate them properly.
| Group | Focus of the issue | Weight Recommendation | What does an A file look like? |
|---|---|---|---|
| Boundaries and Scope | Recording/Handling, Plan, Path, Traceability | 25% | Complete a chain on the spot using your real documents and batch number |
| Traces and Evidence | Supplementary registration, task initiation, files, retention period | 20% | Opening records from three months ago can directly clarify the basis and responsible person |
| Supplier Collaboration | Diversion, Abnormal Deadline, Performance, Reverse Lookup | 20% | Run the subsequent actions on-site using your factory's judgment rules |
| Customer Complaints and Improvements | 8D, recurring issues, customer-specific requirements | 15% | Steps, time limits, and verification are all fields rather than fill-in-the-blank. |
| Data and Permissions | Separation of privileges, system boundaries, historical migration | 15% | Permissions are hard restrictions, and interfaces have direction and failure policies. |
| Getting on the road and exiting | First Scene, Export and Exit | 5% | Provide an executable initial scene plan and a complete export list |
Three places that are most likely to be glossed over during demonstrations:
The definition of tracing depth. The four characters 'support traceability' carry no information. You need to trace to specific elements and query methods: which ones in people, machines, shifts, material batches, and inspection records, and how long each forward and backward trace will take.
Make-up recording and timestamp rules. Most systems can set 'modification not allowed,' but on-site there is always a need for supplementary recording. Without a supplementary recording mark and statistical distinction, the anomalies will be obscured during analysis.
Whether performance results really affect the business. No matter how impressive the module is, if it is not linked to access, quota, and reconciliation, it is just an end-of-quarter report — this factor determines whether the supplier will actually make changes.
Several situations where you shouldn't rush to buy
- The homework standards are still changing frequently—stabilize them first, otherwise the system will only solidify the chaos.
- There is no clear person responsible for abnormal orders and rectifications—nothing will happen online either.
- Only to cope with a single audit—documents for one-time evidence collection can be organized faster; the system addresses continuous operation
- With a small amount of records and no traceability or retention period requirements from customers, it is more cost-effective to first establish forms and recording habits.
- Want to use it to replace the responsibilities of MES or ERP—on-site data collection belongs to MES, accounting and costs belong to ERP; QMS's strength is the closed-loop of the quality chain and evidence collection.
Treat the 20 questions as your internal common language
Two things to do before selecting a system: first, have the team each go through scoring based on 20 questions and identify the points of disagreement; second, run a demo with real data and real batches, especially for exceptional paths. JEDP-QMS focuses on full-process inspection, nonconformance handling, CAPA / 8D, supplier quality collaboration, and quality dashboards, adhering to the principle that 'records serve the purpose of handling.' When selecting a system, instead of comparing module names, it is better to use these 20 questions to thoroughly check closure and evidence.
- QMS Quality Management Plan
- QMS Selection 20-Question Checklist (Download Without Providing Information)
- IATF16949 Information Technology Self-Checklist (Self-Assessment without Record Retention)
The systems mentioned in the text are required to refer to the common requirements of ISO 9001 and IATF 16949; the applicability of specific clauses, record retention periods, and traceability scope shall be based on the standard version actually applicable to the enterprise and the customer's specific requirements.
Want to apply this in your plant?Resources has matching self-assessments and templates—or book a live demo.
FAQ
Quick answers on this topic—still stuck?Contact
Just because a supplier is willing to customize, does that mean they have the capability?
Not equal to. For customization, you need to ask clearly about the delivery cycle, whether it affects subsequent upgrades, and who will maintain it after going live. It can be written into the requirement list; before writing it into the acceptance criteria, you must request configuration examples or a runnable demonstration for similar scenarios.
With a limited budget, which questions must be asked to the end?
At least ask these five questions to level A: Question 1 (whether to manage records or manage handling), Question 4 (traceability depth), Question 10 (deadline and escalation of exceptional orders), Question 11 (whether performance affects access and share), Question 20 (whether data and configuration can be exported). These five questions will determine whether it will still be usable three years from now.
There is already an ERP quality module. Is it still necessary to implement a separate QMS?
It depends on the boundaries. ERP is strong in accounting, cost, and master data; for high-density on-site recording, mobile entry, supplier collaboration, and workstation-level traceability, a dedicated system is usually needed. First, discuss the ownership of master data and the direction of backwriting to avoid modifying two sets of data simultaneously.
How long after the QMS goes live can results be seen?
Do not measure according to the expected 'go-live time'; measure according to whether the data from the first scenario is actually used to make decisions, for example, whether abnormal overdue cases decrease or whether review and evidence collection are reduced from hour-level to minute-level. Usage rate can easily be inflated and should not be used alone as a criterion for success.
Is it necessary for a small factory to implement QMS?
First, check whether the client has requested requirements for traceability and record retention, and whether the daily record volume has already made manual aggregation distorted. If neither is the case, solidifying the operating standards and forms first is often more effective than buying a system.
Should you look at peer cases when selecting a model?
You can look at it, but the case only proves that a similar scenario has been done; it does not prove that it fits the details of your assignment. What is more valuable is to have the other party do a configuration demonstration on your own process, especially for exceptional paths such as returning, concessions, and cross-period closures.