What “done” means for a digital project: an executable acceptance checklist
"'Hasn't all this been done?' and 'Can this even be called finished?' — neither side lied, it's just that 'finished' was never defined. The value of the acceptance checklist lies not in its strictness, but in its existence on the day work begins.
What you'll take away
- Acceptance disputes stem from the lack of completion definitions, not merely from poor execution.
- Four levels of standards — functional availability, data accuracy, user usability, and metric improvement — each level has different ways of providing evidence
- Only testing functionality without testing data is the biggest loophole and also the hardest layer to rework.
- The demo environment is not equal to being available online; scenarios must include real data and real users.
- The observation indicators for 30/60/90 days after going live must be agreed upon in advance, otherwise 'ending the project as soon as it goes live' will leave a system that no one uses.
"Haven't we done all this already?" — the most common line in the final season
At the project example meeting, the implementers opened the feature list and checked off each item: 'The process is complete, the reports are out, and the training has been signed off.' The business side pulled out the real documents: 'This refund can't go through; this number doesn't match the old records; the workshop is still using Excel.' Neither side was lying. They were just using different definitions of 'finished'—a term that was never written as a verifiable sentence in the project initiation meeting.
The value of an acceptance checklist lies not in how long the clauses are, but inIt already existed on the day construction started. Without it, all that remains at the end are positions: one side talks about the scope of delivery, the other side talks about business availability, and the negotiating position will tilt sharply after going live.
Four-Level Standard: From 'Can Click' to 'Worth Keeping'
Many people think that acceptance is just 'checking off the feature list.' A more reliable understanding is: moving from 'can be clicked' to 'worth continuing to use.' The evidence required for the four levels of standards is completely different, and if they are mixed on one signature page, there will definitely be disputes in the end.
| Hierarchy | Test what | How to provide evidence | But so what? |
|---|---|---|---|
| 1. Function available | The internal process for the requirement can go through | Go through the scenario checklist item by item, including abnormal branches | Stuck halfway after going online |
| 2. Data is correct | Calculation, circulation, and report results are reliable | Real data compared to the old method | User Retention Excel |
| 3. Users will use | Key positions can independently complete daily tasks | Usage rate, proportion of systematized documents, practical assessment | The system is running, but the personnel are not moving. |
| 4. Indicator Improvement | The agreed business targets have improved | Baseline Before Renovation vs After Launch | There are costs but no benefits |
The second layer is the largest vulnerability. Function acceptance is easy—you just click once to see if it works. Verifying data accuracy requires cross-checking, involves a lot of work, and is often skipped; yet it's precisely the hardest to redo: half a year after going live, if you find the criteria are wrong, historical data has to be recalculated.
The third layer is the easiest to be overlooked. The technology is launched, but if no one in the business uses it, it's as if it hasn't been launched. Attending training does not mean being able to use it—having key personnel independently complete a daily document in front of you is more convincing than a hundred sign-in sheets.
The fourth layer depends on the first three layers, and there must be a pre-transformation baseline. Without a baseline, this layer cannot be accepted — so baseline collection needs to be arranged at the start of the project, not looked for after going live. The baseline metric must also be fixed: for the same metric, the data collection method used before the transformation must be comparable after going live, otherwise 'improvement' will just become a matter of differing interpretations.
How should acceptance testing scenarios be designed to be tough enough?
Don't just test the normal paths. Three categories are recommended:
- Normal process: standard business goes from start to finish
- Exceptional branches: Return, Partial Completion, Cancellation, Overdue, Concurrency
- Boundary cases: quantity is zero or extremely large, across months or years, batch, permission boundaries
Sampling should select genuine documents, andAt least include one historical case that has had problems——These kinds of forms often reveal the most loopholes.
| Acceptance action | Pass the standard | Who signed |
|---|---|---|
| Scene walkthrough | The list can go through even with abnormal branches | Business Manager |
| Data comparison | Sampling is consistent with the old method or differences can be explained | Business Finance / Planning |
| On-the-job practice | Key positions have no assistance in completing daily documents | Post representative |
| Indicator Observation | 30/60/90 reaches the agreed threshold | Project Steering Committee |
A self-test: Is there an item in the scenario list for a business that 'even you yourselves know you messed up before'? If not, the list isn’t ruthless enough.
Data comparison should not use simple random sampling; weighting by risk is more effective: about 40% of normal closed-loop cases, roughly 30% of historical cases that had errors or disputes, around 20% of cross-month / partially completed / returned cases, and about 10% related to authority boundaries. Each record should include the old result, the new result, the reason for the difference, whether it is acceptable, and the responsible party for the fix—without this comparison table, the second layer is essentially not done.
The manufacturing industry also needs to particularly watch three types of numbers: quantity and unit conversion, amount and tax rate, and inventory / work-in-progress balance after status changes. If these three types are wrong, the business will immediately revert to Excel.
Six Practices That Undermine Acceptance
| Approach | Performance | Consequence |
|---|---|---|
| Demonstration Scene Acceptance | Go through the prepared data | The real data blew up as soon as it went live |
| Only test functionality, not data | Check the features, don't look at the numbers | Introducing production with definition errors |
| Project ends upon launch | Sign off on acceptance on the day of launch, no observation period | No one is using it, and problems are piling up. |
| Unclear requirements boundary | Mixing defects and new requirements | Endless wrangling over the final payment |
| Training check-in will be useful | Only look at attendance, not practical operation | The third layer is essentially ornamental |
| Post-baseline supplementation | Look for pre-transformation data after going live | The fourth layer can never be verified |
The biggest cost among them is 'completion upon launch': skipping the third and fourth stages, leaving all the risks after the launch, at which point the bargaining positions of both parties are completely different.
Acceptance in stages is safer than piling everything up and signing at the end.
| Stage | Acceptance focus | Payment-linked |
|---|---|---|
| Core scenario launched | First floor Second floor | First installment settled |
| Stable use in key positions | Third layer | Mid-term payment |
| End of the indicator observation period | Fourth layer | Final payment |
Disputes are spread throughout the process, rather than accumulating at the end; each batch also provides experience for the next batch.
It is recommended to set an observation window after going live:
| Point in time | What are you looking at? | Want to confirm what |
|---|---|---|
| About 30 days | Key position utilization rate, proportion of document systematization | Has it really been used? |
| About 60 days | Data spot check pass rate, anomaly handling timeliness | Is the data reliable? |
| About 90 days | Agreed business indicators, residuals in manual ledgers | Whether actual value is generated |
Manual ledger remnants are very practical: Even months after going online, some people still privately maintain the same batch of data in Excel, indicating that the system hasn't solved the real problem and it's necessary to go back and find the reason.
The rhythm that can be included in the contract appendix: freeze the four-level standards, scenario list, and baseline collection plan before the start of construction; complete the first and second levels and register unresolved defects in the week of going live; afterwards, track the third and fourth levels according to the table, with defects and changes on separate tracks.
First clearly write down three things in the contract
- Scope of Warranty and Response Standards: What counts as a defect, response, and repair time limit
- The boundary between defects and changes: What free repair, what change in procedure
- Acceptance Criteria and Payment Milestones: How the four levels correspond to payment, and how long is the observation period
The more specific the writing, the less trouble later. The vaguer the acceptance criteria, the more it ultimately relies on personal negotiations—which usually turn out to be disadvantageous for Party A.
Executable boundary example:
| Situation | Inclination | Handle |
|---|---|---|
| The requirements have been specified, but they are not implemented or implemented incorrectly | Defect | Free repairs within the warranty period |
| The document was not written, but it is industry standard, mandatory, and affects the launch. | Negotiation Defects / Minor Changes | Pre-agreed 'go-live blockage items' list |
| New ideas, new fields, new reports after going live | Change | Assessment and Quotation |
| The historical standards after migration are inconsistent with the new standards | According to the contract | The scope of migration and cleaning responsibilities must be clearly written before work begins |
Treat this table as an attachment; it's much more effective than arguing afterward about 'whether it counts as completed'.
When things can be simplified—and what must never be skipped
Extremely small scope, internally built, compliance mandatory launch, clearly allowing pilot failures — the form can be simplified.Data verification sampling is not recommended to be omitted, in any case. If the pilot is allowed to fail, it is also necessary to include in the appendix 'whether to promote after failure and who bears the cost of failure' to prevent the pilot from becoming a long-term parallel with no conclusion.
It is recommended to write the minimum release threshold into the release memo: core scenario exception branches are passed, blocking-level defects are cleared; data comparison sampling reaches the agreed threshold, with responsible persons and repair dates for items that did not pass; key positions have completed hands-on assessment (not just training attendance); rollback/emergency plans are clear — who decides to deactivate, and which side’s data is taken as standard during the dual-track period; observation indicators and data collection responsible persons are specified. Releasing without data comparison and hands-on assessment usually backfires by the end of the first month with 'system inaccurate / no one uses it.'
Acceptance is not a ceremony; it turns 'completion' from a verbal claim into a state that can be evidenced — the supporting materials are more important than the signature page. Signatures can be completed in a day, but checklists, practical records, and observation period reports are what can actually be used if disputes arise later.
"'Completed' is written on the start date, so you don’t have to rely on your voice to finish the job."
When compared to the existing checklist, two things are enough: whether there is a data comparison test, and whether there is a post-launch observation period. If still in the project initiation stage, just include the four-tier standards in the contract appendix—it’s an order of magnitude cheaper than adding the checklist at the end of the project. The implementation paths for the platform and the product are different, but the acceptance logic is similar: for JEDP projects, it is also recommended to include scenario walkthroughs, data comparisons, and observation periods in the appendix, rather than just signing off all at once on the launch day.
- Overview of JEDP Platform Capabilities
- Information Technology Selection Scoring Sheet (Self-Test Without Providing Contact Information)
- 6 Weeks vs 6 Months: Comparison Roadmap Infographic
- How to calculate the ROI of digital investment
Want to apply this in your plant?Resources has matching self-assessments and templates—or book a live demo.
FAQ
Quick answers on this topic—still stuck?Contact
Does the acceptance have to be completed in one go?
It is recommended to divide the process into stages. Confirm in batches according to modules or scenarios, with payment milestones linked accordingly. One-time acceptance will pile up disputes until the end, leaving neither party with room to maneuver.
How to verify data accuracy?
Compare with real data: the same transaction is processed using both the old method and the new system, and the results are compared. The sampling should cover normal and abnormal cases, especially returns, partial completions, and cross-period transactions.
Does user usage count as an acceptance criterion?
It should be quantifiable. Look at the usage rate of key positions, the proportion of systematic documents, and the remnants of manual ledgers. If several months after going live there are still people using Excel to record the same batch of data, this level has not passed.
Who is responsible if problems are found after going live?
During the contract phase, agree on the scope of warranty and response standards, and distinguish between defect repairs and new requirements. If the boundaries are unclear, disputes are most likely to arise later.
What should be done if the fourth-level indicator cannot be improved?
First distinguish whether it is a system problem, a process problem, or an unreasonable baseline. It is recommended to set the fourth level as an observation period goal, linked to part of the final payment, rather than completely tied to 'whether it can go live'.