Outsourcing lease abstracting does not remove the need to review the work. It changes the review question from "did we do this correctly" to "how much of the vendor's output meets the standard, and what happens when it does not." Without a sampling plan, the answer is usually a few spot checks and a general impression.
This guide describes a vendor abstract QA sampling plan for land teams that buy abstracting or lease data services. It is an administrative control. It does not interpret a lease, decide ownership, or replace review by a qualified land professional.
Vendor Abstract QA Sampling Plan: define acceptance first
Write the acceptance standard before looking at the vendor's output. The standard should state what fields are required, what counts as a material error, what counts as a minor formatting issue, and what evidence the abstract must cite. If the standard is vague, every reviewer will apply a different bar.
Use the client's abstract template and field dictionary. Define the unit of review: one lease, one tract, one document, or one abstract record. A plan that mixes units cannot produce a comparable rate. Decide whether a lease with multiple tracts is one unit or several, and keep that definition stable.
Define the defect classes in advance. Typical classes are missing document, wrong document, missing field, incorrect value, unsupported value, wrong party, wrong date, wrong acreage, wrong legal description, and formatting or indexing error. Add a severity level so the team can distinguish a material defect from a cosmetic one.
Build a reproducible sample
The sample should be selected from a defined population and documented so another reviewer can reproduce it. The population may be all abstracts delivered in a month, all abstracts for a project, or all abstracts from a vendor. Record the date range, the delivery batch, the number of records, and any exclusions.
A simple random or systematic selection is easier to defend than choosing files that look interesting. If the team needs to over-sample a risk category, such as leases with complex depth clauses, record the oversampling and report the results separately. Do not present a risk-based sample as a representative estimate of the whole population.
The sample size should reflect the decision. A small pilot may need a larger proportion to find obvious problems. A steady-state review may use a smaller monthly sample plus a targeted review of high-risk documents. Do not invent a statistical confidence claim unless the sampling design supports it.
Write the codebook
The codebook is the reviewer's instruction manual. It should define each field, the allowed values, the source hierarchy, and the defect codes. Include examples of acceptable and unacceptable entries. If two sources conflict, state which one controls and what the reviewer should record.
For each abstract in the sample, record the abstract identifier, lease identifier, reviewer, review date, fields checked, defects found, severity, and disposition. Keep the vendor's original submission and the reviewer's marked copy. If the reviewer corrects the abstract, record the correction as a separate event.
Calibrate reviewers before the first production sample. Have two reviewers score the same small set independently, compare results, and resolve disagreements. Preserve the initial ratings so the team can see where the codebook is ambiguous. A disagreement is useful evidence; hiding it by averaging the scores is not.
Review, score, and route rework
Score each unit against the acceptance standard. A unit can pass, pass with minor defects, fail with rework required, or fail with escalation. The disposition should name the next action and the owner. Do not let a failed unit sit in a "reviewed" status without a rework request.
Send rework requests with the specific defect, the source evidence, and the expected correction. Ask the vendor to return the corrected abstract with a change note. Review the rework against the same codebook. If the correction introduces a new defect, record it rather than closing the original issue.
Track rework turnaround and the rework pass rate. A vendor that fixes defects quickly but repeatedly introduces new ones is not improving. A vendor that returns a corrected file with no change note is making the next review harder.
Separate QA from acceptance
The QA reviewer can identify defects and request corrections. The client's authorized reviewer decides whether the abstract is acceptable for the intended use, whether a material error affects a title or payment decision, and whether the vendor's performance triggers a contract remedy.
Do not let a sampling score become a substitute for professional judgment. A 98 percent pass rate does not mean the two failures are unimportant. Review the failures by severity and by field. A single wrong legal description can matter more than a dozen formatting issues.
Measure the right things
Track the number of units sampled, the defect rate by class and severity, the rework rate, the rework pass rate, median rework turnaround, and the percentage of units with source citations. Report the denominator and the sampling method with the result.
Compare vendors only after confirming that the population, unit, codebook, and reviewer calibration are similar. A lower defect rate may reflect an easier document mix rather than better work. If the mix changes, report the change.
A practical launch sequence
Start with one vendor and one delivery batch. Define the unit and codebook, then review a pilot sample of 20 to 30 units. Use the results to clarify the standard, not to rank the vendor. Expand to a monthly sample once the reviewers agree on the classifications.
Publish the plan, the codebook, and the rework template. Train the reviewers and the vendor contact. Hold a monthly calibration session. Review the defect mix quarterly and adjust the sample toward the highest-risk fields.
For teams that need help maintaining the review log and rework queue, a defined lease data entry service can support the indexing and follow-up. The client keeps acceptance authority. The land file closing certificate checklist helps ensure the QA evidence survives project closeout. The GAO data reliability guidance offers general context for documenting data quality checks, not a lease-specific standard.
Frequently asked questions
How large should the sample be?
It depends on the decision and the risk. Start with a pilot large enough to expose common defects, then set a monthly sample that the team can review consistently. Document the method.
Should the vendor see the defect codes?
Yes, when the codes are part of the acceptance standard. The vendor should know what will be reviewed and how defects are classified.
Can a passing sample close the project?
Only when the sample, the open exceptions, and the acceptance decision are documented. A sample is evidence, not a blanket release.
Ready to Scale Your Landman Business?
Discuss a defined remote support scope for title research, lease administration, GIS coordination, or document work.
