Source Confidence Scoring for Tract Data Reconciliation

A validation study for source-confidence labels that keeps provenance dimensions separate and reserves tract determinations for qualified reviewers. This is an original administrative research protocol, not legal advice and not a substitute for title, regulatory, survey, engineering, tax, or other professional judgment. It is designed to produce auditable descriptive evidence without allowing a score or dashboard to decide a substantive land matter. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Source Confidence Scoring for Tract Data Reconciliation: Research question and decision boundary

The primary question is: Can reviewers apply a multi-dimensional source rubric consistently, and does the rubric help prioritize tract-data conflicts for specialist review? The analysis may identify missing controls, inconsistent definitions, or records for review. It must not determine ownership, interpret an instrument, declare compliance, select a legally controlling source, or evaluate an individual worker. Those decisions remain with named qualified roles. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Observation unit and population

The observation unit is one source-value pair for an identifier, location label, acreage field, or relationship. The target population is conflicting source-value pairs detected in a frozen reconciliation batch. A protocol owner should freeze the extraction window, follow-up cutoff, and eligibility rules before examining outcomes. Include test, duplicate, cancelled, and migrated records only under explicit rules, and publish their counts even when they are excluded from the primary result. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Sampling and stratification

Create a reproducible frame from a read-only export. Deduplicate only after defining whether the unit is a request, field, exception, package, or event. Stratify by field type, source class, age, jurisdiction, and whether the conflict involves a transformed value. Use a random sample within each planned stratum or analyze the full population when volume permits. Oversampling uncommon failure categories is useful for control testing, but weighted population estimates must reverse that oversampling. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Data dictionary

Define every field with allowed values, source system, timestamp semantics, null meaning, and transformation rule. Distinguish unavailable, not applicable, not collected, unreadable, and extraction failure. Preserve raw text and source identifiers in restricted storage, then use study IDs in the analytic table. A second analyst should reproduce a sample from source before calculations begin. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Primary measures

Pre-specified measures are inter-reviewer agreement, missing-provenance rate, specialist-overturn rate, time to escalation, unresolved-conflict age. Report counts and denominators for every rate. For elapsed time, show median, interquartile range, and selected aging bands; averages alone can be dominated by a few old records. For records still open at the cutoff, report age separately rather than silently treating them as complete or dropping them. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Multi-dimensional confidence rubric

Keep provenance, directness, version clarity, observation recency, and reviewer acceptance as separate ordinal dimensions with anchored examples. Provenance asks whether a source can be retrieved; directness asks how closely it records the field; version clarity identifies the edition; recency records observation timing; acceptance shows specialist review. None declares legal priority. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Give two reviewers a balanced conflict set in randomized order without showing the production selection. Measure agreement per dimension and inspect confusion matrices. If reviewers confuse official publication with direct evidence, revise the codebook and repeat calibration. Test usefulness through prioritization: missing-provenance conflicts may wait longer or require more requests, but no score should auto-populate a tract value or choose a “winning” source. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Classification reliability

When a person assigns a reason, source class, completeness label, or outcome, use a written codebook and a shared calibration sample. Two reviewers should independently classify a planned subset. Report raw agreement and a chance-adjusted statistic when category prevalence makes it meaningful. Resolve disagreements for the final dataset, but retain the original ratings for the reliability result. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Analysis plan

Start with a flow table: frame count, excluded records by reason, analytic count, open count, and missingness by key field. Describe each stratum before comparing it. Present unadjusted results first, then a limited sensitivity analysis that holds the measurement window and definitions constant. Do not run a large collection of subgroup tests and select only favorable differences. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Bias and alternative explanations

A score can create false precision, and reviewers may mistake official origin for current applicability or legal priority. Other concerns include inconsistent historical definitions, selective documentation, policy changes during the window, vendor-system migrations, and correlated records from the same project. The report should describe these mechanisms directly and avoid causal verbs such as “caused,” “prevented,” or “improved” unless the design truly supports them. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Missing and conflicting data

Publish missingness for each primary variable and compare it across planned strata. Never replace missing with zero or “complete” by default. Preserve conflicting values in separate columns with their sources and dates. If a reviewer resolves a conflict, store the decision, authority, and evidence as a new field rather than rewriting the research extract. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Sensitivity checks

Repeat the main summaries under reasonable alternatives: excluding an outage period, changing one aging threshold, separating migrated records, or limiting analysis to records with complete provenance. Compare complete-case results with a result that explicitly retains an unknown category. A finding that reverses under a modest definition change should be labeled unstable and used only to plan further review. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Reproducibility and information protection

Retain the approved query, export timestamp, checksum, data dictionary, transformation log, analysis script, output tables, and reviewer notes. Restrict direct identifiers and confidential documents to personnel with a defined need. Published results should use aggregation or suppression rules suitable for the dataset. Reproduction means another authorized analyst can obtain the same table from the same frozen inputs. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Worked analytic scenario

Score provenance, recency, directness, and version clarity separately; never collapse them into an automatic “winning source.” The scenario should be rehearsed with synthetic or de-identified data before production use. Any result that flags a record creates a review priority, not a substantive determination. The project owner should document who may receive the list and what follow-up is permitted. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Interpretation standard

A useful conclusion states the observed population, period, definitions, numerator, denominator, uncertainty, and limitations. It separates source facts from analyst interpretation and operational recommendations. Small strata should be described cautiously, and rankings should be avoided when measurement conditions differ. The next review date should be set before publication so provisional findings are not treated as permanent facts. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

Evidence base

The protocol draws on public guidance for records management, data integrity, privacy, and reproducible public data. These sources define general control principles; they do not establish how a particular lease, tract, filing, or owner record must be handled. Each source was checked on October 12, 2026. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.

1. OMB Circular A-130
2. Federal Data Strategy
3. BLM Land Records
4. Bureau of Indian Affairs Land and Title Records
5. EIA Oil and Petroleum Products
6. USGS Data Management
7. Texas Railroad Commission Research Queries
8. Oklahoma Corporation Commission Oil and Gas
9. New Mexico Oil Conservation Division
10. NARA Records Management

Implementation checklist

Approve the question, authority boundary, population, observation unit, window, cutoff, exclusions, strata, codebook, source inventory, privacy review, quality sample, analysis plan, sensitivity checks, publication audience, and retention rule. Run the protocol on a small frozen extract, record ambiguities, revise once, and version the final plan before full extraction. Changes after results are visible must be disclosed. This source quality application is specific to Source Confidence Scoring for Tract Data Reconciliation; its definitions and evidence cannot be substituted for another study.