Data for Major Projects

101: decide what the data must help you do

Start with a decision and work backward. Who needs to act, what must they understand, and when will the information still be useful? A package manager reviewing delivery readiness needs a different view from a sponsor considering a funding change. Write those needs before deciding which fields to collect or which dashboard to build.

Orion Matthews and Tonya White's reporting presentation connects the groundwork to the implementation: scope, stakeholder needs, contractor requirements, calendars and workflow come before the integrated reporting solution.

Apply that discipline to model data as well. Agree what the model must help someone do, how often it needs updating and which inputs support that use. A logistics review and a quantity takeoff need different information and checks.

Build a shared language for the project

A common code is useful only when people agree what it means. Define the work breakdown structure and write the boundaries of each element: what is included, what is excluded, and which interfaces remain. Check the definitions with the people who estimate, schedule, procure and deliver the work. Resolve competing interpretations before they become apparently comparable numbers.

• Create a data dictionary that names each field, meaning, unit, allowed values, owner and source.
• Map project, finance and reporting-period identifiers across systems. Define the full key needed to distinguish an individual record.
• State the level of detail: a project, package, activity and cost transaction are different things.
• Record currencies, time zones, reporting cutoffs and the distinction between actual, planned and forecast values.

On Green Riyadh, Jovita Stander describes shared location codes connecting the schedule, cost and map, with the green assets intended to remain mapped at handover.

Try a scope change against your own coding model. Remove or split a location, then follow the effect through the schedule, cost records and reporting. The team should be able to identify the same piece of work in each view and explain how its history is retained.

Settle the data wars before connecting systems

A disagreement over the project record is a governance problem before it is an integration problem. A shared platform does not decide who may approve a change, which revision the field should use or what the owner receives at handover. Agree those responsibilities with the organisations supplying and using the information.

Scroll sideways or use the arrow keys to see all columns.

Keep a current working view and an approved period record distinct. They can legitimately answer different questions. Do not erase a discrepancy by forcing two numbers to agree before understanding their dates, scope and approval status. Record the resolution and tell downstream users when their interpretation needs to change.

Connect systems through accountable people

Draw a simple map from source systems to the information people use. For each connection, identify who supplies the records, who checks them, how they arrive and what happens when they are late or incomplete. Review what existing tools already do well before adding another reporting layer. Leave a controlled space for analysts to explore questions that the standard reports do not answer.

Give source owners time to review recurring errors and fix how information is entered. If analysts keep repairing the same field in a reporting spreadsheet, carry that correction back to the source process and its owner.

Turn contractor reporting requirements into a working interface. Agree formats, definitions, cutoffs and correction responsibilities, then walk through an example submission together. Coordinate the contractor calendar with the internal review and publication calendar. A deliverable due date needs enough time for the people receiving it to check and use it.

Test representative records through the whole connection, including identifiers, units, revisions, permissions and failed updates. Confirm that corrections reach the right people and that a failed transfer is visible and recoverable. An available API is a starting point for this test, not evidence that the project has a working information flow.

How to create trust in the data

People trust people. When a project director relies on a number, they also rely on the people who collected it, checked it and explained what it means. Give those people a clear process and the authority to raise a problem before the information is used.

Make that accountability visible from the source record to the report. For each important dataset, name the person responsible for its content and the person accepting it for use. Give them time to investigate discrepancies with the team doing the work. Keep their review attached to a specific version and reporting date.

• Name the owner: who can explain where the information came from and resolve a challenge to it?
• Show the checks: reconcile totals, check completeness and dates, and compare a sample with the underlying field or commercial records.
• Record the review: identify the reviewer, version, cutoff date, checks completed and outstanding exceptions.
• Make acceptance explicit: state who accepted the dataset, when, and for which use. Keep provisional information visibly provisional.
• Keep the conversation open: give users a clear way to question a number, trace its source and reach the accountable person.
• Close the loop: explain corrections, preserve the earlier version and notify the people using affected reports or downstream data.

For example, a package lead could confirm that the installed quantities were checked against signed field records through the agreed cutoff, with two unresolved items listed separately. The controls reviewer records that confirmation with the dataset and accepts it for the period report. This makes the review specific enough for another person to understand and repeat.

Treat sign-off as a conversation about the evidence. If a reviewer cannot explain a significant movement, give them a route to hold or qualify the submission while it is investigated. Trust grows when people consistently make the checks visible, acknowledge what is uncertain and correct mistakes openly.

The wedding of the rivers: validate the combined picture

Orion Matthews uses the "wedding of the rivers" as an analogy for project data integration. A checked schedule meets cost, progress, risk and manual adjustments. Each stream may make sense on its own, yet the combined report can still be wrong. No discipline has a monopoly on clean data; the problem is what happens where the streams meet.

For a hypothetical December report, imagine joining progress measured on 31 December to costs through 25 December and a forecast revised on 4 January. Calling the result "December" hides three different positions. Another trap is mapping one package budget to several schedule activities and then adding that budget once per activity. The files can pass their individual checks while the combined number is misleading.

• Check the meaning, unit, level of detail, cutoff and version on both sides of every connection.
• Test missing matches, duplicate keys, one-to-many joins and reconciled totals before accepting an integrated result.
• Name an owner for the combined dataset as well as owners for each contributing source.
• Preserve the source records and mapping versions so an error can be traced back through the join.

Automation can repeat a bad join just as reliably as a good one.

When should the data move into a warehouse?

Start designing the integration when recurring reports depend on several systems, repeated manual reconciliations, historical comparisons or multiple downstream users. A governed warehouse can be useful then. A simpler controlled dataset may be enough for a small, stable reporting need. Decide from the work, ownership and history you need to support, rather than waiting for every source to be perfect or buying a platform first.

Capture and release are different decisions. Source files can arrive early in a restricted landing area. Record their origin, extraction time, cutoff and version, then check and reconcile them in a working area. Only promote a dataset to the approved reporting layer when its release criteria are met and exceptions have an owner. A warehouse can contain raw, working and approved data; readers must be able to tell which they are using.

• Keep recoverable source evidence separately from disposable processing tables, under the agreed retention policy.
• Record each harvest or load and its mapping version, checks, rejected records and approval status.
• Publish an explicit period, snapshot version, source coverage and exception list to reporting, exports and AI consumers.
• Before changing a code mapping or calculation, identify affected reports and models and test the proposed revision against the issued position.

If a load fails, retain the previous approved dataset with its original timestamp and a visible stale-data notice. Do not combine half of a new harvest with half of the old one and label it current. Reprocess affected dependencies, reconcile the totals, then release and notify consumers together. A warehouse is not automatically an archive, and a downstream dashboard refresh is not approval.

Run the harvest: collect, review, correct, repeat

The presentation's harvest animation shows source data moving into a warehouse and then into reports for review. Updates feed another harvest. A final harvest is locked for historical comparisons. Treat the illustrated rounds as a working pattern; the number of review cycles should fit the project and the reporting deadline.

• Collect the agreed inputs for the reporting period and record their source versions and cutoffs.
• Run checks and review the resulting reports with people who understand the work.
• Resolve discrepancies with source owners, then ingest the corrected information into the next harvest.
• Decide when the dataset is ready to issue and preserve the resulting reporting position.

Include content checks in the harvest. For drawings, compare the file name with the document number, sheet and revision shown inside it; check the expected sheet set for omissions and duplicates. Keep extracted quantities linked to their source drawing so reviewers can inspect a discrepancy before it reaches procurement or a work package.

For a hypothetical equipment package, a changed delivery date should be traceable to its source update and harvest. The report author should be able to explain why this issue differs from the previous one, without rebuilding the answer from emails.

Preserve a reporting position you can reproduce

Keep the current operational view separate from the dataset behind an issued report. Identify the effective cutoff, source versions, harvest, approval and report edition. Also record when information was received and accepted: a correction received in January may describe work performed in December. Keeping both dates helps distinguish what happened from what the team knew when it issued the report.

A period lock prevents ordinary changes to a designated approved version. Apply the control to imports, integrations and automated refreshes as well as on-screen edits. Define who may authorize a revision and how privileged changes are logged. A read-only dashboard is not enough if the tables underneath it can silently change.

• Preserve the approved data, relevant source evidence, mappings and calculations needed to reproduce the issued report.
• Keep change evidence: actor, time, before and after values, reason and authorization. An audit log serves a different purpose from a backup.
• Process a late correction through an identified new revision; retain the original as-issued record according to the records policy.
• Reconcile affected figures and conclusions, reapprove the revision and tell downstream users which edition supersedes which.

Locking preserves a state; review establishes whether it is fit to use. A file hash can help detect changes against a trusted reference, but cannot prove that a number was true. Define and test the whole control, including administrator access and restoration, rather than relying on the word "locked".

Treat reporting data as a project record

Reporting data can become part of the evidence for what was submitted, known, approved or changed. Manage the supporting records alongside the report: source submissions, review comments, approvals, transformations and issued versions. Preserve enough context to explain a figure later, while limiting access to information people need for their role.

Agree the applicable contract, retention, confidentiality and preservation requirements with the project's commercial, legal and records owners. Record what must be kept, where, for how long and who can authorize disposal. If a preservation obligation or legal hold applies, suspend affected deletion and overwrite routines across source systems, warehouse copies and downstream exports as directed. Keeping every copy forever is not a substitute for a records policy.

The practical aim is a traceable record and controlled correction process. Decide the applicable requirements before configuring the technology, then check that evidence remains retrievable when people, systems or suppliers change.

Advanced practice: make history useful for analysis and AI

Consistent records can support comparisons across periods and project stages. Before comparing, check whether scope, definitions and measurement methods match. Keep in-progress forecasts distinct from completed-project outcomes. Preserve the explanation of unusual conditions so a future analyst does not mistake a change in definition for a change in performance.

Retain the explanation alongside the numbers: what scope was included, how progress was measured and which conditions changed. When reviewing a completed project, compare the record with several people's accounts and keep unresolved differences visible. Future teams need to understand why a result is comparable before using it as a benchmark.

For an AI application, begin with the same foundation: known sources, understandable fields and appropriate access. Test any proposed extraction, mapping or commentary against representative project examples. Keep suggestions distinguishable from approved records.

Lessons from the field: operate the system together

The presentation includes standard communications and user training as part of the reporting setup. Make requests recognizable, explain the purpose of each submission and show contributors how their information is used. A documented process needs people who understand their responsibilities and can raise problems before the reporting deadline.

Use that pairing in routine reviews. Let a delivery specialist walk through what happened in the field while an analyst follows the same work through its codes, source records and calculations. Together they can spot a technically valid result that does not describe how the work was actually done.

• A reader can trace one reported figure to its source, definition, period and review state.
• A source owner knows how to correct a submission and when the correction will appear.
• The team can reproduce an issued reporting position after operational data changes.
• Someone owns the dictionary, system interfaces, access and ongoing improvement.
• A delivery specialist and a data specialist can follow the same piece of work from the field into the reported result.