Data Integration

An integration needs a destination, but it also needs a reason.

Someone may be deciding what can be promised to a customer, identifying which product information needs attention or trying to understand why two reports disagree. Each purpose creates different requirements for detail, timing and accuracy.

We investigate the decisions and activities the information will support. Which records matter? How current must they be? What would happen if a value were missing, delayed or wrong? Who understands the conditions under which it was created?

The answers help define the scope. A support workspace may need selected order information without needing every financial detail. A product catalog may need approved descriptions and variant relationships without needing internal purchasing notes.

Systems Integration considers how the applications and their functions work together. Data Integration looks more closely at the information crossing those boundaries: what it represents, how it relates and whether the result remains useful.

Mapping describes how source information corresponds to its destination. Sometimes that is straightforward. Often it exposes assumptions that need a conversation.

Does “customer” mean a person, an organization or an account? Does an order date describe creation, acceptance or payment? Is a price before or after tax, and which currency applies? Does a blank field mean unknown, not applicable or deliberately withheld?

The technical work can align field names and structures. Microsoft documents explicit mappings between different source and destination fields in Azure Data Factory, for example. That capability implements a mapping; it does not decide whether the two fields mean the same thing. Microsoft's mapping documentation

We establish those meanings with the people who create and use the records. A useful mapping specification includes relevant definitions, identifiers, units, allowed values, transformations and handling for exceptions. It should explain enough of the reasoning that another person can maintain it later.

These questions connect to Information Architecture and Content Strategy. Organization and language are not only concerns for the visible website. They influence whether the information underneath it can travel without becoming misleading.

Bringing records together requires a dependable way to understand their relationships.

A product name may change. Two customers may share a name. One business may have several accounts, locations or delivery addresses. Treating a superficial match as proof that two records are the same can create a more serious problem than leaving them separate for review.

We examine identifiers and agreed matching rules, including how unresolved matches are handled. Some relationships can be established automatically; others need a person with the right context. The implementation should not silently turn uncertainty into a confident merge.

We also establish authority. Which system owns the approved product description? Which owns stock movements? Where is a customer allowed to correct their details? If copies disagree, which rules determine what is accepted and what requires investigation?

A single source of truth is a responsibility to design and maintain. It need not mean one physical database, and it should not become a reason to overwrite valid information without understanding it.

When a ceramics supplier sells to both trade customers and individual buyers, we work with the team to establish what a quantity means at each point in the operation.

A warehouse may record cases while the public store sells individual cups. A trade order may reserve a full case, while damage or quality checks make some pieces unavailable for sale. Different finishes also need to retain their own product identities.

Converting cases into pieces does not, by itself, establish what can be sold. We need the verified pack size, a definition of what the source quantity includes, and an understanding of where reservations, damaged stock and other exclusions are represented.

If the source already reports available stock after reservations, subtracting those reservations again would be wrong. If a case size differs by product, a single fixed multiplier would be wrong. If two finishes have been joined by their shared product-family name, the resulting total could hide which finish is actually available.

We would define the relationships and rules with the team, then test representative records: an ordinary product, a different pack size, reserved stock, a missing identifier and a new finish. The mapping should preserve the distinctions the business needs rather than merely make the numbers fit the destination's fields.

The storefront also needs an agreed response when availability cannot be trusted. That might be to withhold a purchase option, display a qualified message or route a request for confirmation, depending on the business and the experience. “No current answer” should not silently become zero stock—or unlimited stock.

A seemingly simple connection can carry product knowledge, operational rules and a promise to a customer. We make those relationships visible with the team before relying on the transferred values.

Transformation may convert units, normalize dates, translate approved status values or reshape a record for its destination. Combining information can also require preserving where it came from and which version was used.

We distinguish changes that preserve meaning from changes that require a business decision. Reformatting a known date is different from guessing which date convention an ambiguous value uses. Mapping an approved status is different from deciding that two processes have the same definition of completion.

The same care applies to omissions. Removing a field because the destination has no obvious place for it may discard information another process relies on. It is better to make that limitation visible and decide how to handle it than to present a technically successful transfer as complete understanding.

Product Information Management has a connected role here: developing and governing the attributes and content that make a product understandable. Data Integration helps carry the agreed information into use. It cannot supply missing product knowledge simply by moving existing records more efficiently.

A successful run tells us that an operation completed according to its technical conditions. We still need to examine whether the result meets its intended purpose.

Our checks can address several related questions:

• Completeness: Did the expected records and required fields arrive, with exclusions accounted for?
• Validity: Do values meet the agreed format, range and business rules?
• Relationships: Are records connected to the correct products, people, locations or transactions?
• Reconciliation: Can differences between source and destination be explained?
• Usefulness: Can the people relying on the result perform the intended task?

Technical consistency checks help, but their scope matters. Microsoft's copy-activity documentation, for example, describes verification using particular file or row-count checks depending on the transfer. Those checks answer a narrower question than whether a product's availability or a customer's status has been interpreted correctly. Microsoft's data consistency guidance

We retain meaningful examples of expected results and examine exceptions. A report should show what was checked, what the findings mean and what remains uncertain—not just display a reassuring total.

Some information needs to move promptly after a change. Other information can be refreshed on an agreed schedule. The choice should follow the purpose and the capabilities of the connected systems.

We consider how updates, repeated records, delayed arrivals, corrections and deletions are handled. A process needs to distinguish new information from something it has already processed, and an older update should not casually replace a newer accepted value.

Reconciliation helps identify when the sources and destinations have drifted apart. Exception handling gives the team a way to understand and resolve records that cannot be processed as intended. Depending on the situation, it may be safer to hold an affected record for review than to force it through with an invented default.

Workflow Automation can help route those exceptions to the appropriate people. Automation should carry the uncertainty to someone who can resolve it, not conceal it behind a completed status.

An integration should carry what its purpose requires, with access appropriate to the destination.

We consider permissions, credentials, the information exposed in logs, how test data is handled and what happens when information must be corrected or removed. Copying a sensitive field into more places creates responsibilities, not merely convenience.

Those boundaries belong in the design from the beginning. The client and relevant specialists help establish the applicable policies and obligations; implementation translates the agreed requirements into the actual system. A working connection alone is not evidence of compliance.

Data Integration can be a focused engagement around an inherited mapping, a recurring discrepancy or a new destination. It can also form part of a wider CMS, commerce, ERP or custom-software project.

We consider supported connectors, visual tools and custom development according to fit, maintainability, ownership and the client's operating needs. Database design concerns how information is stored and related within a system. Data Migration concerns a transition between systems. They may intersect with ongoing integration without becoming the same task.

Our inquiry follows the adaptable cycle:

Discover → Define → Explore → Design / Develop / Market → Learn → Advance → Discover.

We understand the intended use, define shared meanings and acceptance criteria, test representative mappings and implement the agreed connection. We can work directly or alongside the client's team and specialist partners. Documentation and digital workbooks help preserve the definitions, responsibilities and recovery steps around the technical work.

After introduction, we keep learning from discrepancies and actual use. Measurement Strategy helps determine what would show improvement. Data Visualization can make patterns understandable, but a clear chart should not disguise uncertain source information.

Bring us the conflicting records, the report that needs explaining or the manual reconciliation someone performs every week. You do not need to know which system is wrong before we begin. Understanding the difference is part of the work.

Contact Us