Why I Built This
Industrial product data is a mess.
A supplier sends you a PDF. Another sends a spreadsheet. Your old ERP has a CSV export. All three describe the same product, but the values don't match. The PDF says the maximum pressure is 600 psi. The spreadsheet says 500 psi. The ERP has nothing.
What do most tools do? They pick one value and throw the rest away. You never know which one was chosen or why.
That bothered me. In industrial settings — valves, bearings, electrical equipment, HVAC — a wrong spec is not just a data quality issue. It's a liability. So I built Truvo: a platform that reads all those files, keeps every value, shows you the conflicts, and makes sure nothing important is hidden.
The core idea is simple: every value must know where it came from, and humans must stay in control of what gets approved.
Architecture & Monorepo Design
Truvo is a TypeScript monorepo with three core services:
| Service | Tech Stack | Responsibilities |
|---|---|---|
| web/ | React + Vite + Tailwind | Workspace UI, review queue, catalog visualizer |
| api/ | Express REST API + Prisma | JWT auth, file upload, business routes, discovery API |
| worker/ | BullMQ + Redis | Async document parsing, LLM extraction, normalization, conflict engine |
| prisma/ | PostgreSQL + Prisma | Relational schema, attribute history, seed data |
Why split into three services? Processing a 1,000-row spreadsheet or a dense PDF can take seconds or minutes. If the API waited for that, every upload would block. By pushing work into a BullMQ queue backed by Redis, the API responds immediately and the worker processes the file independently. When Redis is not available (like Upstash long-polling limits), the API falls back to executing processing in-process gracefully.
The Processing Pipeline: From Raw Files to Normalized Records
When a file is uploaded, Truvo extracts structured data across spreadsheets and PDFs:
Spreadsheets (XLSX/CSV): The worker iterates every row, resolves product identity by MPN/SPN, matches headers to canonical AttributeDefinitions, parses and normalizes units (e.g. converting psi to bar or inches to mm), and stores values through the trust rules engine with exact sheet and row coordinates.
PDFs (Text & Vision): Text-layer PDFs are extracted via pdf-parse and structured with Anthropic Claude tool schemas. Scanned PDFs render pages via pdftoppm and pass base64 frames to vision models. When no AI API key is present, a deterministic rule-based line parser runs as fallback.
Classification: Products are classified against ETIM, eCl@ss, UNSPSC, DIN, and ASME standards using structured AI tool calls or deterministic keyword frequency scoring.
The Trust Rules Engine: Enforcing Data Integrity
This is the core foundation of Truvo. When a new value arrives for an attribute that already has values, the system does not blindly overwrite. It runs through explicit invariants inside appendAttributeValue():
- Rule R3 (Never overwrite verified data): If a human approved a value, new incoming data becomes a competing suggestion in the review queue — the approved value stays untouched.
- Rule R4 (Detect and surface conflicts): If two sources disagree on an attribute, a Conflict record is generated for human review.
- Source Trust Ranking: Files receive a trust rank (1 = Manufacturer, 2 = Distributor, 3 = Internal). Lower rank numbers take precedence for initial unverified display.
- Conflict Tolerance: Numeric tolerance thresholds (e.g. within 1%) prevent noisy flags on minor rounding differences.
Intelligence, Review Hub & Downstream Integrations
Truvo includes advanced catalog intelligence modules:
2-Phase Duplicate Detection: Uses blocking keys (manufacturer + MPN prefix) followed by Levenshtein distance scoring (60% MPN + 40% Name similarity) to find duplicate products without O(n²) quadratic explosion.
Anomaly Detection with MAD: Uses Median Absolute Deviation instead of standard deviation to detect extreme spec outliers without being skewed by the outliers themselves.
Dynamic Supplier Scoring: Evaluates supplier accuracy over time based on verification rates, reviewer corrections, and conflict win rates to dynamically adjust source trust ranks.
Review Queue Hub: All conflicts, anomalies, duplicates, drift alerts, and cross-catalog matches funnel into a unified review inbox with complete context and 1-click resolution.
Discovery API & Webhooks: Exposes approved records via a JSON-LD REST API with cryptographic auditHash verification and scoped API keys.
The Seven Design Rules Enforced in Code & Key Learnings
These rules are directly enforced in the codebase:
- 1. Never hide the source: Every ProductAttributeValue has sourceId and sourceLocation.
- 2. Never present a suggestion as a fact: Inferred values are status: 'inferred' and isCurrent: false.
- 3. Never silently overwrite approved data: Rule R3 is enforced on every database write.
- 4. Never hide a conflict: Every disagreement generates a review queue item.
- 5. Never claim certainty without support: Confidence scores reflect method and source quality.
- 6. Keep a full history: AttributeHistory records actor, timestamp, old/new value, and reason.
- 7. Keep people in control: Automation accelerates ingestion; humans make the final decisions.
“Product data problems are not just data problems — they're trust problems. Truvo makes that trust visible, traceable, and manageable.”
