Back to Insights
Case Notes

A telematics data layer rebuilt on Elastic: 20× faster ingestion, ~94% lower infrastructure cost, and reports in seconds instead of weeks.

DP
Dillon Peterson
Founder & CEO
Published
April 9, 2026
Reading time
8 min read
A telematics data layer rebuilt on Elastic: 20× faster ingestion, ~94% lower infrastructure cost, and reports in seconds instead of weeks.

As Prime, StandardData rebuilt the data layer behind Par-Tech's TDP fleet-management and vehicle-diagnostics platform on Elastic. Ingestion throughput scaled roughly 20× — from about 600 to about 12,000 documents per second. Infrastructure operating cost dropped by roughly 94%. And a full report that used to take on the order of two weeks now returns in about 15 seconds. We handed the whole package — code, documentation, and knowledge transfer — to the Par-Tech team in September 2025.

Par-Tech, Inc. runs TDP, a commercial telematics platform for fleet management and diagnostics. The data infrastructure had grown costly and slow. A PHP/Laravel service layer read from MySQL plus MongoDB; full scans ran 7.5 to 112 minutes, full logical backups ran two and a half to seven and a half hours, and historical access was capped at 30 days. In the words of our own deliverable document, “Par-Tech was originally in a state where data was costly to store, difficult to analyze, and nearly impossible to rapidly visualize.” Our job was to fix all three without breaking the application on top.

What the program got

20×
Faster ingestion (600 → 12,000 docs/sec)
~94%
Lower infrastructure cost
~15s
Report time (was ~2 weeks)
100%
Output parity vs. legacy, verified field-by-field

What we built

We re-platformed TDP's timeseries data layer onto Elastic — Elasticsearch plus Kibana — deployed inside Par-Tech's own Azure environment, removing the workload's reliance on MySQL and MongoDB. The rewrite was deliberately surgical. We moved the TDP V1 fleet-service timeseries and snapshot API in full from PHP/Laravel to a Python data service that queries Elastic natively, and we proxied it behind the existing fleet-service so authentication middleware and API routes stayed exactly where the application expected them.

Underneath, two pipelines feed the cluster. Binary telematics logs land in Azure Blob, fire an EventGrid signal into an Azure Function, and stream into Elastic at roughly 100 GB/hour — about 2 TB/day of full-resolution data. A 5-minute downsampling transform aggregates gauge metrics into a compact stream so hot-tier snapshots stay fast while full resolution is preserved in cheaper storage. Separately, 100-plus MySQL production tables sync into Elastic through a single self-managed Logstash deployment polling every five minutes — one config replacing a tangle of per-table connectors.

  • Tiered storage with lifecycle management: hot / warm / cold / frozen tiers, with shards scaled from 1 to 3 across availability zones to clear single-node overload.
  • Elastic upgraded 8 → 9 mid-project to unlock built-in downsampling and newer Kibana — a version jump executed without losing the migration's footing.
  • Full Kibana visualization parity with the legacy React app — pie charts, geospatial branch groupings, usage dashboards — with CSV-exporting drilldowns, delivered as version-controlled infrastructure-as-code.
  • 4 operational rule-based alerts (ingestion-error, error-reason code, and prognostic alerts on high engine speed and fuel rate), within the scope's up-to-five-fault allowance.

Why it matters — four reasons this approach is different

01

Throughput that clears the backlog

Raising hot-tier CPU and adding nodes cleared the 429 throttling errors and scaled ingestion roughly 20× — about 600 to about 12,000 documents per second — sustaining ~100 GB/hour of full-resolution telematics.

02

Cheaper, by lifecycle design

Hot/warm/cold/frozen tiering on a right-sized Elastic cluster cut total infrastructure operating cost by roughly 94% versus the prior platform — cost control built into where the data lives, not bolted on after.

03

Answers in seconds

A report that once took on the order of two weeks now returns in about 15 seconds on full-resolution data. Cold-tier full-resolution queries land in 10–15 seconds — in line with the expectations we set with the client.

04

Migrated without breaking the app

Full output parity with the legacy PHP/Mongo system — verified empirically by replaying production data and comparing JSON snapshots field by field, not by assertion.

From two weeks to fifteen seconds

The speed gain is the part operators feel first. The legacy lifecycle answered timeseries questions through full scans that ran anywhere from 7.5 to 112 minutes, and a full report could take on the order of two weeks to assemble. The replatformed system answers across tiers: hot in 0–10 seconds, warm in 10–30 seconds, cold in 30–120 seconds, frozen in 3–15 minutes. Full-resolution data that used to be effectively unqueryable now returns in roughly fifteen.

Query latency by storage tier (delivered)
Legacy full scan7.5–112 min
Frozen tier3–15 min
Cold tier30–120 sec
Warm tier10–30 sec
Hot tier0–10 sec

Delivered tier latencies vs. legacy full scan. Bars are illustrative, not to absolute scale. Source: TDP Timeseries Data Layer Migration deliverable.

How we know it works

A migration is only as good as its proof of equivalence, and we verified ours empirically rather than by assertion. We replayed production data through both the legacy PHP path and the new Python data service and compared JSON snapshots field by field, so “full parity” was a measured result, not an aspiration. On the visualization side, every Kibana chart was checked against its TDP original by organization, so a fleet's dashboards looked and behaved the same on the new stack as the old.

StandardData has maximized Par-Tech's data leverage and usability in this project. Par-Tech was originally in a state where data was costly to store, difficult to analyze, and nearly impossible to rapidly visualize. We leave Par-Tech in a state with continuous data ingestion, cost optimization through effective lifecycle management, and maximal data leverage.StandardData, TDP Timeseries Data Layer Migration deliverable — Background.

Why this matters beyond telematics

The shape of this problem is common. A platform accumulates data faster than its original database design can serve it; queries slow, costs climb, and the analytics the business actually wants become impractical. The fix is rarely a rewrite of the whole application. It is a careful re-platforming of the data layer — right-sized storage tiers, a native query path, a clean ingestion pipeline — done behind the existing API so nothing upstream has to change on day one.

  • Legacy-to-cloud data migration with the application kept intact — the new Python data service sat behind the existing fleet-service proxy, preserving auth and routes.
  • Continuous, high-volume ingestion — ~100 GB/hour streamed into Elastic with a 5-minute downsampling transform keeping hot-tier snapshots fast.
  • Cost control by lifecycle design — hot/warm/cold/frozen tiering that makes long-horizon history affordable to keep instead of something you delete at 30 days.
  • Proof of equivalence — output verified field-by-field against the legacy system by replaying production data, not asserted.
  • A clean handover — code, documentation, and knowledge transfer delivered so the client's own team owns it.
…exceeded our every expectation. Look no further than the talent at Standard Data.Dave Parker, President, Par-Tech, Inc. (written testimonial).

If your platform has these contours — a database straining under timeseries scale, climbing infrastructure cost, and analytics that have become impractical to run — we are happy to walk through how this work would map to your environment.

DP
Written by

Dillon Peterson

Founder & CEO

Founder of StandardData. Writes about AI adoption, federal practice, and the work that has to happen before a model ships.

Field notes from the work.

One letter a month. AI adoption, deployment, and the parts that don't fit in a pitch deck.

No spam. Unsubscribe anytime.

Book a call