Lakehouse and analytics
One governed copy of your data in open table formats, queried by SQL for regulatory returns and dashboards, by Spark for processing, and by models for AI, without a warehouse licence deciding what you can build.
One copy of the data, every engine reads it
A warehouse holds one copy of the data for reports. A data lake holds another for data science. The AI team keeps a third. Each copy is loaded by its own pipeline, reconciled by hand and licensed separately, and the numbers drift apart.
A lakehouse keeps one copy, in open table formats on object storage, with a catalogue that every engine reads: Trino for SQL and dashboards, Spark for processing and transformation, and the model and retrieval pipelines for AI. Quality checks, lineage and access control sit in the catalogue, so they apply to every reader. The engines are open source and the data is portable, so the platform outlives any one vendor.
When it fits
01
Three copies, three answers
The warehouse, the lake and the AI team's extracts give different figures for the same question.
02
The warehouse licence limits what you build
Every new report or data source means more licensed capacity, so requests are queued or refused.
03
Documents and logs don't fit
Scanned records, call logs and sensor data have to sit beside the tables, and the warehouse cannot hold them.
04
Models need the same data as reports
Data scientists rebuild the finance team's logic in notebooks, and the two never agree.
What we do
Lakehouse platform
Apache Iceberg tables on object storage, with a catalogue, Trino and Spark on OpenShift, built as code and sized for your volumes.
Pipelines and transformation
Batch and streaming loads from the platform's feeds, transformations in dbt and Spark, and orchestration in Airflow, with every run logged.
Data quality and lineage
Tests on every load, a catalogue of every table with its owner and lineage, and alerts when a feed is late or a figure moves outside its range.
Regulatory and management reporting
Returns and dashboards built on governed tables, with the numbers traceable to their source and reconciled automatically.
Warehouse migration
Move reports and marts off a licensed warehouse in waves, running both until each report reconciles.
Analytics and self-service
Grafana and Superset dashboards, and governed access for analysts and data scientists to query the same tables.
The lakehouse
Governed tables that every engine shares
The data lands once in Iceberg tables on object storage. The catalogue holds the schema, the lineage, the quality results and the access rules. Trino, Spark and the AI pipelines all read the same tables through that catalogue, so a report and a model built on the same figure agree, and a change in access applies to both.
- Object storage and tables
- One copy of the data in Iceberg tables on S3-compatible storage, on premises.
- Catalogue and governance
- Schemas, lineage, quality results and access rules, applied to every reader.
- Query engines
- Trino for SQL, Spark for processing, and the AI pipelines, all reading the same tables.
- Reports and AI
- Regulatory returns and dashboards, and the models and assistants, from the same figures.
How a lakehouse programme runs
01
Choose the first domain
Pick one area, such as loans or payments, with reports people depend on and a measurable problem with them today.
02
Design
Design the tables, the catalogue, the quality rules and the access model, and size storage and compute from the volumes.
03
Build and reconcile
Build the platform and the first pipelines as code, load the domain, and reconcile every report against the old source until it matches.
04
Extend domain by domain
Add the next domains and feeds, migrate reports off the old warehouse in waves, and retire what the lakehouse has replaced.
What you get
- A lakehouse platform on OpenShift: Iceberg tables, a catalogue, Trino and Spark, built as code
- Pipelines for the feeds in scope, with transformations in dbt or Spark and orchestration in Airflow
- Quality tests, lineage and alerts for every table in scope
- The first domain's reports reconciled against their old source
- Dashboards and governed self-service access for analysts
- Runbooks, documentation and training for your data team
What we build with
Open formats and open engines, with enterprise support where you want it.
Storage and tables
- Apache Iceberg
- Apache Parquet
- Dell ObjectScale
- NetApp StorageGRID
- OpenShift Data Foundation
- MinIO
Engines and pipelines
- Trino
- Starburst
- Apache Spark
- dbt
- Apache Airflow
- Kafka Connect
Catalogue and analytics
- Apache Polaris
- OpenMetadata
- OpenLineage
- Grafana
- Apache Superset
Questions about the lakehouse
Do we have to abandon our data warehouse?
No. Many clients keep the warehouse for the reports it serves well and build the lakehouse beside it, moving reports across only when the lakehouse version reconciles. Some never move everything, and that is fine.
Is a lakehouse fast enough for regulatory reports?
Yes for the reports and dashboards institutions here run. Trino on Iceberg handles the queries, and the pipelines prepare aggregated tables for the heaviest ones. We measure the critical reports during the build and tune before go-live.
Can it run on premises and air-gapped?
Yes. Every component runs on OpenShift with object storage in your data centre, with no dependency on an outside service. It runs that way in our lab.
How does this help the AI programme?
The model inputs and the document pipelines read the same governed tables as the reports, through the same catalogue and access rules. That is what makes an AI answer as correct as the report beside it.
AI data platformBuild the one copy everyone agrees on
Tell us which reports disagree and which data you cannot yet use. We'll propose a first domain, the platform it needs and how to measure the result.