Challenges
As the business expanded into new markets, the amount of data flowing through the platform grew quickly. Subscription activity, customer preferences, delivery operations, recipe data, and supply-chain information all needed to be processed at a much larger scale.
Snowflake was already supporting the company’s analytics needs, but the growing mix of analytics, machine learning, and operational data meant the team wanted a more unified environment. Managing different data workflows separately was gradually adding more effort to day-to-day operations.
The company also needed faster access to fresh data. Logistics teams depended on timely information for deliveries and cold-chain monitoring, while analytics teams needed updated data to understand subscriptions, customer behaviour, and business performance.
The organization wanted a unified lakehouse architecture capable of supporting analytics, data engineering, streaming workloads, and machine learning from a single platform instead of maintaining separate processing environments.
At the same time, machine learning was becoming a bigger part of the platform, particularly for recipe recommendations and customer insights. The next step was therefore not simply handling more data, but creating a foundation where analytics, operational data, and machine learning could grow together without adding unnecessary complexity.
Solution
Lucent Innovation worked with the company to plan and carry out the move from Snowflake to Databricks in phases, keeping the transition manageable while the business continued to operate across multiple markets. As part of its Databricks migration and optimization services, the team reviewed existing workloads and prioritized subscription, customer, logistics, and supply-chain data for migration.
The new Databricks environment was structured using a Bronze, Silver, and Gold architecture. Raw data entered the Bronze layer, was cleaned and organized in the Silver layer, and then prepared in the Gold layer for reporting, analytics, and machine learning. This gave teams access to data in the form they needed while keeping the overall flow consistent and easier to scale.
The next step was bringing analytics, data processing, and machine learning workflows into a more connected environment. This made it easier for different teams to work with the same data without maintaining several separate processes.
Data pipelines were also reorganized so fresh information could reach teams sooner. Logistics teams gained quicker access to delivery and cold-chain data, while business teams could work with more current subscription and performance insights.
Finally, machine learning workflows for recipe recommendations and customer insights were brought closer to the wider data platform, creating a stronger foundation for expanding AI and analytics as new use cases emerge.
