Key Takeaways
- Fivetran built the Kafka connector at the request of carwow.
- The robust and simplified data architecture has freed up 30-50% of the data scientists' time previously spent on ETL.
- With its modern data stack, carwow can more quickly access vital measurements of forward-looking performance, including the propensity for customers to inquire or buy.
Data Stack
- Connectors: Google Ads, Bing Ads, Facebook Ad Insights, Apache Kafka, Heroku PostgreSQL, Outbrain, Taboola
- Pipeline: Fivetran
- Destination: Snowflake
- BI Tool: Tableau
carwow takes the frustration out of the car buying process by making it more transparent and hassle-free. The platform enables people to compare cars and get recommendations, and connects them with a high-quality dealer. While it mainly operates in the UK, carwow has expanded to both Germany and Spain within the past two years and continues to grow rapidly.
Suffering From Slow, Unreliable Syncs
Before its current data stack, carwow had numerous sync processes, each owned by different teams, and running them was costly, time-consuming and often inefficient and unreliable. Syncs would break and impact multiple arms of the business. PostgreSQL data was copied into Redshift once a day and the full dump and sync took up to 12 hours. As the business’ data grew, this worsened. People were arriving in the office at 8:00 am and had no updated data to work with, preventing them from doing any up-to-date analysis. As a solution for the delays, carwow connected some of its larger databases through a Fivetran competitor which, while allowing for incremental syncs, still had quite a lot of bugs.
Bringing on Snowflake and Fivetran
carwow decided to migrate its data from Redshift to Snowflake because it was a cheaper, more scalable, and a much more manageable option that handles and reformats unstructured data well. Simultaneously, carwow decided to reevaluate its pipeline. Syncing costs were getting very high and the business wanted to figure out how to better manage the costs.
The cost model of Fivetran with Snowflake was much better for carwow. For a business growing quickly in data size, having a fixed cost was critical. “We don’t have to worry about Fivetran producing a massive cost, whereas many other tools can quietly swallow your budget,” says Isaac, Senior Developer at carwow. “I knew the CFO would like it because I could show him the total contract cost whereas with our previous solution we were three times over our contracted limit and waiting for a massive bill to come through. Fivetran has reduced our costs.”
Additionally, carwow found that Fivetran had a better customer service response than its previous solution, which had quite a long wait time. Tim Hesse, Director of Data at carwow, views the relationship with Fivetran as more of a partnership. In response to the needs of carwow, Fivetran agreed to build a Kafka connector. “We are really happy with the Kafka connector, we have millions of events from Kafka that we sync every day from different tables. We run it every six hours and it has never broken,” says Ioannis Nianios, a carwow Data Engineer.
Quicker and Easier Access to Data
Setting up Fivetran was simple, which was another selling point for carwow. “I set up the original connectors, made logins for everyone and sent them off on their way — that’s how simple it was. Just connect it and leave it alone,” Isaac says.
Since the initial setup, carwow has been able to conduct analysis from more sources, eliminate time spent on ETL, and free up significant resources. Having a more robust and simplified data architecture has made a big difference for the data scientists, who now put the 30-50% of their time previously spent on ETL towards conducting analysis.
The company can more quickly access vital measurements of forward-looking performance, including the propensity for customers to enquire or buy. “In the past this data was more cumbersome to get. We have a greater speed to insight now,” Hesse explains. Additionally, it is now much easier to replicate reporting across territories to mirror insights, which was previously difficult to achieve.
The most critical benefit of Fivetran for carwow has been improved data reliability. “Every morning when you come into work, yesterday’s data is in Snowflake,” Isaac explains. “It used to be that you couldn’t run queries on the previous day’s data because the sync wasn’t finished. That doesn’t happen anymore. Even when the sync is occasionally delayed by a few hours you can still run queries.”
Driving the Automotive Industry Forward With Data
Its modern data stack is now completely in line with carwow’s belief in buying versus building. “We recently wrote our engineering principles and one of them states ‘buy everything we can possibly buy,’” Isaac explains. “We don’t want to run any of our own software if we can avoid it.”
This principle has enabled the business to focus on the data and its mission of changing the industry. As a whole, the automotive industry lacks data — car manufacturers build cars with very little consideration for what the customer wants. carwow generates consumer insight data about buying behaviour, preferences and conversions on a daily basis that it can use to continue driving the automotive industry forward.
Want to see the impact that centralised data can have on your business? The Fivetran team is available to present a demo of our service, or you can sign up for a free trial here.
About Fivetran: Our standardised technology delivers data into your warehouse the right way. Shaped by the real-world needs of data analysts, Fivetran technology supports agile analytics, enabling data-backed decisions across organisations. After a five-minute setup, Fivetran replicates all your applications, databases, events and files into a high-performance data warehouse. Our standardised cloud pipelines are fully managed and zero-maintenance.
About Snowflake: Snowflake is the only data warehouse built for the cloud, enabling the data-driven enterprise with instant elasticity, secure data sharing and per-second pricing, across multiple clouds. Snowflake combines the power of data warehousing, the flexibility of big data platforms and the elasticity of the cloud at a fraction of the cost of traditional solutions. Snowflake: Your data, no limits.
About Tableau: Tableau helps people and organisations become more data-driven. The Tableau platform provides the breadth and depth of capabilities that enterprises need, and adapts to your environment with unmatched flexibility and choice, while meeting the toughest governance and security requirements. People love using Tableau because it is both powerful and intuitive. Tableau leads the industry with the most passionate user community, over 86,000 customer accounts, and a commitment to customer-focused innovation.
.png)
.svg)
.svg)
.svg)