Digital Virgo Replaces 10-Year-Old Legacy Pipeline Solution With Fivetran

Key Takeaways

• Digital Virgo migrates data from an on-site MySQL database to a Snowflake warehouse.
• One full-time engineer is reassigned to strategic projects after replacing in-house legacy pipeline solution.
• Nearly 300 Google Ads connectors across 40 countries sync data every five minutes.
• Fivetran Transformations helps Digital Virgo consolidate Google Ads data for easier and quicker BI.
• Digital Virgo uses Fivetran to provide agency clients with on-demand reporting.
• Account managers have an overview of Google Ads policy data in a daily Looker report instead of checking account by account via the Google UI.

Data Stack

• Pipeline: Fivetran
• Sources: Bing Ads, Facebook Ad Account, Facebook Ad Insights, Google Ads, Google Analytics, Snapchat Ads
• Destination: Snowflake
• BI Tool: Looker & Tableau
• SI Partner: Sutter Mills

Digital Virgo offers telecom payment services, monetisation solutions and digital marketing in 40+ countries. The company helps operators, merchants and digital advertising platforms optimise payments; monetise content, services and audiences; and ensure performance through mobile marketing campaigns.

Disparate Data Sources Across the World

Digital Virgo is made up of multiple companies, all of which had been using different data stacks and tools. Its headquarters in France was using an on-site MySQL data warehouse connected to Qlikview, while other regions were using other destinations and BI tools. The CEO launched a project to unify the tools and secure a single source of truth.

At the same time, Digital Virgo was outgrowing its legacy tool to collect data. API maintenance was expensive and teams wanted a simpler solution for managing data integrations. To meet the goals of the CEO and simplify integration, Digital Virgo brought on SI Sutter Mills.

Swapping a Legacy System for Automated ELT

When Digital Virgo developed its in-house pipeline solution nearly ten years ago, it was just starting on Google Ads, and one full-time developer had to work hard to keep up with the evolution of the API. “We reached a size where this system was no longer scalable,” explains Anthony Gianastasio, Head of Analytics at Digital Virgo. “Adding a field could take one to two weeks — it was a mess. The team could only choose 20-30 accounts to sync every five minutes, with the remaining accounts syncing every hour, and we were only able to replicate basic data. The replicated data was highly aggregated because our MySQL database wasn’t able to handle all of the data.”

Google Ads is the company’s most important advertising source, with over 300 Google Ads accounts across nearly 40 countries. Every hour, it pulled Google Ads data into its MySQL database; the data was then processed by its BI team into another MySQL database, which functioned as the company’s legacy data warehouse. “We knew we had to outsource our ETL and focus on analytics,” says Gianastasio. “We needed flexibility for how quickly we could sync our data and wanted it immediately available for visualisation.”

With the guidance of Sutter Mills, Digital Virgo chose Fivetran to replicate its data sources, many of which are advertising sources. It added Snowflake as its warehouse and both Tableau and Looker for its BI. “It took maybe two days to build all of our connectors,” says Gianastasio. “Once we validated the data with the team it was just a matter of going into the UI and adding the connectors. We no longer need to choose which accounts to sync every five minutes, we now get everything quickly.”

Powered by Fivetran, Globally

Gianastasio notes that Digital Virgo invests a huge amount of money into Google Ads. “We collect 1.5 billion hits in Google Analytics per month,” he says. “It’s where everything happens for us. The Google Ads connector was the most critical factor in our decision to go with Fivetran.” Digital Virgo has nearly 300 Google Ads connectors syncing multiple accounts: “We deploy Google Ads resources in each country and centralise the data into Snowflake, so that we can present our acquisition team with a single dashboard with global Google data.” The Powered by Fivetran model makes this affordable and scalable for the business.

Powered by Fivetran is making it easier for the company to expand into a new area: running campaigns for clients. “Last year we acquired an agency, and we have specific things we need to track for each customer, and different campaigns have different dimensions,” explains Gianastasio. “With Fivetran, we have the flexibility to do this without having to code anything. We just set up the connector in the Fivetran UI and can easily adapt to each client use case. It enables us to add new customers quickly and scale this side of our business.”

Transformations to Consolidate Data

Each Google Ads connector syncs five to 10 accounts every five minutes. This enables quick syncing, but results in many tables in the warehouse. Digital Virgo’s data team set up a transformation for each connector to bring all of the data into one consolidated table. “Every time a Fivetran connector syncs the data, it triggers a transformation to bring the new data and write it into the consolidated table,” Gianastasio says. “At the end we have six tables, one for each report type. We can point Looker and Tableau to those tables and we have a consolidated view of all of our Google Ads data synced every five minutes, which is extremely powerful.”

Insight Into Google Policy

Digital Virgo is using both Tableau and Looker across the business. In Tableau, it is building out its core BI — the numbers that the company needs at its fingertips daily, such as sales, turnovers and acquisitions across all 40+ countries. With Looker, the operational teams, like the team managing Google campaigns and the product team managing the websites, can conduct exploratory analysis and use Look ML to get additional details.

Digital Virgo is currently building out dashboards in both tools and delivering them to the team, encouraging everyone to abandon the legacy solution. Gianastasio knows that having on-demand data will be a game-changer for the entire business, especially in monitoring Google Ads: “We were running into issues with the Google Policy bot blocking our ads by mistake, which is a big loss for us. The Google Ads UI doesn’t allow for a big picture of policy status across accounts. You have to check account by account, and given the amount of accounts we manage, that was a nightmare for us. Now the team receives a report from Looker every morning that provides an overview of the policy side across all accounts, without even opening Google Ads.”

Ready to see the positive impact that outsourcing your ELT can have on your business? Sign up for a personalised demo of Fivetran or start your free trial today.

About Fivetran: Shaped by the real-world needs of data analysts, Fivetran technology is the smartest, fastest way to replicate your applications, databases, events and files into a high-performance cloud warehouse. Fivetran connectors deploy in minutes, require zero maintenance, and automatically adjust to source changes — so your data team can stop worrying about engineering and focus on driving insights.

About Snowflake: Snowflake is the leading data warehouse built for the cloud. Its unique architecture delivers proven breakthroughs in performance, concurrency and simplicity. For the first time, multiple groups can access petabytes of data at the same time, up to 200 times faster and 10 times less expensive than solutions not built for the cloud. Snowflake is a fully managed service with a pay-as-you-go-model that works on structured and semi-structured data.

About Looker: Looker is a modern platform for data that offers data analytics and business insights to every department at scale, and easily integrates into applications to deliver data directly into the decision-making process.

About Tableau: Tableau helps people see and understand their data. The Tableau platform provides the breadth and depth of capabilities that enterprises need, and adapts to your environment with unmatched flexibility and choice, while meeting the toughest governance and security requirements. People love using Tableau because it is both powerful and intuitive. Tableau leads the industry with the most passionate user community, over 86,000 customer accounts, and a commitment to customer-focused innovation.