Key Takeaways
- Out of seven trialled solutions, Fivetran proves to be the only one that can successfully sync and update GuruShots' large amount of event data.
- With Fivetran, the development and operations team regains the 25-50% of its time previously spent on data migration.
- Generating a basic report used to take three hours; now it takes one minute.
- By analysing reports in Tableau, the business was able to improve voting, the UI, and overall insight into user activity in real time — a project it had been waiting for over a year to be able to do.
Data Stack
- Pipeline: Fivetran
- Sources: Freshdesk, Intercom, MySQL
- Destination: Snowflake
- BI Tool: Tableau
- Systems Integrator (SI): iKnowlogy
GuruShots is a platform for people who love taking and sharing photos. Its mission is to reinvent the way that people interact with and share their photos. Users can compete in challenges, vote on photos, give feedback and win prizes. With over three billion votes per month, the Israeli startup had a lot of event data that it was struggling to centralise.
The dev ops team had to write its own scripts, make backups of its MySQL database, send the backups to S3, change the format of the records, and then load them into Snowflake. “It was not stable, it was not something that we could trust,” Eran Hazout, CTO of GuruShots, explains. “Every time we had to update the data, or something broke, we needed to remigrate all of the data. If we changed something or added another table we would have to rewrite all of the scripts.”
Solving Data Ingestion Issues With Fivetran
Hazout trialled about seven different solutions for centralising GuruShots’ MySQL data into Snowflake. Working with its SI, iKnowlogy, GuruShots created a Fivetran free trial through Snowflake Partner Connect, an easy way to set up Fivetran connectors through the Snowflake UI. Fivetran was the only data replication solution that successfully synced historical data, handled row updates, and had a clear, upfront price. Once all of the data was in Snowflake and reliable, Hazout was sold.
“Fivetran has helped us a lot," Hazout says. "It saves a lot of time and a lot of money. We were planning to build the system internally to do what Fivetran does for us, but Fivetran can handle our record updates.”
Using Fivetran eliminated the need to hire another employee to handle the migrations, something GuruShots had been planning to do. Further, Fivetran saves the Dev Ops team at least 10-20 hours a week that were previously spent on data migration.
Making the Most of Snowflake
“Snowflake is an incredible product," Hazout says. "We tried many other solutions, but Snowflake by far is much better and much faster. Things that used to take a few hours in the past now take one or two minutes in Snowflake. The price you pay for the benefits you get is incredible. It doesn’t make sense to use any other solution.”
GuruShots also uses Snowflake to easily handle and modify large volumes of records to meet changing business needs: “Say we have a table with one billion records and we want to change the columns or the schema of the table. We export the data from Snowflake into a CSV and create a new table. Using only MySQL, this could take days and now we can do it with Snowflake and MySQL together and it takes one hour.”
According to Hazout, Fivetran and Snowflake work perfectly together, and the combination has saved the company money: “Our monthly spending on Snowflake is significantly lower with Fivetran. We used to have to load the data for a few hours from S3 into Snowflake. Now, Fivetran directly loads the data to Snowflake much more quickly.”
Quicker Reporting
With all of the data in one place, the business uses Tableau to analyse it. “Tableau is a nice product and works well with Snowflake,” Hazout says. “Most people in the company use Tableau on a daily basis. We use it mainly to understand if everything is working well and to see the activity in real time.”
Prior to Fivetran, the BI team had trouble conducting impactful analysis. It would take a lot of time to run scripts and migrate the data again to make sure the team had the data to work with. “Now, we don’t have to wait around for the data. We are able to generate reports significantly more quickly,” says Hazout. “Previously, to get the daily active users and their activities we would have to come in early in the morning, press the button to sync the data, and give it two to three hours to generate the report. Now it takes one minute.”
In addition to quicker reporting, a modern data stack has enabled the business to generate more reports. “We either had to wait a long time for reports or just not have them at all,” Hazout explains. “For instance, we couldn’t gain insights into the activity or number of users voting on a challenge. By the time we migrated the billions of events into Snowflake, there would already be a huge margin between the report and the real-time data, so we never understood the real-time activity. With Fivetran all we have to do is click ‘sync now’ and within 10-15 minutes we can curate the data in Snowflake and visualize the insights with Tableau.”
Optimising User Experience
Having the event data has enabled the business to tackle projects that it had been unable to do in the past. GuruShots wanted to change the algorithm supporting its voting system — a critical component of its product. But in order to accomplish this, it needed the data and reports in real time from MySQL into Snowflake. That way, the team could make the adjustments to the algorithm and use Tableau to create reports to see how the users were behaving and responding to those changes in near-real time. “We were waiting to do this project for about a year,” Hazout says. “With Fivetran, Snowflake and Tableau it was simple to access the data and see the responses, as a result we were able to optimise the voting system.”
Ready to start centralising your data for improved business decisions and processes? Sign up for a personalised demo to see Fivetran in action or begin your free trial today.
About Fivetran: Shaped by the real-world needs of data analysts, Fivetran technology is the smartest, fastest way to replicate your applications, databases, events and files into a high-performance cloud warehouse. Fivetran connectors deploy in minutes, require zero maintenance, and automatically adjust to source changes — so your data team can stop worrying about engineering and focus on driving insights.
About Snowflake: Snowflake is the only data warehouse built for the cloud, enabling the data-driven enterprise with instant elasticity, secure data sharing and per-second pricing, across multiple clouds. Snowflake combines the power of data warehousing, the flexibility of big data platforms and the elasticity of the cloud at a fraction of the cost of traditional solutions. Snowflake: Your data, no limits.
About Tableau: Tableau helps people see and understand their data. The Tableau platform provides the breadth and depth of capabilities that enterprises need, and adapts to your environment with unmatched flexibility and choice, while meeting the toughest governance and security requirements. People love using Tableau because it is both powerful and intuitive. Tableau leads the industry with the most passionate user community, over 86,000 customer accounts, and a commitment to customer-focused innovation.
About iKnowlogy: iKnowlogy is a professional services company founded by seasoned data scientists with rich experience in planning and implementing advanced, robust and scalable end-to-end BI, Big Data and data-science solutions. iKnowlogy’s goal is to help companies get business insights from data and focus on growth by addressing the business needs first. We build data-solutions and provide insights that focus on growth, instead of just handling the implementation of complex technical systems.
.png)
.svg)
.svg)
.svg)