Sunday, 01 November 2020

Fivetran Integrates with dbt for Automatic Data Transformations

posted by Fivetran
Tuesday 22 September 20

Fivetran, the leading automated data integration provider, today announced an integration with dbt, the popular open-source analytics engineering tool maintained by Fishtown Analytics. This integration, called Fivetran dbt Transformations, enables Fivetran customers to orchestrate the cleaning, testing, transformation, modelling and documentation of data sets to make data immediately ready for analysis. Because Fivetran already automates the ingestion of data into cloud data warehouses and is now also able to orchestrate the handoff of data to dbt for transformations…

Fivetran, the leading automated data integration provider, today announced an integration with dbt, the popular open-source analytics engineering tool maintained by Fishtown Analytics. This integration, called Fivetran dbt Transformations, enables Fivetran customers to orchestrate the cleaning, testing, transformation, modelling and documentation of data sets to make data immediately ready for analysis. Because Fivetran already automates the ingestion of data into cloud data warehouses and is now also able to orchestrate the handoff of data to dbt for transformations, customers save time and money that would have been spent writing complex Python scripts and managing orchestration tools for their data integration and analytics engineering processes.

“The importance of data analytics continues to be paramount for growing companies, and the exponential growth in data at our fingertips only means there is more to harness, store, and analyse to make revenue-impacting business decisions,” said Martin Casado, general partner at Andreessen Horowitz, an investor in both companies. “Fivetran and dbt share a mission to simplify the complexities of data integration and transformation, and speed up time to data-driven decisions. This integration sets the foundation for a modern data stack in global enterprises that is poised to become the de facto standard ELT process across the industry.”

Whereas traditional data integration tools focus on transforming data before loading it into a data warehouse with a graphical interface (i.e., ETL, or extract-transform-load), Fivetran’s automated data connectors flip the last two processes through the company’s unique ELT (extract-load-transform) approach. Integrating with dbt allows Fivetran to support more sophisticated data operations, with both tools leveraging the full compute power of modern cloud data warehouses.

“By integrating dbt with Fivetran, we are giving customers the ability to manage all their data pipelines within the Fivetran toolset,” said George Fraser, CEO and co-founder of Fivetran. “dbt, with its powerful data modelling capabilities, is an ideal complement to our best-in-class extract and load processes. The integration of both products gives customers an end-to-end solution that makes the entire ELT process as simple and reliable as electricity.”

dbt enables anyone with knowledge of SQL, the primary language of data analysts, to model data following software engineering best practices including version control, peer review, and automated testing. Using Fivetran dbt Transformations and pre-built dbt packages, designed on top of Fivetran schemas, data teams can dramatically reduce time to value.

“Fivetran dbt Transformations is one of the first big commitments made by a vendor in the modern data stack to support dbt as a first-class part of its product. This integration will make it easier for more data analysts to start using dbt, adopt the dbt viewpoint, and join the vibrant dbt community,” said Tristan Handy, CEO of Fishtown Analytics. “We’re incredibly excited to see companies like Fivetran making real bets on dbt as the open standard in data transformation.”

“dbt allows us to set up repeatable data transformations. We can schedule jobs to create data tables for us that surface in downstream tables,” said Evin Anderson, data engineering manager at Autodesk. “With this setup, we should be able to remain agnostic to the specific tools that we use.”

Working with all major cloud data warehouses, including Amazon Redshift, Google BigQuery and Snowflake, Fivetran technology makes the resource-intensive struggle to build and maintain data connectors to each individual data source a thing of the past. Fivetran’s 150-plus pre-built connectors automate the entire process and are updated as APIs and schemas change. As the first data integration company to integrate with dbt, Fivetran is bringing the 10,000+ user dbt community into the Fivetran ecosystem.


View more of the latest press releases from across the industry or post your company's news.

Since you're here...

...the Telecoms industry is characterised by constant change and evolution. That's why it's crucial for telecoms professionals to keep up-to-date with what is happening. Join 35,000+ of your peers and sign up to our free newsletter service today, to be in the know about what is going on. PLUS, as a member you can submit your own press releases!

See all membership options

Please enable JavaScript to view the comments powered by Disqus.

Newsletter signup

Quickly get on board and up to date with the telecoms industry