Azure Data Factory (ADF) is a fully managed, serverless data integration solution for ingesting, preparing, and transforming all your data at scale. It enables every organization in every industry to use it for a rich variety of use cases: data Engineering, migrating their on-premises SSIS packages to Azure, operational data integration, analytics, ingesting data into data warehouses, and more.
Azure Data Factory enables organizations to ingest data from a rich variety of data sources. Whether the data source is on-premises, multi-cloud, or provided by Software-as-a-Service (SaaS) providers, Azure Data Factory connects to all of them at no additional licensing cost. Using the Copy Activity, you can copy data between different data stores.
For many organizations, there will be enterprise data sources that are on-premises. Azure Data Factory enables organizations to connect to these on-premises data sources using a Self-Hosted Integration Runtime (we will cover the Integration Runtime concept in the next section). The Self-hosted integration runtime enables organizations to move data between on-premises and cloud data sources, without requiring you to open any incoming network ports. This makes it easy for anyone to install the runtime and enable hybrid cloud data integration.
Azure Data Factory enables any developer to accelerate the development of data transformations with code-free data flows. By using the ADF Studio, any developer can design data transformation without writing any code. To design a data flow in Azure Data Factory, you first specify the data sources that you want to get data from, and then you can apply a rich set of transformation on the data, before writing it to a data store. Underneath the hood, Azure Data Factory runs these data flows for you at scale using a Spark cluster. Whether it is working with megabytes of data (MB) to terabytes of data (TB), Azure Data Factory runs the data transformation at spark scale, without you having to set up a Spark cluster, or tune it. In many ways, the data transformation
Azure Data Factory supports secure data integration, by connecting to private endpoints that are supported by various Azure data stores. To offload the burden of managing your own virtual network, Azure Data Factory manages the virtual network underneath the hood. This makes it easy for you to set up a Data Factory and making sure all data integration happens securely in a virtual network.
Azure Data Factory enables any developer to use it as part of a continuous integration and delivery (CI/CD) process. CI/CD with Azure Data Factory enables a developer to move Data Factory assets (pipelines, data flows, linked services, and more) from one environment (development, test, production) to another. Out of the box, Azure Data Factory provides native integration with Azure DevOps and GitHub.
Many resources are available on Azure Docs and Azure Data Factory YouTube Channel to help the technical community get started with Azure Data Factory. In addition, you can get started by using the Azure Data Factory Learning Path.
We can’t wait to see what you can build with Azure Data Factory!
Copyright 2021 Acropolix Technologies.