Today’s landscape for data warehousing is a rapidly evolving space and most of that evolution is happening in the cloud. The cloud has long offered ways to scale analytic workloads for data warehousing with services like Azure SQL Data Warehouse. However, this only solves one part of the data warehousing workload and as the complexity and velocity of data ingestion grow, there are tools needed to be able to scale with the ingestion and orchestration of data. Azure offers the Azure Data Factory service to be able to handle this task. This service is the data orchestration tool of choice that can handle both the constantly shifting cloud data sources and terabytes of flat files both structured and unstructured. It can also spin up other Azure resources such as HDInsight for your Spark jobs or Data Lake Analytics for your U-SQL jobs.

Traditional data warehouses have long been a mainstay, but with the increasing amount of data sources and processing demands, ETL tools such as SQL Server Integration Services (SSIS) need the scalability that the cloud can offer. While you may have heard of Azure Data Factory and thought the only way to use this for data orchestration was to rewrite all your SSIS packages, it now has the ability to run your SSIS packages in managed Azure-SSIS Integration Runtimes (IRs), so you can scale the processing to your growing needs. In this eBook we will go over why you would want to migrate your existing SSIS workloads to Azure Data Factory and address common considerations and concerns. We’ll then walk you through the technical details of creating an Azure-SSIS IR and then show you how to upload, execute, and monitor your packages through Azure Data Factory using the tools you are probably are familiar with like SQL Server Management Studio (SSMS).

Sources:

https://azure.microsoft.com/en-us/resources/azure-data-factory-ssis-in-the-cloud/

Last modified: September 9, 2019

Author

Comments

Write a Reply or Comment