Search

SQL Server Parallel Data Warehouse (PDW) is Microsoft’s massively parallel processing (MPP) offering and is available as part of Microsoft’s Analytics Platform System (APS). APS is a turnkey solution focused on big data analytics. It offers two regions, or software options, to customers: PDW and HDInsight, which is Microsoft’s 100% Apache Hadoop distribution. PDW is built upon the SQL Server platform, although it is a separate product with a build of SQL Server specifically designed to support MPP operations.

Data Quality Services (DQS) was first introduced in SQL Server 2012. It provides data correction and data deduplication functionality—key components for most Extract, Transform, and Load (ETL) processes. This chapter describes how DQS integrates with SSIS and provides patterns that enable you to achieve reliable, low-effort data cleansing within your ETL packages.

SSIS frameworks are the next logical step after SSIS design patterns because frameworks comprise many patterns. At a minimum, an SSIS framework should provide package execution control and monitoring. That sounds simple, but I assure you, it is not. Execution control requires a working knowledge of the Integration Services runtime API. Understanding the subtleties of tight—and loose—coupling is not merely helpful, it can make or ruin your day (or data integration solution).

Data Quality Services (DQS) is a new product in SQL Server 2012 that provides data cleansing functionality — a key component for most ETL processes. This chapter describes how DQS integrates with SSIS, and provides patterns that enable you to achieve reliable, low effort data cleansing within your ETL packages.

In earlier versions of Integration Services, the data movement platform did not include a management framework, which is the implementation of the execution, logging, and deployment of the Integration Services packages. To try to fill this hole, developers created their own management framework to use in their organizations. As with any custom solution, the developers found that the management framework needed to be cared for and upgraded when new versions or new packages were introduced into the system.

You likely purchased this book to learn how to be a more productive SQL Server Integration Services (SSIS) developer. I applaud your desire and decision, and I sincerely hope the information contained herein has provided ideas and information to help you be more productive. I am always on the lookout for ways to become a better data integration developer. Specifically, I seek out ways to improve code quality and reduce the amount of time required to build solutions. Those goals are what motivated me to begin practicing patterns-based development in the first place, which eventually led to the idea for this book.