SSIS - SQL Server Integration Services. A Preview

1. SSIS provides data cleansing, extensibility, and interoperability.
2. SSIS comes with a lot of pre-built data-cleansing functionality, including completely integrated functions, such as fuzzy matching and fuzzy grouping, that use algorithms to match or group disparate data to a configurable degree of accuracy.
3. SSIS offers broad extensibility points for third-party component vendors, meaning that if functionality is not available out of the box, SSIS lets you build your own components or add a third-party component to solve your problem.
4. SSIS can load data directly into Analysis Services cubes, and it also offers robust data-mining features for including scalable data-mining model creation, training, and predictions.
5. SSIS is seamlessly integrated with SQL Server Reporting Services integration, which lets you treat an SSIS package as the data source for reporting.
6. SSIS also has greatly improved performance and scalability that allow you to host complex, high-volume ETL applications on lightweight servers, enabling you to scale down.
7. SSIS can also help reduce ETL data staging areas and help minimize performance costs associated with data staging (disk I/O and serial processing). This is made possible by the ability to perform complex data transformations, data cleansing, and high-volume lookups—all inline from source to destination.
8. SSIS also provides a new feature, the Slowly Changing Dimension (SCD) wizard. Through the SCD interface, you can rapidly generate all the steps and required code to add unique handling of history to multiple attributes in a given dimension.
9. The development environment for SSIS, known as Business Intelligence Development Studio, is hosted in Visual Studio, enabling scripting and other programming tasks that take advantage of that enterprise development environment.
10. SSIS now fully supports the Microsoft .NET Framework, allowing developers to program SSIS in their choice of .NET-compliant languages, as well as native code.
11. The Data Transformation run-time engine is exposed both as a native COM object model and as an entirely managed object model. Although the Data Transformation engine is written in native code, it is available though a signed Primary Interop Assembly (PIA) that enables full, managed access to it.

-------------------------------------------------------------------------------------------

The content was copied from the following link, Just in case if you need further information please follow the link below:

In this chapter:

• The Import and Export Wizard
• Creating a Package
• Working with Connection Managers
• Building Control Flows
• Building Data Flows
• Creating Event Handlers
• Saving and Running Packages

Microsoft says that SQL Server Integration Services (SSIS) “is a platform for building high performance data integration solutions, including extraction, transformation, and load (ETL) packages for data warehousing.” A simpler way to think of SSIS is that it’s the solution for automating SQL Server. SSIS provides a way to build packages made up of tasks that can move data around from place to place and alter it on the way. There are visual designers (hosted within Business Intelligence Development Studio) to help you build these packages as well as an API for programming SSIS objects from other applications.

In this chapter, you’ll see how to build and use SSIS packages. First, though, we’ll look at a simpler facet of SSIS: The SQL Server Import and Export Wizard.

If you choose to use the supplied solution files rather than building your own, you’ll need to edit the properties of the OLE DB Connection Managers within the projects to point to your own test server. You’ll learn more about Connection Managers in the "Working with Connection Managers" section later in this chapter.

SSIS Tutorial: The Import and Export Wizard

Though SSIS is almost infinitely customizable, Microsoft has produced a simple wizard to handle some of the most common ETL tasks: importing data to or exporting data from a SQL Server database. The Import and Export Wizard protects you from the complexity of SSIS while allowing you to move data between any of these data sources:

• SQL Server databases
• Flat files
• Microsoft Access databases

• Microsoft Excel worksheets
• Other OLE DB providers

You can launch the Import and Export wizard from the Tasks entry on the shortcut menu of any database in the Object Explorer window of SQL Server Management Studio.

Try It!

To import some data using the Import and Export Wizard, follow these steps:

1. Launch SQL Server Management Studio and log in to your test server.
2. Open a new query window.
3. Select the master database from the Available Databases combo box on the toolbar.
4. Enter this text into the query window:
5. CREATE DATABASE Chapter16
6. Click the Execute toolbar button to create a new database.
7. Expand the Databases node in Object Explorer
8. Right-click on the Chapter16 database and select Tasks > Import Data.
9. Read the first page of the Import and Export Wizard and click Next.
10. Select SQL Native Client for the data source and provide login information for your test server.
11. Select the AdventureWorks database as the source of the data to import.
12. Click Next.
13. Because you’re importing data, the next page of the wizard will default to connection information for the Chapter16 database. Click Next.
14. Select Copy Data From One or More Tables or Views and click Next. Note that if you only want to import part of a table you can use a query as the data source instead.
15. Select the HumanResources.Department, HumanResources.Employee, HumanResources.EmployeeAddress, HumanResources.EmployeeDepartmentHistory, and HumanResources.EmployeePayHistory tables, as show in Figure 16-1. As you select tables, the wizard will automatically assign names for the target tables.
    
    
    
    Figure 16-1: Selecting tables to import
    
    
16. Click the Edit button in the Mapping column for the HumanResources.Department table.
17. The Column Mappings dialog box lets you change the name, data type, and other properties of the destination table columns. You can also set other options here, such as whether to overwrite or append data when importing data to an existing table. Click Cancel when you’re done inspecting the options.
18. Click Next.
19. Check Execute Immediately and click Next.
20. Click Finish to perform the import. SQL Server will display progress as it performs the import, as shown in Figure 16-2.
    
    
    
    Figure 16-2: Import Wizard results
    
    
21. Click Close to dismiss the report.
22. Expand the Tables node of the Chapter16 database to verify that the import succeeded.

In addition to executing its operations immediately, the Import and Export Wizard can also save a package for later execution. You’ll learn more about packages in the remainder of this chapter.

SSIS Tutorial: Creating a Package

The Import and Export Wizard is easy to use, but it only taps a small part of the functionality of SSIS. To really appreciate the full power of SSIS, you’ll need to use BIDS to build an SSIS package. A package is a collection of SSIS objects including:

• Connections to data sources.
• Data flows, which include the sources and destinations that extract and load data, the transformations that modify and extend data, and the paths that link sources, transformations, and destinations.
• Control flows, which include tasks and containers that execute when the package runs. You can organize tasks in sequences and in loops.
• Event handlers, which are workflows that runs in response to the events raised by a package, task, or container.

You’ll see how to build each of these components of a package in later sections of the chapter, but first, let’s fire up BIDS and create a new SSIS package.

Try It!

To create a new SSIS package, follow these steps:

1. Launch Business Intelligence Development Studio.
2. Select File > New > Project.
3. Select the Business Intelligence Projects project type.
4. Select the Integration Services Project template.
5. Select a convenient location.
6. Name the new project ISProject1 and click OK.

Figure 16-3 shows the new, empty package.

Figure 16-3: Empty SSIS package

SSIS Tutorial: Working with Connection Managers

SSIS uses connection managers to integrate different data sources into packages. SSIS includes a wide variety of different connection managers that allow you to move data around from place to place. Table 16-1 lists the available connection managers.

Connection Manager	Handles
ADO Connection Manager	Connecting to ADO objects such as a Recordset.
ADO.NET Connection Manager	Connecting to data sources through an ADO.NET provider.
Analysis Services Connection Manager	Connecting to an Analysis Services database or cube.
Excel Connection Manager	Connecting to an Excel worksheet.
File Connection Manager	Connecting to a file or folder.
Flat File Connection Manager	Connecting to delimited or fixed width flat files.
FTP Connection Manager	Connecting to an FTP data source.
HTTP Connection Manager	Connecting to an HTTP data source.
MSMQ Connection Manager	Connecting to a Microsoft Message Queue.
Multiple Files Connection Manager	Connecting to a set of files, such as all text files on a particular hard drive.
Multiple Flat Files Connection Manager	Connecting to a set of flat files.
ODBC Connection Manager	Connecting to an ODBC data source.
OLE DB Connection Manager	Connecting to an OLE DB data source.
SMO Connection Manager	Connecting to a server via SMO.
SMTP Connection Manager	Connecting to a Simple Mail Transfer Protocol server.
SQL Server Mobile Connection Manager	Connecting to a SQL Server Mobile database.
WMI Connection Manager	Connecting to Windows Management Instrumentation data.

Table 16-1: Available Connection Managers

To create a Connection Manager, you right-click anywhere in the Connection Managers area of a package in BIDS and choose the appropriate shortcut from the shortcut menu. Each Connection Manager has its own custom configuration dialog box with specific options that you need to fill out.

Try It!

To add some connection managers to your package, follow these steps:

1. Right-click in the Connection Managers area of your new package and select New OLE DB Connection.
2. Note that the configuration dialog box will show the data connections that you created in Chapter 15; data connections are shared across Analysis Services and Integration Services projects. Click New to create a new data connection.
3. In the Connection Manager dialog box, select the SQL Native Client provider.
4. Select your test server and provide login information.
5. Select the Chapter16 database.
6. Click OK.
7. In the Configure OLE DB Connection Manager dialog box, click OK.
8. Right-click in the Connection Managers area of your new package and select New Flat File Connection.
9. Enter DepartmentList as the Connection Manager Name.
10. Enter C:\Departments.txt as the File Name (this file will be supplied by your instructor).
11. Check the Column Names in the First Data Row checkbox. Figure 16-4 shows the completed General page of the dialog box.
    
    

Figure 16-4: Defining a Flat File Connection Manager


12. Click the Advanced icon to move to the Advanced page of the dialog box
13. Click the New button.
14. Change the Name of the new column to DepartmentName.
15. Click OK.
16. Right-click the DepartmentList Connection Manager and select Copy.
17. Right-click in the Connection Managers area and select Paste.
18. Click on the new DepartmentList 1 connection to select it.
19. Use the Properties Window to change properties of the new connection. Change the Name property to DepartmentListBackup. Change the ConnectionString property to C:\DepartmentsBackup.txt.

Figure 16-5 shows the SSIS package with the three Connection Managers defined.

Figure 16-5: An SSIS package with two Connection Managers

SSIS Tutorial: Building Control Flows

The Control Flow tab of the Package Designer is where you tell SSIS what the package will do. You create your control flow by dragging and dropping items from the toolbox to the surface, and then dragging and dropping connections between the objects. The objects you can drop here break up into four different groups:

• Tasks are things that SSIS can do, such as execute SQL statements or transfer objects from one SQL Server to another. Table 16-2 lists the available tasks.
• Maintenance Plan tasks are a special group of tasks that handle jobs such as checking database integrity and rebuilding indexes. Table 16-3 lists the maintenance plan tasks.
• The Data Flow Task is a general purpose task for ETL (extract, transform, and load) operations on data. There’s a separate design tab for building the details of a Data Flow Task.
• Containers are objects that can hold a group of tasks. Table 16-4 lists the available containers.

Task	Purpose
ActiveX Script	Execute an ActiveX Script
Analysis Services Execute DDL	Execute DDL query statements against an Analysis Services server
Analysis Services Processing	Process an Analysis Services cube
Bulk Insert	Insert data from a file into a database
Data Mining Query	Execute a data mining query
Execute DTS 2000 Package	Execute a Data Transformation Services Package (DTS was the SQL Server 2000 version of SSIS)
Execute Package	Execute an SSIS package
Execute Process	Shell out to a Windows application
Execute SQL	Run a SQL query
File System	Perform file system operations such as copy or delete
FTP	Perform FTP operations
Message Queue	Send or receive messages via MSMQ
Script	Execute a custom task
Send Mail	Send e-mail
Transfer Database	Transfer an entire database between two SQL Servers
Transfer Error Messages	Transfer custom error messages between two SQL Servers
Transfer Jobs	Transfer jobs between two SQL Servers
Transfer Logins	Transfer logins between two SQL Servers
Transfer Master Stored Procedures	Transfer stored procedures from the master database on one SQL Server to the master database on another SQL Server
Transfer SQL Server Objects	Transfer objects between two SQL Servers
Web Service	Execute a SOAP Web method
WMI Data Reader	Read data via WMI
WMI Event Watcher	Wait for a WMI event
XML	Perform operations on XML data

Table 16-2: SSIS control flow tasks

Task	Purpose
Back Up Database	Back up an entire database to file or tape
Check Database Integrity	Perform database consistency checks
Execute SQL Server Agent Job	Run a job
Execute T-SQL Statement	Run any T-SQL script
History Cleanup	Clean out history tables for other maintenance tasks
Maintenance Cleanup	Clean up files left by other maintenance tasks
Notify Operator	Send e-mail to SQL Server operators
Rebuild Index	Rebuild a SQL Server index
Reorganize Index	Compacts and defragments an index
Shrink Database	Shrinks a database
Update Statistics	Update statistics used to calculate query plans

Table 16-3: SSIS maintenance plan tasks

Container	Purpose
For Loop	Repeat a task a fixed number of times
Foreach	Repeat a task by enumerating over a group of objects
Sequence	Group multiple tasks into a single unit for easier management

Table 16-4: SSIS containers

Try It!

To add control flows to the package you've been building, follow these steps:

1. If the Toolbox isn't visible already, hover your mouse over the Toolbox tab until it slides out from the side of the BIDS window. Use the pushpin button in the Toolbox title bar to keep the Toolbox visible.
2. Make sure the Control Flow tab is selected in the Package Designer.
3. Drag a File System Task from the Toolbox and drop it on the Package Designer.
4. Drag a Data Flow Task from the Toolbox and drop it on the Package Designer, somewhere below the File System task.
5. Click on the File System Task on the Package Designer to select it.
6. Drag the green arrow from the bottom of the File System Task and drop it on top of the Data Flow Task. This tells SSIS the order of tasks when the File System Task succeeds.
7. Double-click the connection between the two tasks to open the Precedence Constraint Editor.
8. Change the Value from Success to Completion, because you want the Data Flow Task to execute whether the File System Task succeeds or not.
9. Click OK.
10. Select the File System task in the designer. Use the Properties Window to set properties of the File System Task. Set the Source property to DepartmentList. Set the Destination property to DepartmentListBackup. Set the OverwriteDestinationFile property to True.

Figure 16-6 shows the completed set of control flows.

Figure 16-6: Adding control flows

As it stands, this package uses the file system task to copy the file specified by the DepartmentList connection to the file specified by the DepartmentListBackup connection, overwriting any target file that already exists. It then executes the data flow task. In the next section, you’ll see how to configure the data flow task.

SSIS Tutorial: Building Data Flows

The Data Flow tab of the Package Designer is where you specify the details of any Data Flow tasks that you’ve added on the Control Flow tab. Data Flows are made up of various objects that you drag and drop from the Toolbox:

• Data Flow Sources are ways that data gets into the system. Table 16-5 lists the available data flow sources.
• Data Flow Transformations let you alter and manipulate the data in various ways. Table 16-6 lists the available data flow transformations.
• Data Flow Destinations are the places that you can send the transformed data. Table 16-7 lists the available data flow destinations.

Source	Use
DataReader	Extracts data from a database using a .NET DataReader
Excel	Extracts data from an Excel workbook
Flat File	Extracts data from a flat file
OLE DB	Extracts data from a database using an OLE DB provider
Raw File	Extracts data from a raw file
XML	Extracts data from an XML file

Table 16-5: Data flow sources

Transformation	Effect
Aggregate	Aggregates and groups values in a dataset
Audit	Adds audit information to a dataset
Character Map	Applies string operations to character data
Conditional Split	Evaluates and splits up rows in a dataset
Copy Column	Copies a column of data
Data Conversion	Converts data to a different datatype
Data Mining Query	Runs a data mining query
Derived Column	Calculates a new column from existing data
Export Column	Exports data from a column to a file
Fuzzy Grouping	Groups rows that contain similar values
Fuzzy Lookup	Looks up values using fuzzy matching
Import Column	Imports data from a file to a column
Lookup	Looks up values in a reference dataset
Merge	Merges two sorted datasets
Merge Join	Merges data from two datasets by using a join
Multicast	Creates copies of a dataset
OLE DB Command	Executes a SQL command on each row in a dataset
Percentage Sampling	Extracts a subset of rows from a dataset
Pivot	Builds a pivot table from a dataset
Row Count	Counts the rows of a dataset
Row Sampling	Extracts a sample of rows from a dataset
Script Component	Executes a custom script
Slowly Changing Dimension	Updates a slowly changing dimension in a cube
Sort	Sorts data
Term Extraction	Extracts data from a column
Term Lookup	Looks up the frequency of a term in a column
Union All	Merges multiple datasets
Unpivot	Normalizes a pivot table

Table 16-6: Data Flow Transformations

Destination	Use
Data Mining Model Training	Sends data to an Analysis Services data mining model
DataReader	Sends data to an in-memory ADO.NET DataReader
Dimension Processing	Processes a cube dimension
Excel	Sends data to an Excel worksheet
Flat File	Sends data to a flat file
OLE DB	Sends data to an OLE DB database
Partition Processing	Processes an Analysis Services partition
Raw File	Sends data to a raw file
Recordset	Sends data to an in-memory ADO Recordset
SQL Server	Sends data to a SQL Server database
SQL Server Mobile	Sends data to a SQL Server Mobile database

Table 16-7: Data Flow Destinations

Try It!

To customize the data flow task in the package you're building, follow these steps:

1. Select the Data Flow tab in the Package Designer. The single Data Flow Task in the package will automatically be selected in the combo box.
2. Drag an OLE DB Source from the Toolbox and drop it on the Package Designer.
3. Drag a Character Map Transformation from the Toolbox and drop it on the Package Designer.
4. Drag a Flat File Destination from the Toolbox and drop it on the Package Designer.
5. Click on the OLE DB Source on the Package Designer to select it.
6. Drag the green arrow from the bottom of the OLE DB Source and drop it on top of the Character Map Transformation.
7. Click on the Character Map Transformation on the Package Designer to select it.
8. Drag the green arrow from the bottom of the Character Map Transformation and drop it on top of the Flat File Destination.
9. Double-click the OLE DB Source to open the OLE DB Source Editor.
10. Select the HumanResources.Department view. Figure 16-7 shows the completed OLE DB Source Editor.



Figure 16-7: Setting up the OLE DB Source


11. Click OK.
12. Double-click the Character Map Transformation.
13. Check the Name column.
14. Select In-Place Change in the Destination column.
15. Select the Uppercase operation. Figure 16-8 shows the completed Character Map Transformation Editor.



Figure 16-8: Setting up the Character Map Transformation

16. Click OK.
17. Double-click the Flat File Destination.
18. Select the DepartmentList Flat File Connection Manager.
19. Select the Mappings page of the dialog box.
20. Drag the Name column from the Available Input Columns list and drop it on top of the DepartmentName column in the Available Destination Columns list. Figure 16-9 shows the completed Mappings page.



Figure 16-9: Configuring the Flat File Destination

21. Click OK.

Figure 16-10 shows the completed set of data flows.

Figure 16-10: Adding data flows

The data flows in this package take a table from the Chapter16 database, transform one of the columns in that table to all uppercase characters, and then write that transformed column out to a flat file.

SSIS Tutorial: Creating Event Handlers

SSIS packages also support a complete event system. You can attach event handlers to a variety of events for the package itself or for the individual tasks within a package. Events within a package “bubble up.” That is, suppose an error occurs within a task inside of a package. If you’ve defined an OnError event handler for the task, then that event handler is called. Otherwise, an OnError event handler for the package itself is called. If no event handler is defined for the package either, the event is ignored.

Event handlers are defined on the Event Handlers tab of the Package Designer. When you create an event handler, you handle the event by building an entire secondary SSIS package, and you have access to the full complement of data flows, control flows, and event handlers to deal with the original event.

By adding event handlers to the OnError event that call the Send Mail task, you can notify operators by e-mail if anything goes wrong in the course of running an SSIS package.

Try It!

To add an event handler to the package we’ve been building, follow these steps:

1. Open SQL Server Management Studio and connect to your test server.
2. Create a new query and select the Chapter16 database in the available databases list on the toolbar.
3. Enter this text into a query window:
   
   CREATE TABLE DepartmentExports(
   ExportID int IDENTITY(1,1) NOT NULL,
   ExportTime datetime NOT NULL
   CONSTRAINT DF_DepartmentExports_ExportTime DEFAULT (GETDATE()),
   CONSTRAINT PK_DepartmentExports PRIMARY KEY CLUSTERED
   (
   ExportID ASC
   )
   
   
4. Click the Execute toolbar button to create the table.
5. Switch back to the Package Designer in BIDS.
6. Select the Event Handlers tab.
7. Expand the Package and then the Executables node.
8. Select the Data Flow Task in the Executable dropdown list.
9. Select the OnPostExecute event handler.
10. Click the hyperlink on the design surface to create the event handler.
11. Drag an Execute SQL task from the Toolbox and drop it on the Package Designer.
12. Double-click the Execute SQL task to open the Execute SQL Task Editor.
13. Select the OLE DB connection manager as the task's connection.
14. Set the SQL Statement property to INSERT INTO DepartmentExports (ExportTime) VALUES (GETDATE()).
15. Click OK to create the event handler.

This event handler will be called when the Data Flow Task finishes executing, and will insert one new row into the tracking table when it is called.

SSIS Tutorial: Saving and Running Packages

Now that you've created an entire SSIS package, you’re probably ready to run it and see what it does. But first, let’s look at the options for saving SSIS packages. When you work in BIDS, your SSIS package is saved as an XML file (with the extension dtsx) directly in the normal Windows file system. But that’s not the only option. Packages can also be saved in the msdb database in SQL Server itself, or in a special area of the file system called the Package Store.

Storing SSIS packages in the Package Store or the msdb database makes it easier to access and manage them from SQL Server’s administrative and command-line tools without needing to have any knowledge of the physical layout of the server’s hard drive.

Saving Packages to Alternate Locations

To save a package to the msdb database or the Package Store, you use the File > Save Package As menu item within BIDS.

Try It!

To store copies of the package you've developed, follow these steps.

1. Select File > Save Copy of Package.dtsx As from the BIDS menus.
2. Select SSIS Package Store as the Package Location.
3. Select the name of your test server.
4. Enter /File System/ExportDepartments as the package path.
5. Click OK.
6. Select File > Save Copy of Package.dtsx As from the BIDS menus.
7. Select SQL Server as the Package Location.
8. Select the name of your test server and fill in your authentication information.
9. Enter ExportDepartments as the package path.
10. Click OK.

Running a Package

You can run the final package from either BIDS or SQL Server Management Studio. When you’re developing a package, it’s convenient to run it directly from BIDS. When the package has been deployed to a production server (and saved to the msdb database or the Package Store) you’ll probably want to run it from SQL Server Management Studio.

SQL Server also includes a command-line utility, dtsexec, that lets you run packages from batch files.

Running a Package from BIDS

With the package open in BIDS, you can run it using the standard Visual Studio tools for running a project. Choose any of these options:

• Right-click the package in Solution Explorer and select Execute Package.
• Click the Start Debugging toolbar button.
• Press F5.

Try It!

To run the package that you have loaded in BIDS, follow these steps:

1. Click the Start Debugging toolbar button. SSIS will execute the package, highlighting the steps in the package as they are completed. You can select any tab to watch what’s going on. For example, if you select the Control Flow tab, you’ll see tasks highlighted, as shown in Figure 16-11.



Figure 16-11: Executing a package in the debugger


2. When the package finishes executing, click the hyperlink underneath the Connection Managers pane to stop the debugger.
3. Click the Execution Results tab to see detailed information on the package, as shown in Figure 16-12.



Figure 16-12: Information on package execution

All of the events you see in the Execution Results pane are things that you can create event handlers to react to within the package. As you can see, DTS issues a quite a number of events, from progress events to warnings about extra columns of data that we retrieved but never used.

Running a Package from SQL Server Management Studio

To run a package from SQL Server Management Studio, you need to connect Object Browser to SSIS.

Try It!

1. In SQL Server Management Studio, click the Connect button at the top of the Object Explorer window.
2. Select Integration Services.
3. Choose the server with Integration Services installed and click Connect. This will add an Integration Services node at the bottom of Object Explorer.
4. Expand the Stored Packages node. You’ll see that you can drill down into the File System node to find packages in the Package Store, or the MSDB node to find packages stored in the msdb database.
5. Expand the File System node.
6. Right-click on the ExportDepartments package and select Run Package. This will open the Execute Package utility, shown in Figure 16-13.



Figure 16-13: Executing a package from SQL Server Management Studio


7. Click Execute.
8. Click Close twice to dismiss the progress dialog box and the Execute Package Utility.
9. Enter this text into a query window with the Chapter16 database selected:

   SELECT * FROM DepartmentExports

10. Click the Execute toolbar button to verify that the package was run. You should see one entry for when the package was run from BIDS and one from when you ran it from SQL Server Management Studio.

SSIS Tutorial: Exercises

One common use of SSIS is in data warehousing - collecting data from a variety of different sources into a single database that can be used for unified reporting. In this exercise you’ll use SSIS to perform a simple data warehousing task.

Use SSIS to create a text file, c:\EmployeeList.txt, containing the last names and network logins of the AdventureWorks employees. Retrieve the last names from the Person.Contact table in the AdventureWorks database. Retrieve the logins from the HumanResources.Employee table in the Chapter16 database.

You can use the Merge Join data flow transformation to join data from two sources. One tip: the inputs to this transformation need to be sorted on the joining column.

Solutions to Exercises

1. Launch Business Intelligence Development Studio
2. Select File > New > Project.
3. Select the Business Intelligence Projects project type.
4. Select the Integration Services Project template.
5. Select a convenient location.
6. Name the new project ISProject2 and click OK.
7. Right-click in the Connection Managers area of your new package and select New OLE DB Connection.
8. Click New to create a new data connection.
9. In the Connection Manager dialog box, select the SQL Native Client provider.
10. Select your test server and provide login information.
11. Select the AdventureWorks database.
12. Click OK.
13. Right-click in the Connection Managers area of your new package and select New OLE DB Connection.
14. Select the existing connection to the Chapter16 database and click OK.
15. Right-click in the Connection Managers area of your new package and select New Flat File Connection.
16. Enter EmployeeList as the Connection Manager Name.
17. Enter C:\Employees.txt as the File Name.
18. Check the Column Names in the First Data Row checkbox.
19. Click the Advanced icon to move to the Advanced page of the dialog box.
20. Click the New button.
21. Change the Name of the new column to LastName.
22. Click the New button.
23. Change the Name of the new column to Login.
24. Click OK.
25. Select the Control Flow tab in the Package Designer.
26. Drag a Data Flow Task from the Toolbox and drop it on the Package Designer.
27. Select the Data Flow tab in the Package Designer. The single Data Flow Task in the package will automatically be selected in the combo box.
28. Drag an OLE DB Source from the Toolbox and drop it on the Package Designer.
29. Drag a second OLE DB Source from the Toolbox and drop it on the Package Designer.
30. Drag a Sort Transformation from the Toolbox and drop it on the Package Designer.
31. Drag a second Sort Transformation from the Toolbox and drop it on the Package Designer.
32. Drag a Merge Join Transformation from the Toolbox and drop it on the Package Designer.
33. Drag a Flat File Destination from the Toolbox and drop it on the Package Designer.
34. Click on the first OLE DB Source on the Package Designer to select it.
35. Drag the green arrow from the bottom of the first OLE DB Source and drop it on top of the first Sort Transformation.
36. Click on the second OLE DB Source on the Package Designer to select it.
37. Drag the green arrow from the bottom of the second OLE DB Source and drop it on top of the second Sort Transformation.
38. Click on the first Sort Transformation on the Package Designer to select it.
39. Drag the green arrow from the bottom of the first Sort Transformation and drop it on top of the Merge Join Transformation.
40. In the Input Output Selection dialog box, select the Merge Join Left Input.
41. Click OK.
42. Click on the second Sort Transformation on the Package Designer to select it.
43. Drag the green arrow from the bottom of the second Sort Transformation and drop it on top of the Merge Join Transformation.
44. Click on the Merge Join Transformation on the Package Designer to select it.
45. Drag the green arrow from the bottom of the Merge Join Transformation and drop it on top of the Flat File Destination. Figure 16-14 shows the Data Flow tab with the connections between tasks.
    
    

Figure 16-14: Data flows to merge two sources


46. Double-click the first OLE DB Source to open the OLE DB Source Editor.
47. Select the connection to the AdventureWorks database.
48. Select the Person.Contact view.
49. Click OK.
50. Double-click the second OLE DB Source to open the OLE DB Source Editor.
51. Select the connection to the Chapter16 database.
52. Select the HumanResources.Employee view.
53. Click OK.
54. Double-click the first Sort Transformation.
55. Check the ContactID column.
56. Click OK
57. Double-click the second Sort Transformation.
58. Check the ContactID column.
59. Click OK
60. Double-click the Merge Join Transformation.
61. Check the Join Key checkbox for the ContactID column in both tables.
62. Check the selection checkbox for the LastName column in the left-hand table and the ContactID and LoginID columns in the right-hand table. Figure 16-15 shows the completed Merge Join Transformation Editor.
    
    

Figure 16-15: Setting up a Merge Join


63. Click OK.
64. Double-click the Flat File Destination.
65. Select the EmployeeList Flat File Connection Manager.
66. Select the Mappings page of the dialog box.
67. The LastName columns will be automatically mapped. Drag the LoginID column from the Available Input Columns list and drop it on top of the Login column in the Available Destination Columns list.
68. Click OK.
69. Right-click the package in Solution Explorer and select Execute Package.
70. Stop debugging when the package is finished executing.
71. Open the c:\Employees.txt file to inspect the results.

----------------------------

Passing Variables to and from an ‘Execute SQL Task and Script Task’.

The two tasks do fundamentally the same thing, which is to try and date-stamp a file. The final variable value “FullPath” could then be easily used by a File System Task to copy/move or delete a file perhaps.

I suppose most SQL Server developers would be more comfortable knocking up this fairly trivial code in SQL, but the difficulty is in passing and catching the input variables in the task. This example demonstrates the problem.

I have set up a package with three String variables called

• FileName, which has a data type of String and an initial value of “Import.txt”
• FolderName, which has a data type of String and an initial value of “c:\”
• FullPath, which has a data type of String and no initial value

… and an ‘Execute SQL Task and a Script’ Task.

The package is called, rather imaginatively, “Package3”. The scope of the variables is at the package level. One thing to note when you set up variables (choose SSIS-VARIABLES from the top menu) is to make sure you have clicked on the package and not a task when you create or add a variable. If you create a variable while being clicked on a task (therefore with task scope) then the variable will disappear from the list when you click up to the package. Sorry to be an old dog but I initially found this a bit confusing.

The simplest way to inspect your variables is to set a break-point on the task (right click on the task and choose EDIT BREAKPOINTS) for the OnPostExecute event of the task. This will then allow you to inspect the value of the variable after the task has completed its execution. The red dots on the tasks indicate that there are already breakpoints set up on the task.

Doing it the 'Execute SQL Task' way

In the ‘Execute SQL Task Editor’ in the ‘Parameter Mapping’ section, (double-click on the task and choose Parameter mapping), I have set it up so that the two variables i.e. User::FolderName and User::FileName are added to the dialogue box. Each has a Direction of “Input” which seems logical and a data type of VARCHAR. The parameter names that I have used are just the ordinal positions of 0 and 1, which is what the context help suggests. In other words, the value of User::FolderName is the first input parameter and User::FileName is the second input parameter. The parameter lengths are 50. In other words, we are dealing with a varchar(50) parameter. The initial default values for these, when you set them up, are -1 which tells me nothing I am afraid.

For the Result Set options on the Execute SQL Task, I have put in the aliased name of the output column from my query, which, giving my poetic instincts full rein, I have called FullPathFromQuery, and I want to pass this to my variable User::FullPath.

‘So what about the query?’ you may ask. Well, if we go back to the General option on the Execute SQL Task Editor, you can see the code and I will list it again here

DECLARE @YesterdaysDate varchar(8)

DECLARE @Filename varchar(50)

DECLARE @Folder varchar(50)

DECLARE @Etc varchar(50)

SET @Folder = ?

SET @Filename = ?

--SET @Etc = ?

SELECT @YesterdaysDate = CONVERT(varchar(8),DATEADD(dd,-1,getdate()),112)

SELECT @Folder + SUBSTRING(@Filename,1,CHARINDEX('.txt',@Filename)-1) + '_' + @YesterdaysDate + '.txt' AS FullPathFromQuery

For such trivial code you would not want to set up a stored procedure I suspect, so the option of passing parameters to a stored procedure is not really there.

The only way to pick up these input variable values is to use question marks “?” in the order that they are passed. This query as it stands will clearly not parse in query analyser window so you can only really test your code by running it in the task: This is not ideal.

You must also set the ResultSet option to be “Single row”.

If you run this to the break point one can see that the variable User::FullPath has been assigned the value C:\\Import_200870805.txt ( I ran this on 6^th Aug 2008) so the code must be working.

Using a Script Task instead

Perhaps a simpler way to do this is just to use the Script Task. The trivial code again demonstrates how to pick up and assign values to and from variables in the vb.net code. You need to tell the task which variables it is going to use by adding them to the ReadOnlyVariables and ReadWriteVariables options in the Script Task editor window and also be aware that the variables names are case-sensitive.

The code which shows the passing and assignment of the variables in the script is shown below.

Imports System

Imports System.Data

Imports System.Math

Imports Microsoft.SqlServer.Dts.Runtime

Public Class ScriptMain

Public Sub Main()

Dim strFolder As String

Dim strFilename As String

Dim strTomorrow As String

Dim strNewFullPath As String

'do path in script

strFolder = Dts.Variables("FolderName").Value.ToString

strFilename = Dts.Variables("FileName").Value.ToString()

strTomorrow = CStr(Format(Today().AddDays(+1), "yyyyMMdd"))

strNewFullPath = strFolder & strFilename.Substring(0, strFilename.IndexOf(".txt")) & "_" & strTomorrow & ".txt"

'display new value of path

'MsgBox(strNewFullPath)

Dts.Variables("FullPath").Value = strNewFullPath

Dts.TaskResult = Dts.Results.Success

End Sub

End Class

If you put a breakpoint on the task the value of the variable can be inspected to give tomorrows date, and it should look like this…

SSRS - SQL Server Reporting Services.

SSRS : The Reporting Services Architecture

PS: The images are resized, click on the image to get the original size of the image.

For a Relatively new Product, Reporting Services has a quite complex architecture. The full Reporting Services architecture includes development tools, administration tools and report viewers. There are a number of ways to get to Reporting Services Programmatically, including SOAP and WMI interfaces.

Figure 1 Shows a simplified diagram of the main Reporting Services components that we will be using.

Figure 1: Report Server Architecture

In this chapter you'll learn about these components:

• Report Server is the core engine that drives Reporting Services.
• Report Manager is a Web-based administrative interface for Reporting Services.
• Report Designer is a developer tool for building complex reports.
• Report Builder is a simplified end-user tool for building reports.
• The Report Server database stores report definitions. Reports themselves can make use of data from many different data sources.

SSRS: Using Report Designer

Reporting Services includes two tools for creating reports:

• Report Designer can create reports of any complexity that Reporting Services supports, but requires you to understand the structure of your data and to be able to navigate the Visual Studio user interface.
• Report Builder provides a simpler user interface for creating ad hoc reports, directed primarily at business users rather than developers. Report Builder requires a developer or administrator to set up a data model before end users can create reports.

We will start our Reporting services with Report Designer. Report Designer runs inside the Business Intelligence Development Studio shell and offers several ways to create reports. You can either use the Report Wizard to quickly create a Report or you can use a set of design tools to build a report from scratch. You can also use the design tools to modify a report created with the wizard.

Using the Report Wizard
The easiest way to create a report in Report Designer is to use the Report Wizard. Like all wizards, the Report Wizard walk you through the process in step-by-step fashion. You can make the following choices in the wizard.

The easiest way to create a report in Report Designer is to use the Report Wizard. Like all wizards in step-by-step fashion. You can make the following choices in the wizard:

• The Data source to use.
• The query to use to retrieve data.
• Whether to use a tabular or matrix layout for the report.
• How to group the retrieved data.
• What visual style to use.
• Where to deploy the finished report.

To create a simple report using the Report Wizard, follow these steps as mentioned below:

1. Launch Business Intelligence Development Studio->Start Menu->Program Files->SQL Server 2005->Business Intelligence Development Studio.
2. Select File->New->Project.
3. Select Business Intelligence Projects, Project Type.
4. Select the Report Server Project Wizard template.
5. Name the new Report as ProductReport1 and select a desired location to save the Project.
6. Now Click OK.
7. Read the first page of the Report Wizard and click Next.
8. Name the new Data source AdventureWorksDS.
9. Click the Edit Button.
10. Log on to your Test Server.
11. Select the AdventureWorks Database and then Click OK.
12. Click the Credentials button then select Use Windows Authentication and Click OK.
13. Check the Make this is a shared Data Source Check Box. This will make this particular data source available to other Reporting Services applications in the future and Click on Next button.
14. Click on the Query Builder Button, if the full query designer interface does not display by default then click the query designer toolbar button at the far left end of the toolbar.
    

Figure 2 shows the full query designer interface.

Figure 2:Query Builder.

15. Click the Add table toolbar button and then Select the Product table and Click Add.
16. Click on the Close button
17. Check the Name, ProductNumber,Color, and ListPrice Columns and then Click OK.
18. Click on Next button and then Select the Tabular Layout and Click Next.
19. Now Move the Color column to the Group area and the other three columns to the
Detail area as show in Figure 3

Figure 3: Grouping columns in the report.

2o. Click on Next button and Select the Stepped layout and Click on Next again.
21. Select the Ocean style and Click on Next button.
22. Accept the default deployment location and then Click on Next button.
23. Now Name the report ProductReport1.
24. Check the option Preview Report Check box and the click on Finish Button.

Figure 18-4 Figure 4 shows the finished report, Open in Report Designer

Figure 4: Shows Report created by Report Wizard and shows the main features of Report Designer.

The Datasets window shows the Data that is available to the report.

• The main design window lets you view the report itself. You can see a preview of the report, work with the report in a layout designer, or work with the query that returns the data for the report.
• The Solution Explorer, Output and Properties windows are Standard Visual Studio windows.
  

Modifying a Report

Now that you have created a report with Report Wizard, you can modify it with Report Designer and If you have used sort of Visual report design tool in the past then you should have no issues in making changes here and the possibilities could be:

• You can change the available Data or the sort order for the report by modifying the query on the Data tab.
• You can resize or rearrange controls on the layout tab.
• You can use the Properties window to change properties of Individual controls including their font, alignment, colors and so on.
  

In Order to Modify the report you just created, please follow these steps.

1. Click on the Data tab to view the query for the report.
2. Select a Descending sort type for the ListPrice Column and then Click the layout tab.
3. Now Click in the text box at the top of the report, where the report name is displayed.
4. Use the Properties window to change the Value Property of this control to Products by Color and then Click on the header for the Product Column.
5. Place the cursor between the Name and Product Number columns to display a double-header arrow. Hold down the mouse button and drag the cursor to the right to widen the name column.
6. Place the cursor between the Product Number and ListPrice columns to display a double headed arrow. Hold down the mouse button and drag the cursor to the right to widen the Product Number column.
7. Click on the Preview tab to view the modified report, as shown in Figure 5

Figure 5: Modified Product Report

Designing a Report From Scratch

You can also use Report Designer to build your own reports starting from scratch. In general, you will follow these steps to create a report:

1. Create a Report Project in Business Intelligence Design Studio.
2. Associate one or more Datasets with the report.
3. Create a query from Dataset.
4. Build the report layout.

To Create a fresh report in Report Designer, follow these steps:

1. Select->Close Solution to close the existing report.
2. Select->New->Project.
3. Select the Business Intelligence Projects Project Project type.
4. Select the Report Server Project template.
5. Name the new Report ProductReport2 and Pick a convenient location to save it in.
6. Right-Click on report ProductReport2 and pick a convenient location to save it in.
7. Select the Report template and Name the new report ProductReport2.rdl and Click Add.
8. Select the Data tab in the Report Designer.
9. Click the Datasets dropdown and select to open the Data source dialog box.
10. Name the new Data source AdventureWorksDS and then Click on Edit button.
11. Now, Connect to your test server and choose the AdventureWorks Database and Click on OK.
    
12. Click on OK again to create the data source.
13. Click the Query Designer toolbar button inside the Data tab to display the full query designer user interface.
14. Click the Add Table button.
15. Select the location table and Click on Add and then Click on Close.
16. Check the boxes for the Name and Cost Rate Columns.
17. Sort the Dataset in ascending order by Name.
18. Click the layout tab to show the blank report layout.
19. Hover your mouse over the toolbox tab to display the available controls for the report.
20. Double Click the List control.
21. Expand the List Control to the width of the report.
22. Expand the Dataset in the Datasets window to show the column names.
23. Drag the Name and Cost Rate Columns from the Datasets window and drop them in the List Control. Expand them horizontally untill each takes up about half of the List control. Figure 6 shows the report in layout view.
24. Select the Preview tab to see the report with Data.
    

Figure 6: Designing a report from scratch

SSRS :Publishing a Report

Creating reports in Business Intelligence Development Studio is good for developers, but it does not help users at all. In order for the reports you build to be available to others, you must publish them to your Reporting Services Server. To publish a report, you can use the Build and Deploy menu items in Buisness Intelligence Development Studio. Well, before you do this you need to check the Projects configuration to make sure that you have selected and appropriate server for the deployment.

To Publish a report , please follow these steps.

1. Close the ProductReport2 Project.
2. Open the ProductReport1 Project, Select Project->ProductReport1 Properties.
3. Click on the Configuration Manager Button.
4. Set the Active Solution Configuration to Production, Make sure the Configuration is set to Production and both the Build and deploy Checkboxes are checked.
5. Figure 7 Shows the Configuration Manager.

Figure 7: Setting the active Configuration.

6. Click on Close.
7. Fill in the URL for your Report Server. If you are developing on the same computer where Reporting Services is installed and you installed in the default configuration, this will be http://localhost/ReportServer. Figure 18-8 8 shows the completed Property Pages.

Figure 8 Setting Report Project Properties

8. Click OK.
9. Select Build->Deply ProductReport1. The Output Window will trak the progress
of BIDS in deploying your report , as shown in Figure 18-9 9. Depending on the
speed of your computer, building the report may take some time.

Figure 9 Deploying a Report.

10. Launch SQL Server Management Studio.
11. Click the Connect Button in Object Explorer and Select Reporting Services.
12. Now, Connect to your test server.
13. Expand the Home node in the Object Explorer tree to find the ProductReport1 Project.
14. Expand the ProductReport1 Project to find the ProductReport1 Report.
15. Right-Click on the report and select View Report.

16. If Prompted, supply your username and password. The report will open in your Web
browser.

SSRS: Using Report Build

Report Designer gives you one way to create reports for Reporting Services, but it is not the only way to do so, SQL Server 2005 also includes a tool directed at end users names Report Builder. Unlike Report Designer, which is aimed at Developers, Report Builder presents a simplified view of the report-buidling process and is intended for buisness analysts and othe end users.

Buildinf a Data Model

Report Builder does not let end users explore all of a SQL Server Database. Instead, it depends on a data model: a preselected group of tables and relationships that a developer has identified as suitable for end-user reporting. To build a data model, you use Business Intelligence Development studio. Data Models Contain three things:

• Data Sources Connect the Data model to actual data.
• Data Source Views Draw Data from Data sources.
• Report Models Contain entities that end users can use on reports.

To Create a Data Model, Follow these steps:

1. If it is not already open, launch Business Intelligence Development Studio.
2. Select File->New->Project.
3. Select the Business Intelligence Projects Project type.
4. Select the Report Model Project Template.
5. Name the new Project AWSales and Save it in a convenient location and Click OK.
6. Right-Click on Data sources in Solution Explorer and Select Add New Data Source.
7. Read the first page of the Add New Data Source Wizard and Click Next and then Click on New.
8. In the Connection Manager Dialog box Connect to the AdventureWorks Database on your test server and Click OK and the Click on Next.
9. Name the new Data source AdventureWorks and Click on Finish.
10. Right Click on Data source Views in Solution Explorer and Select Add New Data Source View.
11. Read the first page of the Add New Data source View Wizard and Click Next.
12. Select the AdventureWorks Data Source and click Next.
13. Select the Production.Product Table and Click the >Button to move it to the included Objects listbox.
14. Select the Sales.SalesOrderDetail table and Click the > button to move it to the included Object listbox.
15. Select the Sales.SalesOrderDetail table and Click the > button to move it to the Included Objects Listbox.
16. Select the Sales.SalesOrderDetail table and Click the > button to move it to the included Objects listbox.
17. Click the Add Related Tables button.
18. Click on Next and then Click on Finish.
19. Right- Click on Report Models in Solution explorer and Select Add New Report Model.
20. Read the first page of the Report Model Wizard and Click Next.
21. Select the Adventure works Data source view and Click Next.
22. Keep the default rules selection, as show in Figure 10 and Click on Next.

Figure 10: Creating Entries for end-user reporting.

23 . Choose the Update Statistics Option and Click Next.
24 . Click Run to Complete the wizard.
25 . Click Finish, if you get a warning that a file was modified outside the source
editor then Click YES.
26 . Select Build and Deploy AWSales to Deploy the report model to the local
Reporting Services Server.

Building a Report
Report Builder itself is a ClickOnce Windows Forms application. That means that end users install it from the browser, but once installed it is not a browser-based application. To get started with Report Builder, Browse to your Reporting Services Home page. Typically this will have a URL such as http://Server Name/Reports(or http://localhost/Reports If you are running the browser on the same box with SQL Server 2005 itself) Figure 11 shows the Reporting services home page

Figure 11: Reporting Services Home Page

To Install Report Builder, Click the Report Builder link in the home page menu bar. Report Builder will automatically load up all of the available report models and wait for you to choose one to build a report from.

Lets try it out

1. Open a browser window and navigate to http://localhost/Reports( or to the appropriate Report Server URL if you are not working on the report server).
2. Click the Report Builder Link.
3. Depending on your operating System, you may have to confirm that you want to run the application.
4. After Report Builder is installed, Select the AdventureWorks report model and the table report layout. Click OK. Figure 12 shows the new blank report that Report Builder will create

Figure 12: New Report in Report Builder

PS: The Explorer window to the left of the design surface shows all of the tables in the report model. Beneath that, the fields window shows the entities in the currently-selected table. Note that not everything in this window is a column in the table: The report model also contains aggregate entities such as Total safety Stock Level and automatically calculated fields.

5. Select the Product Table.
6. Drag the Name field and drop it in the area labeled Drag and Drop Column fields.
7. Click on Special Offer Products in the Explorer window to show related child tables.
8. Click on Sales Order Details.
9. Drag the Total Order QTY field and drop it to the right of the Name field.
10. Click Where it says Click to Add Title and type Product Sales.
11. Click the RUN Report button to Produce the report shown in Figure 13

Figure 13: Report in Report Builder

12 . Click the sort and Group Toolbar button.
13 . Select to sort by Total Order QTY Descending and then Click OK.
14 . Select File-> Save and then Name the New Report Product Sales.
15 . Click Save. This will publish the report back to Reporting Services server that you
Originally downloaded Report Builder form.

SSRS: Using Report Manager
The Web Home page for Reporting Services Provides a complete interface for managing reports (as well as other objects such as Data sources and models) after they are created. This interface, Knows as Report Manager is intended primarily for database administrators but as a developer you should know about its capabilities for managing and modifying reports.

When you click on a Report in Report Manager, you will see the reports Data as shown in Figure 14.

Figure 14: Report in Report Manager

Note that reports in Report Manager open in a tabbed Interface. The four tabs allow you to perform various functions.

• View allows you to see the current Data in the report.
• Properties lets you adjust such things as the reports Name, Data Source, Security Credentials, Caching and End user Security.
• History shows you saved snapshots of the report.
• Subscriptions lets you create subscriptions to the report. Subscriptions allow you to set up periodic delivery of reports to end users by e-mail or file share.

Exercises
Use Report Builder to create a report from the AdventureWorks Data Model showing the minimum and maximum order quantity for orders taken by each sales person in the company. You will find the necessary data in the SalesOrderHeader and SalesOrderDetail tables

Solutions to Exercises

1. Open Browser window and navigate to http://Localhost/Reports( or to the appropriate Report Server URL if you are not working on the report server).
2. Click the Report Builder link.
3. Select the AdventureWorks report model and the table report layout and then Click OK.
4. Select the SalesOrder header table.
5. Drag the SalesPersonID field and drop it in the area labeled Drag and drop Column Fields.
6. Click on Sales Order Details in the Explorer window.
7. Expand the Total Order Qty Field in the Fields window to show the alternative fields beneath it.
8. Drag the Min Order QTY field and drop it to the right of the Name field.
9. Drag the Max Order QTY field and Drop it to the right of the Min Order QTY Field.
10. Click Where it says Click to Add Title and type Sales Performance.
11. Click the Run Report Button to Produce the Report shown in Figure 15

1.