Featherman’s Turning your Query into a Stored Procedure ©

You have progressed from the being the analyst that has queries and views provided to them by a DBA for copying, to an analyst that understands data models, table relationships, and the rudiments of T-SQL. Now it is your turn to learn the skills to get promoted to senior analyst. One of the rungs you must climb is becoming the query designer that provides queries packed with analytics and Key performance Indicators (KPI’s). You will in this module create powerful analytical queries and turn them into stored procedures that managers and junior analysts can utilize. You can provide your queries to others, in the form of views and stored procedures.

A view is a saved query that can be called and re-used saving time and reducing errors. Views are provided to analysts so that the analyst can easily get access to the data they need. DBA’s like views in that they restrict the analyst to a pre-defined set of rows and columns, thereby shielding the database tables from the prying eyes of the analyst. Views then provide security.

A stored procedure is also a saved query, or set of transactions that are processed in a group. SP’s allow the passing in of parameter values and the output (return) of calculation results. SP’s allow the user to group a set of SQL commands. A procedure can accept parameters and execute its SQL statements depending on those parameters. Here we use a SP to dynamically filter the resultset returned from a SELECT query. In a sense we are saving the query as a SP to more easily access its functionality from the data visualization layer. More info on SP’s is in the appendix.

Both views and stored procedures are not distributed like some code that is emailed. Views and stored procedures are reviewed by DBA’s for accuracy and then hosted on the data warehouse server. Analysts are given user ID’s and password rights to them. You too can have your T-SQL work of art vetted, approved and stored on the corporate server.

There is a process of building stored procedures and using them as an IT assets:

1. First you build a base query that pulls the data columns you need

2. Next you add the columns of calculated fields – the Key Performance Indicators –

3. Test the query using different filters and then parameterize the query as needed

4. Next you turn the query into a stored procedure

5. Test using the stored procedure, calling it from the intended data visualization layer (perhaps Excel, SSRS, Tableau)

6. Next it’s time to figure out how this IT asset is going to be used. Will the stored procedure be called from the data visualization layer? Will it be used to extract transform and load data from one database to another? Often an ETL process will pull data from 5 tables, whittle it down to one table with 10 or 15 columns packed with analytics. Where does this new table belong? Do you copy this new ETL’d data into a table for storage (appending, or overwriting the prior data) or do you just use the data in-memory as you pass the results into a report or spreadsheet? Procedures for usage of the stored procedure must be considered and delineated.

7. Now document the stored procedure, how to call it, who can use it and see the results, when to call it, what it means, a description of the columns and their meanings, a description of the data columns and what tables and databases they came from, and meaning of the KPI’s, etc.

8. Present the stored procedure to its ultimate user, showing the user how to use it, providing them documentation, training and empathetic support.

Here is an example of a stored procedure (note you can only create a stored procedure in the database where you

USE [database name]; GO

CREATE PROCEDURE spStoredProcedureName @parameter1 datatype, @parameter2 datatypeAS

BEGINSELECT Query Goes here…END

USE [Featherman_Analytics]; GO

CREATE PROCEDURE spSalesAnalytics @CustomerID int, @Year intAS

BEGINSELECT Query Goes here…END

Follow these steps to convert your query to a stored procedure (SP): Note: the steps below walk you through the creation and testing of a stored procedure. If you prefer you can just use the format and syntax similar to that shown in the table above. You can use the steps below just the first few times you create a stored procedure - until the process seems routine.

1. Save your query. Then save a copy of it. Put away the original for safe keeping.

2. Begin to edit the query. Add a new row at the top. Type DECLARE in a separate line and declare the name of your variable giving it a datatype. Now you are database programming! This variable will be used to receive values from the program user. The values will be used to filter the analytics report that is generated. The stored procedure will be parameterized meaning variant – rather than hard coded. The arguments (the parameters, aka criteria or filter values) can be set a optional but here are required inputs for your SP.

DECLARE @intProductCategory int

Here you just declared one new variable, we could have declared more. We are creating a stored procedure that can receive the filter criteria value (here product category 1,2,3) from the program user. The WHERE clause of the SQL query will receive the criteria supplied by the program user and passed into the variable. In your code change the WHERE clause to:

WHERE ProductCategory = @intProductCategory

3. After the last DECLARE line, place the word BEGIN on a new line. Place the word END on the last line of the query.

4. Add a new line with SET statements before the word BEGIN. Give values to your variables and use them in your query (here in the WHERE statement) For example:

SET @intProductCategory = 1

Run the query and refine as necessary using the sample parameter values. This is the end of the testing. In the next few steps where you turn your query into a parameterized stored procedure you cannot run the query, it won’t provide results.

5. –Comment out the SET lines, they were just used to test out the query.

6. Now insert the word GO after your USE statement similar to as follows.

USE [Featherman_Analytics]; GO

Next on the top of the script add a new line and write CREATE PROCEDURE [procedure_name]. Do not use sp_procedurename as this is how MSFT writes their procedures and you want yours to stand out from them. Your stored procedures are stored inside your database, so your first stored procedure might be called [AdventureWorks2012DW].[dbo].[SeniorAnalystStep1_sp]

7. Remove the word DECLARE from each line that creates your variables, separate the variables by commas.

Put the word AS after the variables and before the BEGIN. Execute the query. You should see the response Command successfully completed. Refresh your database and see your SP in your database’s programmability folder. Done.

8. The whole thing might look like the following (if you have >1 variable then separate them by comas).

USE [AdventureWorks2012DW];GOCREATE PROCEDURE SeniorAnalystStep1_sp@intProductCategory AS int

AS

--SET @intProductCategory = 1

BEGIN

SQL CODE goes here

END

USE [Featherman_Analytics]; GO

CREATE PROCEDURE spSalesAnalytics @CustomerID int, @Year intAS

BEGINSELECT Query Goes here…END

Example – Turning a query into a stored procedure – this is just for reference – skip down to make your own stored procedure in your database.

USE [Featherman_Analytics];

DECLARE @CustomerID int = 3

SELECT [CustomerID], COUNT([Sale_ID]) AS [Total #TA], SUM([Total_Sale]) as [Total Revenue]

, FORMAT(AVG([Total_Sale]), 'N0') as [Customers Average Invoice Amount ]

FROM [featherman].[sales]

WHERE [CustomerID] = @CustomerID

GROUP BY [CustomerID]

This query is useful in that you can learn how to do a group by query, and you can see that using variables is helpful. You can change the value in the declare line.

Now the query needs to be made more useful. If you turn it into a stored procedure that accepts different values for the customerID, then the query becomes useful for analytics.

USE [Featherman_Analytics];

GO

CREATE PROCEDURE spSalesAnalytics1Customer

@CustomerID int

AS

BEGIN

SELECT [CustomerID], COUNT([Sale_ID]) AS [Total #TA]

, SUM([Total_Sale]) as [Total Revenue]

, FORMAT(AVG([Total_Sale]), 'N0') as [Customers Average Invoice Amount ]

FROM [featherman].[sales]

WHERE [CustomerID] = @CustomerID

GROUP BY [CustomerID]

END

The author of this document has read rights to the cited database so they can add the stored procedure and receive the well-loved following message. You can make a stored procedure in your own database.

Command(s) completed successfully.

USE [Featherman_Analytics];

Execute spSalesAnalytics1Customer 7

Examples: We start with a query that uses CASE functioning to categorize rows based on business rules. First you need to copy the major

USE [AdventureWorksDW2012];

SELECT [ProductKey], [EnglishProductCategoryName] as [Category]

,[EnglishProductSubcategoryName] as [Sub Category]

,[ProductAlternateKey] as [Part #],[ModelName] as [Model]

, [EnglishProductName] as [Product], [Color], [StandardCost] as [Cost]

, [DealerPrice] as [Dealer Price], [ListPrice] as [Web Price]

--INTO [YOUR dbo].[AW_Products_Flattened]

FROM [dbo].[DimProduct] as p

INNER JOIN [dbo].[DimProductSubcategory] as sc

ON sc.[ProductSubcategoryKey]= p.[ProductSubcategoryKey]

INNER JOIN [dbo].[DimProductcategory] as c

ON c.[ProductcategoryKey]= sc.[ProductcategoryKey]

WHERE [FinishedGoodsFlag] = 1

ORDER BY [EnglishProductCategoryName]

, [EnglishProductSubcategoryName]

USE [Featherman_Analytics];

SELECT [ProductKey], [Part #], [Model], [Product], [Web Price]

, CASE

WHEN [Web Price] > 3300 then 'Expensive'

WHEN [Web Price] > 2000 AND [Web Price] <=3500 then 'Moderate'

WHEN [Web Price] > 0 AND [Web Price] <=2400 then 'Cheap'

ELSE 'Cheap'

END

AS [Price Category]

FROM [dbo].[AW_Products_Flattened] as p

WHERE [Sub Category] = 'Mountain Bikes'

Hey use your database version not Feathermans

This query uses hard coded numbers so is just a teaching tool.

USE YOUR DATABASE not Feathermans

USE [Featherman_Analytics];

DECLARE @AveragePrice as decimal = (SELECT AVG ([Web Price])

FROM [dbo].[AW_Products_Flattened]

WHERE [Sub Category] = 'Mountain Bikes')

SELECT [ProductKey], [Part #], [Model], [Product], [Web Price], CASE

WHEN [Web Price] > @AveragePrice *1.5 then 'Top End'

WHEN [Web Price] > @AveragePrice * 1.1 AND [Web Price] < (@AveragePrice * 1.5) then 'Expensive'

WHEN [Web Price] > @AveragePrice * .75 AND [Web Price] <= @AveragePrice * 1.1 then 'Moderate'

WHEN [Web Price] > 0 AND [Web Price] <=@AveragePrice * .75 then 'Low end'

END AS [Price Category]

FROM [dbo].[AW_Products_Flattened] as p

WHERE [Sub Category] = 'Mountain Bikes'

ORDER BY [Web Price] DESC

This query will dynamically break the products into groups using the SELECT CASE. The groups are a) a top end of products 1.5 times greater than the avg. web price for the sub-category

b) an expensive group of products 10% higher than the sub-category avg. yet lower than the top end

c) an moderate group of products lower than the expensive group yet higher than 75% of the sub-category avg. This is the middle group

d) a low end group of products that is less than 75% of the sub category avg.

OK – now we can turn the query into a saved stored procedure. This is an example of a hard-coded stored procedure – that will always show the

USE [Featherman_Analytics]

GO

ALTER PROCEDURE [featherman].[spSubCategoryPriceAnalysis]

@SubCategory AS varchar(50)

AS

BEGIN

DECLARE @AveragePrice AS DECIMAL = (SELECT AVG([Web Price])

FROM [dbo].[AW_Products_Flattened]

WHERE [Sub Category] = @SubCategory)

SELECT [ProductKey], [Part #], [Model], [Product], [Web Price]

, CASE

WHEN [Web Price] > @AveragePrice *1.5 then 'Top End'

WHEN [Web Price] > @AveragePrice * 1.1 AND [Web Price] < (@AveragePrice * 1.5) then 'Expensive'

WHEN [Web Price] > @AveragePrice * .75 AND [Web Price] <= @AveragePrice * 1.1 then 'Moderate'

WHEN [Web Price] > 0 AND [Web Price] <=@AveragePrice * .75

then 'Low end'

END

AS [Price Category]

FROM [dbo].[AW_Products_Flattened] as p

WHERE [Sub Category] = @SubCategory

ORDER BY [Web Price] DESC

ENDEND

Now that we have a stored procedure – you can run it. You have to pass a correct name of a sub-category.

EXEC [spSubCategoryPriceAnalysis] 'Wheels'

This is how you turn the query into a stored procedure that expects just the one parameter (sub category).

Average price is a local variable that is not passed into the stored procedure so it is declared after the term BEGIN. Note there is a BEGIN and an END to designate where the syntax is.

Once you create the stored procedure you can

a) call it from SSMS using

EXEC [featherman].[spSubCategoryPriceAnalysis] 'Wheels'

GO

b) from Excel’s Power Query

Resources

Portions derived from: http://stackoverflow.com/questions/19413076/turning-a-query-into-a-stored-procedure

This link shows how to deal with NULL values

https://www.mssqltips.com/sqlservertutorial/162/how-to-create-a-sql-server-stored-procedure-with-parameters/

Appendix

More on stored procedures. This is an excerpt from wiki

A stored procedure is a subroutine available to applications that access a relational database system. A stored procedure is actually stored in the database data dictionary.

Typical use for stored procedures include data validation (integrated into the database) or access control mechanisms. Furthermore, stored procedures can consolidate and centralize logic that was originally implemented in applications. Extensive or complex processing that requires execution of several SQL statements is moved into stored procedures, and all applications call the procedures. One can use nested stored procedures by executing one stored procedure from within another.

Stored procedures allow programmers to embed business logic as an API in the database, which can simplify data management and reduce the need to encode the logic elsewhere in client programs. This can result in a lesser likelihood of data corruption by faulty client programs. The database system can ensure data integrity and consistency with the help of stored procedures

Being revised – this stored procedure will use output variables

USE [Featherman_Analytics];

GO

CREATE PROCEDURE spSalesAnalytics1CustomerOutPut @CustomerID int, @TotalNumberTA int OUTPUT, @TotalRevenue decimal OUTPUT, @AvgSale decimal OUTPUT

AS

BEGIN

SELECT @TotalNumberTA = COUNT([Sale_ID])

, @TotalRevenue = SUM([Total_Sale]) , @AvgSale = FORMAT(AVG([Total_Sale]), 'N0')

FROM [featherman].[sales]

WHERE [CustomerID] = @CustomerID

GROUP BY [CustomerID]

END