http://sommarskog.se/error_handling/Part2.html
Following things are important to notice:
-- Steps: -- 1. Replace Placeholders in this Script -- 2. Create Certificate/Private Key/PFX File -- 3. Adapt your C#/VB .NET Project with the SQL CLR Assembly to use the Certificate (PFX file). -- 4. Make changes to Server Instance to enable Common Language Runtime. Server-Instanz so einstellen, dass die strikte Security deaktiviert wird für CLR. -- 5. Create CLR Assembly in Sql Server. -- 6. (optionally) Drop old CLR Functions. -- 7. Create new CLR Functions. /* This Article is based on input from: https://github.com/MicrosoftDocs/sql-docs/issues/2052 (Comment of Solomon Rutzky) https://nielsberglund.com/2017/07/01/sqlclr-and-certificates/ (Niels Berglund) https://nielsberglund.com/2017/07/23/sql-server-2017-sqlclr---whitelisting-assemblies/ (Niels Berglund) Regards to all of you - thank you so much for your publishment! */ -- 1... /* Replace each of the following Placeholders using Ctrl+H with your own values: DWH -> Database name where the Assembly should be deployed to (excluding []) Win10Sdk64bitPath -> Path to the installation of Windows 10 SDK 64bit: C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64 CertFolderPath -> Path to the folder where the Certificate, Private Key and PFX files will be saved (excluding ""). e.g.: C:\DWH\Certificates AssemblyFullPath -> Full Path to the DLL file to create the CRL Assembly from (excluding ""). e.g.: C:\DWH\SqlServerClrFunctionsLib.dll SqlClrAssemblyName -> Name of Assembly under which it should be registered in your Database. e.g.: ClrFunctions CertAuthority -> Canonical Name of the Certificate's Authority (excluding "") e.g.: CompanyName Authority CertName -> Name of the Certificate file (excluding .cer, folders and excluding "") e.g.: Assemblies.CompanyName.Public CertPvkName -> Name of the PVK file (excluding .pvk, folders and excluding "") e.g.: Assemblies.CompanyName.PrivateKey CertPfxName -> Name of the PFX file (excluding .pfx, folders and excluding "") e.g.: Assemblies.CompanyName.PrivateKey CertPvkPassword -> Your strong Private Key Password (excluding "") CertPfxPassword -> Another (less-strong) Password to protect the PFX file (excluding "") */ -- 2.... -- Use Windows 10 SDK makecert.exe and pvk2pfx.exe to create a (public) Certificate, Private Key and export a PFX file: -- https://developer.microsoft.com/de-de/windows/downloads/windows-10-sdk/ -- 1. "Win10Sdk64bitPath\makecert.exe" -r -pe -n "CN=CertAuthority" -a sha256 -sky signature -cy authority -sv "CertFolderPath\CertPvkName.pvk" -len 2048 -m 144 "CertFolderPath\CertName.cer" -- you will be prompted for a Private Key Password - enter: CertPvkName -- -- 2. "Win10Sdk64bitPath\pvk2pfx.exe" -pvk "CertFolderPath\CertPvkName.pvk" -spc "CertFolderPath\CertName.cer" -pfx "CertFolderPath\CertPfxName.pfx" -pi "CertPvkName" -po "CertPfxPassword" -- 3.... -- 1. Copy previously created PFX file to the root of your C# .Net Project so that $(ProjectDir) matches the directory of the PFX file. -- 2. Sign DLL in Visual Studio using the created PFX file in step 2 by adding the following command to ---->>>> Post Build Event: <<<<<---- -- "Win10Sdk64bitPath\signtool.exe" sign /f "$(ProjectDir)CertPfxName.pfx" /p "CertPfxPassword" $(TargetDir)$(TargetFileName) -- 3. Build Project -- -- 4. Copy your signed DLL to "AssemblyFullPath" of the SQL Server. -- 4.... USE MASTER GO EXEC sp_configure 'show advanced options', 1; RECONFIGURE; GO EXEC sp_configure 'clr_enabled', 1 RECONFIGURE GO EXEC sp_configure 'clr strict security', 0; RECONFIGURE; GO -- Re-visit current server configuration (displays a list of all server settings): EXEC sp_configure GO /* Optional: Sometimes it might be necessary to change the Owner of the Database to the 'SA' (or 'dbo' or another Server Admin). -- This might be the case if your Database has been imported/restored from another server (or by another user). USE [DWH]; --SELECT owner_sid FROM sys.databases WHERE database_id = DB_ID('DWH') SELECT name FROM sys.database_principals WHERE sid = (SELECT TOP 1 owner_sid FROM sys.databases WHERE database_id = DB_ID('DWH')); EXEC sp_changedbowner @loginame = 'sa'; */ -- 5.... -- a.) Ensure, the SQL Server Service User (see services.msc and look out for your SQL Server Instance) -- has Read & Execute Permissions to the Assembly File USE [DWH]; -- Important to set context to the Database where you want to register your DLL! GO CREATE ASSEMBLY SqlClrAssemblyName from 'AssemblyFullPath' WITH PERMISSION_SET = UNSAFE -- use UNSAFE to be aligned with the Permission granted to the Certificate Login previsouly created GO -- In case of SQL Server Error Message 10134: It might be necessary to add your Assembly to the trusted_assemblies if your Functions won't execute... /* SELECT * FROM sys.assemblies a WHERE a.name LIKE '%SqlClrAssemblyName%'; -- Grab the BINARY of the Assembly file: SELECT content FROM sys.assembly_files WHERE name LIKE '%SqlClrAssemblyName%' DECLARE @hash varbinary(64), @clrName nvarchar(4000) = (SELECT name FROM sys.assemblies a WHERE a.name LIKE '%SqlClrAssemblyName%'); SELECT @hash = HASHBYTES('SHA2_512', (SELECT content FROM sys.assembly_files WHERE name LIKE '%SqlClrAssemblyName%')); SELECT @hash as Hash, @clrName as Name; EXEC sp_add_trusted_assembly @hash = @hash, @description = @clrName; SELECT * FROM sys.trusted_assemblies */ -- 6..... USE [DWH]; GO BEGIN TRAN DROP FUNCTION IF EXISTS common.ToBit; -- tbd... DROP FUNCTION IF EXISTS ... -- in case, you run into trouble and need to re-create the Assembly, execute this after Dropping previously created functions... -- DROP ASSEMBLY SqlClrAssemblyName; -- 7..... GO CREATE FUNCTION common.ToBit(@value sql_variant, @valueIfNull bit, @valueIfConvertError bit) RETURNS bit EXTERNAL NAME SqlClrAssemblyName.Scalar.ToBit; GO COMMIT TRAN /* After modifying the DLL it might be necessary to apply the following: ALTER ASSEMBLY SqlClrAssemblyName from 'AssemblyFullPath' WITH PERMISSION_SET = UNSAFE , UNCHECKED DATA; -- !! After changing the Certificate of the DLL, you might be required to update the HASH in the trusted_assemblies by re-invoking the Step 5. !! -- tbd... Create or Drop & re-create Functions... */
using System; using System.Text; using System.Data.SqlTypes; using System.Collections; using System.Globalization; using System.Collections.Generic; using System.Text.RegularExpressions; using Common; using SqlServerClrFunctionsLib; using Microsoft.SqlServer.Server; // TODO: For Unit Testing: Set the following options in this Project's Properties: // - Project | Build | Optimize Code => 0 // // TODO: For Deployment to SQL Server: Set the following options in this Project's Properties: // - Project | Build | Optimize Code => 1 /// <summary> /// Constraints for CLR Functions: /// https://docs.microsoft.com/en-us/sql/relational-databases/user-defined-functions/create-user-defined-functions-database-engine?view=sql-server-2017 /// /// Data Type Mapping: /// https://docs.microsoft.com/en-us/sql/relational-databases/clr-integration-database-objects-types-net-framework/mapping-clr-parameter-data?view=sql-server-2017 /// Extract: (SQL Server -> System.Data.SqlTypes.* -> .NET CLR Data Type /// bigint -> SqlInt64 -> Int64, Nullable{Int64} /// bit -> SqlBoolean -> Boolean, Nullable{Boolean} /// date -> SqlDateTime -> DateTime, Nullable{DateTime} /// decimal -> SqlDecimal -> Decimal, Nullable{Decimal} /// float -> SqlDouble -> Double, Nullable{Double} /// nvarchar -> SqlChars | SqlString -> String | Char[] // SQLChars is a better match for data transfer and access, and SQLString is a better match for performing String operations. /// nvarchar(1) -> SqlChars | SqlString -> Char | Nullable{char} /// sql_variant -> None -> Object /// </summary> public class Scalar { [SqlFunction(DataAccess = DataAccessKind.None, IsDeterministic = true)] public static string LTrim(string value, string charsToTrim) { if (value == null || value == "") return value; else { return value.TrimStart(charsToTrim.ToCharArray()); } } /// <summary> /// Description: Konvertiert den übergebenen Wert @value in den entsprechenden Ja/Nein (Bit)-Wert (1 Bit). /// Es werden auch Strings mit Ja/Nein, J/N, Yes/No, Y/N, Si/No, True/False erkannt, sowie X/- erkannt. /// /// Wenn @value NULL ist, wird @valueIfNull zurückgegeben. /// Wenn ein Konvertierungsfehler auftritt, wird @valueIfConvertError zurückgegeben. /// </summary> /// <param name="value"></param> /// <param name="valueIfNull"></param> /// <param name="valueIfConvertError"></param> /// <returns></returns> [SqlFunction(DataAccess = DataAccessKind.None, IsDeterministic = true)] public static bool? ToBit(Object value, bool? valueIfNull, bool? valueIfConvertError) { convertStringAgain: try { long lngValue; if (value == DBNull.Value) return valueIfNull; else if (value.GetType() == typeof(SqlBoolean)) return (bool?)((SqlBoolean)value).Value; else if (value is bool) return (bool)value; else if (value.GetType() == typeof(SqlString)) { string s = ((SqlString)value).Value; if (s.StartsWithAny(StringComparison.CurrentCultureIgnoreCase, "J", "Y", "S", "T", "W", "1", "X")) return true; else if (s.Length == 0 || s.StartsWithAny(StringComparison.CurrentCultureIgnoreCase, "N", "F", "0", "-", " ")) return false; } else if (value.GetType() == typeof(SqlChars)) { char[] val = ((SqlChars)value).Value; char c = val.Length > 0 ? val[0] : ' '; if (c.In('J', 'Y', 'S', 'T', 'W', '1', 'j', 'y', 's', 't', 'w', 'X', 'x')) return true; else if (c.In('N', 'F', '0', 'n', 'f', '-', ' ')) return false; } else if (value.GetType() == typeof(string)) { value = new SqlString((string)value); goto convertStringAgain; } else if (long.TryParse(Convert.ToString(value), out lngValue)) return lngValue != 0; return valueIfConvertError; } catch (Exception ex) { throw new InvalidOperationException(string.Format("Exception when converting the value '{0}' of type '{1}' to Bool! Message: {2}", value, value.GetType().FullName, ex.Message)); } } }
What is the “best” PacketSize argument to establish a connection to a SQL Server?
The answer is: It depends (on your very specific network)!
In our case a setting of a value higher than 1500 Bytes does not have a positive outcome. That is because the MTU of the Network has a maximum of 1.500 Bytes per Packet. This is often the case, because the implementation of the IP Protocol v2 uses this as the maximum packet size.
To figure out, what is the maximum value in your network, use the following Power Shell Script, originally authored by Thomas Stringer (http://sqlsalt.com), found in this great article by Robert L Davies (http://sqlsoldier.net/wp/sqlserver/networkpacketsizetofiddlewithornottofiddlewith):
$UpperBoundPacketSize = 1500 if ($args -eq "-Q") {$QuietMode = 1} else {$QuietMode = 0} $IpToPing = "mp-bi" do { if ($QuietMode -eq 0) {Write-Host "Testing packet size $UpperBoundPacketSize"} $PingOut = ping $IpToPing -n 1 -l $UpperBoundPacketSize -f $UpperBoundPacketSize -= 1 } while ($PingOut[2] -like "*fragment*") $UpperBoundPacketSize += 1 $Mtu = $UpperBoundPacketSize + 28 Write-Host "MTU: $Mtu" -ForegroundColor Green
To use the OpenQuery() procedures within SQL Server to query a SQL Server Analysis Services Cube, you need to add a Linked Server with Credentials of a valid Domain User, as below:
USE [master] GO /****** Object: LinkedServer [CUBE_FINANCEV2] Script Date: 10/15/2019 11:53:16 AM ******/ EXEC master.dbo.sp_addlinkedserver @server = N'CUBE_FINANCEV2', @srvproduct=N'MSOLAP', @provider=N'MSOLAP', @datasrc=N'mp-bi', @catalog=N'Finance-v2' /* For security reasons the linked server remote logins password is changed with ######## */ EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'CUBE_FINANCEV2',@useself=N'False',@locallogin=NULL,@rmtuser=N'MUC\mp_BI_SQL_AS',@rmtpassword='########' EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'CUBE_FINANCEV2',@useself=N'True',@locallogin=N'MUC\mp_BI_SQL_AS',@rmtuser=NULL,@rmtpassword=NULL GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'collation compatible', @optvalue=N'false' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'data access', @optvalue=N'true' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'dist', @optvalue=N'false' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'pub', @optvalue=N'false' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'rpc', @optvalue=N'false' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'rpc out', @optvalue=N'false' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'sub', @optvalue=N'false' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'connect timeout', @optvalue=N'45' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'collation name', @optvalue=null GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'lazy schema validation', @optvalue=N'false' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'query timeout', @optvalue=N'0' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'use remote collation', @optvalue=N'true' GO EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'remote proc transaction promotion', @optvalue=N'true' GO
To Query the Processing Status or the last date and time of processing of the Cube, you may create and execute the following Stored Procedure (that uses the created Linked Server from above):
USE [DWH] GO /****** Object: StoredProcedure [etl].[GetCubeProcessingStatus] Script Date: 14.10.2019 18:21:42 ******/ SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO -- ============================================= -- Author: Cordt Bien -- Create date: 29.05.2019 -- -- Last Modified: Date, Name (short reason.) -- -- Description: Returns the date and time of the last processing of a cube or perspective. May return multiple rows. -- -- Info: EXEC etl.GetCubeProcessingStatus 'Finance-v2', NULL -- OR EXEC etl.GetCubeProcessingStatus 'Finance-v2', 'Datev-Finance' -- ============================================= CREATE PROCEDURE [etl].[GetCubeProcessingStatus] @cubeDatabaseName sysname = 'Finance-v2', -- must not be null! @cubeOrPerspectiveName sysname = NULL -- optional to limit returned result to one row. --WITH EXECUTE AS 'InternalExecutor' -- (IF YOU USE THE "WITH EXECUTE AS" clause, the Execution will fail due to Security Context Exceptions of the Linked Server) AS BEGIN SET NOCOUNT ON; -- Check Preconditions BEGIN TRY IF @cubeDatabaseName IS NULL RAISERROR('cubeDatabaseName must not be null!', 11, 1); END TRY BEGIN CATCH EXEC internal.ReThrowError @@PROCID; RETURN -1; END CATCH -- Implementation BEGIN TRY IF @cubeDatabaseName = 'Finance-v2' BEGIN SELECT DatabaseName, CubeOrPerspectiveName, CubeOrPerspectiveDisplayName, DATEADD(hour, DATEDIFF(hour, GETUTCDATE(), GETDATE()), LastProcessDate) as LastProcessDate, BaseCubeName FROM OpenQuery(CUBE_FINANCEV2 , N'SELECT [CATALOG_NAME] AS DatabaseName,[CUBE_NAME] AS CubeOrPerspectiveName,[CUBE_CAPTION] AS CubeOrPerspectiveDisplayName,[LAST_DATA_UPDATE] AS LastProcessDate,[BASE_CUBE_NAME] AS BaseCubeName FROM $system.mdschema_cubes WHERE CUBE_SOURCE=1') WHERE (@cubeOrPerspectiveName IS NULL OR CAST(CubeOrPerspectiveName as nvarchar(128)) = @cubeOrPerspectiveName); END ELSE BEGIN RAISERROR('A cubeDatabaseName other than "Finance-v2" is currently not supported or implemented! DEV-Info: Add a new linked server to the requested cube and insert it to this stored procedure (etl.GetCubeProcessingStatus).', 11, 1); SELECT TOP 0 DatabaseName, CubeOrPerspectiveName, CubeOrPerspectiveDisplayName, DATEADD(hour, DATEDIFF(hour, GETUTCDATE(), GETDATE()), LastProcessDate) as LastProcessDate, BaseCubeName FROM OpenQuery(CUBE_FINANCEV2 , N'SELECT [CATALOG_NAME] AS DatabaseName,[CUBE_NAME] AS CubeOrPerspectiveName,[CUBE_CAPTION] AS CubeOrPerspectiveDisplayName,[LAST_DATA_UPDATE] AS LastProcessDate,[BASE_CUBE_NAME] AS BaseCubeName FROM $system.mdschema_cubes WHERE CUBE_SOURCE=1') WHERE (@cubeOrPerspectiveName IS NULL OR CAST(CubeOrPerspectiveName as nvarchar(128)) = @cubeOrPerspectiveName); END END TRY BEGIN CATCH EXEC internal.ReThrowError @@PROCID; RETURN -2; END CATCH END
This short article explains the steps necessary to create and deploy a C# Library to SQL Server to use the faster execution of Scalar Functions, e.g. to convert any value to DateTime.
TODO: Describe C# Project and Code
Activate the “Optimize Code” and “Delay Sign Only” checkmarks in the Properties of the Project and Compile the Project.
Copy the compiled Lib.dll to a Location on the SQL Server, e.g. C:\DWH\Lib.dll
The SQL Server Database Engine User (see Services) requires Read & Execute Permissions for the File Location.
sp_configure 'clr strict security', 0; GO RECONFIGURE; GO USE DWH; GO -- DLL neu registrieren... CREATE ASSEMBLY ClrFunctions from 'C:\DWH\SqlServerClrFunctionsLib.dll' WITH PERMISSION_SET = SAFE;
CREATE FUNCTION common.ToDateTime(@value sql_variant, @valueIfNull datetime2(7), @valueIfConvertError datetime2(7)) RETURNS datetime2(7) EXTERNAL NAME ClrFunctions.Scalar.ToDateTime; GO
-- When code has been changed within DLL: ALTER ASSEMBLY ClrFunctions from 'C:\DWH\SqlServerClrFunctionsLib.dll' WITH PERMISSION_SET = SAFE , UNCHECKED DATA; -- ^^ it's possible that this step does not succeed. A restart of the Database Engine Service may be required. -- To Change the Signature of a Function, first drop then re-create function... BEGIN TRAN DROP FUNCTION common.ToBit; GO CREATE FUNCTION common.ToBit(@value sql_variant, @valueIfNull bit, @valueIfConvertError bit) RETURNS bit EXTERNAL NAME ClrFunctions.Scalar.ToBit; GO COMMIT
https://www.codeproject.com/articles/8874/strong-names-explained
When you change the SQL Server Account or attach Database Files from another computer you may notice the Access Denied error message in SQL Server Management Studio.
To overcome this issue, you may run the following Power Shell script to add the SQL Service User and a userdefined Local Admins Group to each single .mdf, .ldf, .ndf file. Manually setting permissions via Windows Explorer on the parent directory does not change the permissions for each file within.
It might be required to run he Script with elevated Administrative Rights.
function GetFiles($path, [string[]]$exclude) { foreach ($item in Get-ChildItem $path) { if ($exclude | Where {$item -like $_}) { continue } if (Test-Path $item.FullName -PathType Container) { $item.FullName GetFiles $item.FullName $exclude } else { $item.FullName } } } function SetPermissions($file) { if ($file -like "*.mdf" -or $file -like "*.ndf" -or $file -like "*.ldf" -or $file -like "*.ndf*") { $acl = Get-Acl $file $AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule("{YOURDOMAIN}\{SQLSERVERDATABASEENGINESERVICEACCOUNT}","FullControl","Allow") $acl.SetAccessRule($AccessRule) $acl | Set-Acl $file $acl = Get-Acl $file $AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule("{LOCALSERVER}\{LOCALSERVERADMINSGROUP}","FullControl","Allow") $acl.SetAccessRule($AccessRule) $acl | Set-Acl $file } else { $file } } # First Directory to iterate recursively... foreach ($file in GetFiles("G:\DWH")) { SetPermissions($file) } # Second Directory to iterate recursively... foreach ($file in GetFiles("P:\DWH")) { SetPermissions($file) }
When you script your database and want to include the dbo.SYSSSISLOG table, because you’ve created Views or procedures referencing it, the following piece of code does the job:
if not exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[sysssislog]')) CREATE TABLE [sysssislog] ( [id] [int] NOT NULL IDENTITY PRIMARY KEY, [event] [sysname] NOT NULL, [computer] [nvarchar] (128) NOT NULL, [operator] [nvarchar] (128) NOT NULL, [source] [nvarchar] (1024) NOT NULL, [sourceid] [uniqueidentifier] NOT NULL, [executionid] [uniqueidentifier] NOT NULL, [starttime] [datetime] NOT NULL, [endtime] [datetime] NOT NULL, [datacode] [int] NOT NULL, [databytes] [image] NULL, [message] [nvarchar] (2048) NOT NULL,) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] go CREATE PROCEDURE [dbo].[sp_ssis_addlogentry] @event sysname, @computer nvarchar(128), @operator nvarchar(128), @source nvarchar(1024), @sourceid uniqueidentifier, @executionid uniqueidentifier, @starttime datetime, @endtime datetime, @datacode int, @databytes image, @message nvarchar(2048)AS INSERT INTO sysssislog ( event, computer, operator, source, sourceid, executionid, starttime, endtime, datacode, databytes, message ) VALUES ( @event, @computer, @operator, @source, @sourceid, @executionid, @starttime, @endtime, @datacode, @databytes, @message ) RETURN 0 go execute [sp_MS_marksystemobject] N'[dbo].[sp_ssis_addlogentry]' go execute [sp_MS_marksystemobject] N'[sysssislog]' go
The previous script is the same code that Visual Studio or SSIS itself executes when running a ETL Package the first time.