Power Query (M-Language) Apply Where Clause to Table DateTime.Now() / GetDate()

If you want to apply a WHERE clause to the Data of a table, for example to limit the result to only valid rows (where GETDATE() between ValidFrom and ValidTo) go to the Advanced Query Editor and add the Table.SelectRows() statement, as follows:

let
    Quelle = Sql.Database("localhost\DWH_DEV", "DWH", [CommandTimeout=#duration(0, 0, 5, 0)]),
    etl_Datastore = Quelle{[Schema="etl",Item="Datastore"]}[Data],
    ds = Table.SelectRows(etl_Datastore, each [ValidFrom] <= DateTime.Date(DateTime.LocalNow()) and [ValidTo] >=DateTime.Date(DateTime.LocalNow()))
in
    ds

SQL Server Connection Packet Size

What is the “best” PacketSize argument to establish a connection to a SQL Server?
The answer is: It depends (on your very specific network)!

In our case a setting of a value higher than 1500 Bytes does not have a positive outcome. That is because the MTU of the Network has a maximum of 1.500 Bytes per Packet. This is often the case, because the implementation of the IP Protocol v2 uses this as the maximum packet size.

To figure out, what is the maximum value in your network, use the following Power Shell Script, originally authored by Thomas Stringer (http://sqlsalt.com), found in this great article by Robert L Davies (http://sqlsoldier.net/wp/sqlserver/networkpacketsizetofiddlewithornottofiddlewith):

$UpperBoundPacketSize = 1500
 
if ($args -eq "-Q") {$QuietMode = 1}
else {$QuietMode = 0}
 
$IpToPing = "mp-bi"
 
do {
    if ($QuietMode -eq 0) {Write-Host "Testing packet size $UpperBoundPacketSize"}
    $PingOut = ping $IpToPing -n 1 -l $UpperBoundPacketSize -f
    $UpperBoundPacketSize -= 1
} while ($PingOut[2] -like "*fragment*")
 
$UpperBoundPacketSize += 1
 
$Mtu = $UpperBoundPacketSize + 28
 
Write-Host "MTU: $Mtu" -ForegroundColor Green

Reset MySQL Root Password On Windows

If the installation took place using the Windows Installer, you need to do the following:

  1. Stop the MySQL service (via services.msc)
  2. Create a Text File with Read Access to Everyone (easiest way) containing the following code. In this example the file is named C:\Temp\mysql_reset_root.txt:
  3. Invoke the following cmd command (depending on your MySQL Version you may adjust the directory name) with Administrative Access:
    In my case after invoking the command it dit not return a value and idled for more than 30 minutes. But I was able to login very shortly.
-- Text File Content of C:\Temp\mysql_reset_root.txt

ALTER USER 'root'@'localhost' IDENTIFIED BY '{YOURNEWPASSWORDHERE}';
-- Run Cmd with Administrator rights...

C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld --defaults-file="C:\\ProgramData\\MySQL\\MySQL Server 8.0\\my.ini" --init-file=C:\\Temp\\mysql_reset_root.txt

For more information go here… https://dev.mysql.com/doc/mysql-windows-excerpt/5.7/en/resetting-permissions-windows.html

Add Linked Server to OLAP Cube in SQL Server

To use the OpenQuery() procedures within SQL Server to query a SQL Server Analysis Services Cube, you need to add a Linked Server with Credentials of a valid Domain User, as below:

USE [master]
GO

/****** Object:  LinkedServer [CUBE_FINANCEV2]    Script Date: 10/15/2019 11:53:16 AM ******/
EXEC master.dbo.sp_addlinkedserver @server = N'CUBE_FINANCEV2', @srvproduct=N'MSOLAP', @provider=N'MSOLAP', @datasrc=N'mp-bi', @catalog=N'Finance-v2'
 /* For security reasons the linked server remote logins password is changed with ######## */
EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'CUBE_FINANCEV2',@useself=N'False',@locallogin=NULL,@rmtuser=N'MUC\mp_BI_SQL_AS',@rmtpassword='########'
EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'CUBE_FINANCEV2',@useself=N'True',@locallogin=N'MUC\mp_BI_SQL_AS',@rmtuser=NULL,@rmtpassword=NULL
GO

EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'collation compatible', @optvalue=N'false'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'data access', @optvalue=N'true'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'dist', @optvalue=N'false'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'pub', @optvalue=N'false'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'rpc', @optvalue=N'false'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'rpc out', @optvalue=N'false'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'sub', @optvalue=N'false'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'connect timeout', @optvalue=N'45'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'collation name', @optvalue=null
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'lazy schema validation', @optvalue=N'false'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'query timeout', @optvalue=N'0'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'use remote collation', @optvalue=N'true'
GO
EXEC master.dbo.sp_serveroption @server=N'CUBE_FINANCEV2', @optname=N'remote proc transaction promotion', @optvalue=N'true'
GO


To Query the Processing Status or the last date and time of processing of the Cube, you may create and execute the following Stored Procedure (that uses the created Linked Server from above):

USE [DWH]
GO
/****** Object:  StoredProcedure [etl].[GetCubeProcessingStatus]    Script Date: 14.10.2019 18:21:42 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author:		Cordt Bien
-- Create date:		29.05.2019
--
-- Last Modified:	Date, Name (short reason.)
--
-- Description:		Returns the date and time of the last processing of a cube or perspective. May return multiple rows.
--
-- Info:			EXEC etl.GetCubeProcessingStatus 'Finance-v2', NULL
--			OR		EXEC etl.GetCubeProcessingStatus 'Finance-v2', 'Datev-Finance'
-- =============================================
CREATE PROCEDURE [etl].[GetCubeProcessingStatus]
		@cubeDatabaseName sysname = 'Finance-v2',	-- must not be null!
		@cubeOrPerspectiveName sysname = NULL		-- optional to limit returned result to one row.
--WITH EXECUTE AS 'InternalExecutor' -- (IF YOU USE THE "WITH EXECUTE AS" clause, the Execution will fail due to Security Context Exceptions of the Linked Server)
AS
BEGIN
	SET NOCOUNT ON;

	-- Check Preconditions
	BEGIN TRY
		IF @cubeDatabaseName IS NULL RAISERROR('cubeDatabaseName must not be null!', 11, 1);

	END TRY
	BEGIN CATCH	
		EXEC internal.ReThrowError @@PROCID;
		RETURN -1;
	END CATCH

	-- Implementation
	BEGIN TRY

		IF @cubeDatabaseName = 'Finance-v2' BEGIN
			SELECT DatabaseName, CubeOrPerspectiveName, CubeOrPerspectiveDisplayName, DATEADD(hour, DATEDIFF(hour, GETUTCDATE(), GETDATE()), LastProcessDate) as LastProcessDate, BaseCubeName 
			FROM OpenQuery(CUBE_FINANCEV2
					, N'SELECT [CATALOG_NAME]  AS DatabaseName,[CUBE_NAME] AS  CubeOrPerspectiveName,[CUBE_CAPTION] AS CubeOrPerspectiveDisplayName,[LAST_DATA_UPDATE] AS LastProcessDate,[BASE_CUBE_NAME] AS BaseCubeName FROM $system.mdschema_cubes WHERE CUBE_SOURCE=1')
			WHERE (@cubeOrPerspectiveName IS NULL OR CAST(CubeOrPerspectiveName as nvarchar(128)) = @cubeOrPerspectiveName);

		END
		ELSE BEGIN
			RAISERROR('A cubeDatabaseName other than "Finance-v2" is currently not supported or implemented! DEV-Info: Add a new linked server to the requested cube and insert it to this stored procedure (etl.GetCubeProcessingStatus).', 11, 1);

			SELECT TOP 0 DatabaseName, CubeOrPerspectiveName, CubeOrPerspectiveDisplayName, DATEADD(hour, DATEDIFF(hour, GETUTCDATE(), GETDATE()), LastProcessDate) as LastProcessDate, BaseCubeName 
			FROM OpenQuery(CUBE_FINANCEV2
					, N'SELECT [CATALOG_NAME]  AS DatabaseName,[CUBE_NAME] AS  CubeOrPerspectiveName,[CUBE_CAPTION] AS CubeOrPerspectiveDisplayName,[LAST_DATA_UPDATE] AS LastProcessDate,[BASE_CUBE_NAME] AS BaseCubeName FROM $system.mdschema_cubes WHERE CUBE_SOURCE=1')
			WHERE (@cubeOrPerspectiveName IS NULL OR CAST(CubeOrPerspectiveName as nvarchar(128)) = @cubeOrPerspectiveName);
		END

	END TRY
	BEGIN CATCH	
		EXEC internal.ReThrowError @@PROCID;
		RETURN -2;
	END CATCH
END

Register C# CLR Assembly in SQL Server

This short article explains the steps necessary to create and deploy a C# Library to SQL Server to use the faster execution of Scalar Functions, e.g. to convert any value to DateTime.

TODO: Describe C# Project and Code

Project Build Steps

Activate the “Optimize Code” and “Delay Sign Only” checkmarks in the Properties of the Project and Compile the Project.

Deployment of Lib to SQL Server

Copy the compiled Lib.dll to a Location on the SQL Server, e.g. C:\DWH\Lib.dll
The SQL Server Database Engine User (see Services) requires Read & Execute Permissions for the File Location.

Execute Script in SSMS to Register Assembly

sp_configure 'clr strict security', 0;  
GO  
RECONFIGURE;  
GO  
USE DWH;
GO

-- DLL neu registrieren...
CREATE ASSEMBLY ClrFunctions from 'C:\DWH\SqlServerClrFunctionsLib.dll' WITH PERMISSION_SET = SAFE;

Register a new Scalar Function

CREATE FUNCTION common.ToDateTime(@value sql_variant,
	@valueIfNull datetime2(7),
	@valueIfConvertError datetime2(7)) RETURNS datetime2(7)
	EXTERNAL NAME ClrFunctions.Scalar.ToDateTime;
GO

Some Scripts for Maintanence

-- When code has been changed within DLL:
	ALTER ASSEMBLY ClrFunctions from 'C:\DWH\SqlServerClrFunctionsLib.dll' WITH PERMISSION_SET = SAFE , UNCHECKED DATA;
-- ^^ it's possible that this step does not succeed. A restart of the Database Engine Service may be required.

-- To Change the Signature of a Function, first drop then re-create function...
BEGIN TRAN

DROP FUNCTION common.ToBit;
GO
CREATE FUNCTION common.ToBit(@value sql_variant,
	@valueIfNull bit,
	@valueIfConvertError bit) RETURNS bit
	EXTERNAL NAME ClrFunctions.Scalar.ToBit;
GO

COMMIT

More Information to Strong Names and its purpose

https://www.codeproject.com/articles/8874/strong-names-explained