SQL Server: Tips and Tricks - 2 (2017)
How to Find Percentage of Empty or Blank Values in every Column of all the Tables in SQL Server Database
Scenario:
You are working as SQL Server developer / TSQL Developer for Insurance Company. You need to perform some analysis of data in one of the database. You are asked to provide Percentage of Empty or Blank Values in every Column of all the tables in Database. Don't mix Empty or Blank values with Null values.
As blank or empty value ('') can be saved in string type columns with data type such as char, nchar, varchar, nvarchar,text and ntext. We will be checking only in columns which has these data types.
Solution:
This information can be really helpful for analysis and we can see if we are loading or entering correct data. If corrections are required after analysis we can write update statements to fix the data.
For this requirement we will be using Cursor. As it is going to run on each column which has data type char, nchar, varchar, nvarchar, text or ntext. It is going to be expensive. If your database is small, you might be fine to run anytime. If you have a big database and it is used by many users or processes, you might want to run this code on weekends or after hours.
One other way can be, take the full back of your production database, restore to another server maybe UAT and run the script there.
USE YourDatabaseName
GO
DECLARE @DatabaseName VARCHAR(100)
DECLARE @SchemaName VARCHAR(100)
DECLARE @TableName VARCHAR(100)
DECLARE @ColumnName VARCHAR(100)
DECLARE @FullyQualifiedTableName VARCHAR(500)
Declare @DataType VARCHAR(50)
--Create Temp Table to Save Results
IF OBJECT_ID('tempdb..#Results') IS NOT NULL
DROP TABLE #Results
CREATE TABLE #Results (
DatabaseName VARCHAR(100)
,SchemaName VARCHAR(100)
,TableName VARCHAR(100)
,ColumnName VARCHAR(100)
,ColumnDataType VARCHAR(50)
,TotalTableRowCount INT
,EmptyRecordCount INT
)
DECLARE Cur CURSOR
FOR
SELECT C.Table_CataLog
,C.Table_Schema
,C.Table_Name
,C.Column_Name
,'[' + C.Table_CataLog + ']' + '.[' + C.Table_Schema + '].'
+ '[' + C.Table_Name + ']' AS FullQualifiedTableName,
C.Data_Type
FROM information_schema.Columns C
INNER JOIN information_Schema.Tables T ON C.Table_Name = T.Table_Name
AND T.Table_Type = 'BASE TABLE'
and (C.Data_Type like '%CHAR%'
or C.Data_Type like '%Text%')
OPEN Cur
FETCH NEXT
FROM Cur
INTO @DatabaseName
,@SchemaName
,@TableName
,@ColumnName
,@FullyQualifiedTableName
,@DataType
WHILE @@FETCH_STATUS = 0
BEGIN
DECLARE @SQL VARCHAR(MAX) = NULL
DECLARE @ColumnNameTRIM VARCHAR(50)
IF @DataType like '%Text%'
BEGIN
SET @ColumnNameTRIM='LTRIM(RTRIM(CAST('+@ColumnName + ' AS NVARCHAR(MAX))))'
END
ELSE
BEGIN
SET @ColumnNameTRIM='LTRIM(RTRIM('+@ColumnName + '))'
END
SET @SQL = ' Select ''' + @DatabaseName + ''' AS DatabaseName, '''
+ @SchemaName + ''' AS TableName,
''' + @TableName + ''' AS SchemaName,
''' + @ColumnName + ''' AS ColumnName,
''' + @DataType + ''' AS ColumnName,
(Select count(*) from ' + @FullyQualifiedTableName + ' with (nolock))
AS TotalTableRowCount,
count(*) as EmptyRecordCount from ' + @FullyQualifiedTableName
+ 'with (nolock) Where '+@ColumnNameTRIM+'='''''
Print @SQL
INSERT INTO #Results
EXEC (@SQL)
FETCH NEXT
FROM Cur
INTO @DatabaseName
,@SchemaName
,@TableName
,@ColumnName
,@FullyQualifiedTableName
,@DataType
END
CLOSE Cur
DEALLOCATE Cur
SELECT *,
Cast((EmptyRecordCount/Cast(TotalTableRowCount as Numeric(13,1)))*100 AS Numeric(4,1))
AS EmptyValuesPercentPerColumn
from #Results
Where EmptyRecordCount<>0
--drop table #Results
Here are the result of above query for one of my database.
