We have alerts set up to record logon failures. This is done for security reasons to alert us of anything suspicious. Historically, on a daily basis the login failures are minimal and mostly due to an incorrectly entered password. Since upgrading to SQL Server 2012 last summer and with using AlwaysOn High Availabity the login failures dramatically increased in all environments. The login failures are happening on SQL Servers that and the origin is from another server using AlwaysOn High Availabity.
I ran a profile trace to detect what is happening. In all of the instances, a user had a connection to the instance of SQL Server with AlwaysOn High Availability through SSMS and was doing nothing.
Looking at the in the profiler trace it looks like replication code but it is being run under a random user that is logged on instead of a system account. For example:
exec sp_executesql N'SELECT
ISNULL((case dmi.mirroring_redo_queue_type when N''UNLIMITED'' then 0 else dmi.mirroring_redo_queue end),0) AS [MirroringRedoQueueMaxSize],
ISNULL(dmi.mirroring_connection_timeout,0) AS [MirroringTimeout],
ISNULL(dmi.mirroring_partner_name,'''') AS [MirroringPartner],
ISNULL(dmi.mirroring_partner_instance,'''') AS [MirroringPartnerInstance],
ISNULL(dmi.mirroring_role,0) AS [MirroringRole],
ISNULL(dmi.mirroring_safety_level + 1, 0) AS [MirroringSafetyLevel],
ISNULL(dmi.mirroring_state + 1, 0) AS [MirroringStatus],
ISNULL(dmi.mirroring_witness_name,'''') AS [MirroringWitness],
ISNULL(dmi.mirroring_witness_state + 1, 0) AS [MirroringWitnessStatus],
CAST(case when dmi.mirroring_partner_name is null then 0 else 1 end AS bit) AS [IsMirroringEnabled],
ISNULL(dmi.mirroring_guid,''00000000-0000-0000-0000-0000000000000000'') AS [MirroringID],
ISNULL(dmi.mirroring_role_sequence,0) AS [MirroringRoleSequence],
ISNULL(dmi.mirroring_safety_sequence,0) AS [MirroringSafetySequence],
ISNULL(dmi.mirroring_failover_lsn,0) AS [MirroringFailoverLogSequenceNumber],
dtb.is_ansi_null_default_on AS [AnsiNullDefault],
dtb.is_ansi_nulls_on AS [AnsiNullsEnabled],
dtb.is_ansi_padding_on AS [AnsiPaddingEnabled],
dtb.is_ansi_warnings_on AS [AnsiWarningsEnabled],
dtb.is_arithabort_on AS [ArithmeticAbortEnabled],
dtb.is_auto_shrink_on AS [AutoShrink],
dtb.is_cursor_close_on_commit_on AS [CloseCursorsOnCommitEnabled],
dtb.is_concat_null_yields_null_on AS [ConcatenateNullYieldsNull],
dtb.is_numeric_roundabort_on AS [NumericRoundAbortEnabled],
dtb.is_quoted_identifier_on AS [QuotedIdentifiersEnabled],
dtb.is_read_only AS [ReadOnly],
dtb.is_recursive_triggers_on AS [RecursiveTriggersEnabled],
dtb.is_local_cursor_default AS [LocalCursorsDefault],
dtb.page_verify_option AS [PageVerify],
dtb.recovery_model AS [RecoveryModel],
dtb.user_access AS [UserAccess],
dtb.is_db_chaining_on AS [DatabaseOwnershipChaining],
dtb.is_auto_update_stats_async_on AS [AutoUpdateStatisticsAsync],
dtb.is_date_correlation_on AS [DateCorrelationOptimization],
dtb.is_trustworthy_on AS [Trustworthy],
dtb.name AS [Name],
dtb.database_id AS [ID],
dtb.create_date AS [CreateDate],
dtb.is_auto_create_stats_on AS [AutoCreateStatisticsEnabled],
dtb.is_auto_update_stats_on AS [AutoUpdateStatisticsEnabled],
dtb.is_parameterization_forced AS [IsParameterizationForced],
dtb.is_read_committed_snapshot_on AS [IsReadCommittedSnapshotOn],
dtb.is_auto_close_on AS [AutoClose],
dtb.is_broker_enabled AS [BrokerEnabled],
CAST(isnull(dtb.source_database_id, 0) AS bit) AS [IsDatabaseSnapshot],
ISNULL(DB_NAME(dtb.source_database_id), N'''') AS [DatabaseSnapshotBaseName],
dtb.is_fulltext_enabled AS [IsFullTextEnabled],
dtb.service_broker_guid AS [ServiceBrokerGuid],
dtb.snapshot_isolation_state AS [SnapshotIsolationState],
(dtb.is_published*1+dtb.is_subscribed*2+dtb.is_merge_published*4) AS [ReplicationOptions],
suser_sname(dtb.owner_sid) AS [Owner],
ISNULL(dtb.log_reuse_wait,0) AS [LogReuseWaitStatus],
drs.recovery_fork_guid AS [RecoveryForkGuid],
drs.database_guid AS [DatabaseGuid],
CAST((case when drs.last_log_backup_lsn is not null then 1 else 0 end) AS bit) AS [HasFullBackup],
CAST(case when dtb.name in (''master'',''model'',''msdb'',''tempdb'') then 1 else dtb.is_distributor end AS bit) AS [IsSystemObject],
CAST(case when ctb.database_id is null then 0 else 1 end AS bit) AS [ChangeTrackingEnabled],
CAST(ISNULL(ctb.is_auto_cleanup_on,0) AS bit) AS [ChangeTrackingAutoCleanUp],
ISNULL(ctb.retention_period,0) AS [ChangeTrackingRetentionPeriod],
CAST(ISNULL(ctb.retention_period_units,0) AS tinyint) AS [ChangeTrackingRetentionPeriodUnits],
dtb.containment AS [ContainmentType],
dtb.default_language_lcid AS [DefaultLanguageLcid],
dtb.default_language_name AS [DefaultLanguageName],
dtb.default_fulltext_language_lcid AS [DefaultFullTextLanguageLcid],
ISNULL(dtb.default_fulltext_language_name,N'''') AS [DefaultFullTextLanguageName],
CAST(dtb.is_nested_triggers_on AS bit) AS [NestedTriggersEnabled],
CAST(dtb.is_transform_noise_words_on AS bit) AS [TransformNoiseWords],
dtb.two_digit_year_cutoff AS [TwoDigitYearCutoff],
dtb.target_recovery_time_in_seconds AS [TargetRecoveryTime],
case
when dtb.collation_name is null then 0x200
else 0
end |
case
when 1 = dtb.is_in_standby then 0x40
else 0
end |
case dtb.state
when 1 then 0x2
when 2 then 0x8
when 3 then 0x4
when 4 then 0x10
when 5 then 0x100
when 6 then 0x20
else 1
end
AS [Status],
CAST(( case LOWER(convert( nvarchar(128), DATABASEPROPERTYEX(dtb.name, ''Updateability''))) when ''read_write'' then 1 else 0 end) AS bit) AS [IsUpdateable],
CAST(dtb.is_encrypted AS bit) AS [EncryptionEnabled],
CAST(dtb.is_honor_broker_priority_on AS bit) AS [HonorBrokerPriority],
CAST(has_dbaccess(dtb.name) AS bit) AS [IsAccessible],
ISNULL(fsopt.directory_name , N'''') AS [FilestreamDirectoryName],
ISNULL(fsopt.non_transacted_access , 0) AS [FilestreamNonTransactedAccess],
dtb.name AS [DatabaseName2],
dtb.containment AS [ContainmentType2]
FROM
master.sys.databases AS dtb
LEFT OUTER JOIN sys.database_mirroring AS dmi ON dmi.database_id = dtb.database_id
LEFT OUTER JOIN sys.database_recovery_status AS drs ON drs.database_id = dtb.database_id
LEFT OUTER JOIN sys.change_tracking_databases AS ctb ON ctb.database_id = dtb.database_id
LEFT OUTER JOIN sys.database_filestream_options AS fsopt ON fsopt.database_id = dtb.database_id
WHERE
(dtb.name=@_msparam_0)',N'@_msparam_0 nvarchar(4000)',@_msparam_0=N'DATABASENAME'
It runs this code repeatedly and for all databases on the SQL Server instance. These databases are for different applications (some of which communicate with each other). The SQL login ID it runs under may be the dbo of one database but not have permissions to other databases.
Here is the code that I trace that generates a login failure on another SQL Server:
SELECT
SCHEMA_NAME(udf.schema_id) AS [Schema],
udf.name AS [Name],
udf.object_id AS [ID],
(case when 'FN' = udf.type then 1 when 'FS' = udf.type then 1 when 'IF' = udf.type then 3 when 'TF' = udf.type then 2 when 'FT' = udf.type then 2 else 0 end) AS [FunctionType],
CASE WHEN udf.type IN ('FN','IF','TF') THEN 1 WHEN udf.type IN ('FS','FT') THEN 2 ELSE 1 END AS [ImplementationType],
CAST(
case
when udf.is_ms_shipped = 1 then 1
when (
select
major_id
from
sys.extended_properties
where
major_id = udf.object_id and
minor_id = 0 and
class = 1 and
name = N'microsoft_database_tools_support')
is not null then 1
else 0
end
AS bit) AS [IsSystemObject],
CAST(CASE WHEN ISNULL(smudf.definition, ssmudf.definition) IS NULL THEN 1 ELSE 0 END AS bit) AS [IsEncrypted],
CAST(ISNULL(OBJECTPROPERTYEX(udf.object_id, N'IsSchemaBound'),0) AS bit) AS [IsSchemaBound],
usrt.name AS [DataType],
ISNULL(baset.name, N'') AS [SystemType],
CAST(CASE WHEN baset.name IN (N'nchar', N'nvarchar') AND ret_param.max_length <> -1 THEN ret_param.max_length/2 ELSE ret_param.max_length END AS int) AS [Length],
CAST(ret_param.precision AS int) AS [NumericPrecision],
CAST(ret_param.scale AS int) AS [NumericScale],
ISNULL(xscret_param.name, N'') AS [XmlSchemaNamespace],
ISNULL(s2ret_param.name, N'') AS [XmlSchemaNamespaceSchema],
ISNULL( (case ret_param.is_xml_document when 1 then 2 else 1 end), 0) AS [XmlDocumentConstraint],
s1ret_param.name AS [DataTypeSchema]
FROM
sys.all_objects AS udf
LEFT OUTER JOIN sys.sql_modules AS smudf ON smudf.object_id = udf.object_id
LEFT OUTER JOIN sys.system_sql_modules AS ssmudf ON ssmudf.object_id = udf.object_id
LEFT OUTER JOIN sys.all_parameters AS ret_param ON ret_param.object_id = udf.object_id and ret_param.is_output = @_msparam_0
LEFT OUTER JOIN sys.types AS usrt ON usrt.user_type_id = ret_param.user_type_id
LEFT OUTER JOIN sys.types AS baset ON (baset.user_type_id = ret_param.system_type_id and baset.user_type_id = baset.system_type_id) or ((baset.system_type_id = ret_param.system_type_id) and (baset.user_type_id = ret_param.user_type_id) and (baset.is_user_defined
= 0) and (baset.is_assembly_type = 1))
LEFT OUTER JOIN sys.xml_schema_collections AS xscret_param ON xscret_param.xml_collection_id = ret_param.xml_collection_id
LEFT OUTER JOIN sys.schemas AS s2ret_param ON s2ret_param.schema_id = xscret_param.schema_id
LEFT OUTER JOIN sys.schemas AS s1ret_param ON s1ret_param.schema_id = usrt.schema_id
WHERE
(udf.type in ('TF', 'FN', 'IF', 'FS', 'FT'))
ORDER BY
[Schema] ASC,[Name] ASC
The query above accesses master.sys.all_objects and was run under a database where the login has no permissions other then that the guest account is enabled. I logged in with the SQL Login that generated the login failure and then shortened the query and ran it under the same database:
select * from sys.all_objects
where type in ('TF', 'FN', 'IF', 'FS', 'FT')
This caused the same login failure on an intance of SQL Server on a different server. The problem is that this database has a table-vlued function that queries a synonym which accesses a database on a different server through a linked server. The SQL Server login that this code runs under is not a local login on this linked server so it causes a login failure on the server it is trying to connect to.
This happening throughout the day and run using a connection that people have open and unaware to them.
Why is it run under these conditions? This looks like a polling problem. Is there any setting I could change so that these adhoc queries are not run under random user connections but under a designated account?