Merge pull request #309829 from silasmendes-ms/synapse-serverless-missing-stats-error-19731

Queries on external tables may not complete due to missing stats

Author: prmerger-automator[bot]

articles/synapse-analytics/known-issues.md

@@ -30,13 +30,14 @@ To learn more about Azure Synapse Analytics, see the [Azure Synapse Analytics Ov
 |Azure Synapse dedicated SQL pool|[Queries failing with Data Exfiltration Error](#queries-failing-with-data-exfiltration-error)|Has workaround|
 |Azure Synapse dedicated SQL pool|[UPDATE STATISTICS statement fails with error: "The provided statistics stream is corrupt."](#update-statistics-failure)|Has workaround|
 |Azure Synapse dedicated SQL pool|[Enable TDE gateway timeouts in ARM deployment](#enable-tde-gateway-timeouts-in-arm-deployment)|Has workaround|
+|Azure Synapse dedicated SQL pool|[Proxied connections can be affected by Gateway, resulting in connection failures](#proxied-connections-may-result-in-failure-due-to-gateway)|No workaround|
 |Azure Synapse serverless SQL pool|[Query failures from serverless SQL pool to Azure Cosmos DB analytical store](#query-failures-from-serverless-sql-pool-to-azure-cosmos-db-analytical-store)|Has workaround|
 |Azure Synapse serverless SQL pool|[Azure Cosmos DB analytical store view propagates wrong attributes in the column](#azure-cosmos-db-analytical-store-view-propagates-wrong-attributes-in-the-column)|Has workaround|
 |Azure Synapse serverless SQL pool|[Query failures in serverless SQL pools](#query-failures-in-serverless-sql-pools)|Has workaround|
 |Azure Synapse serverless SQL pool|[Storage access issues due to authorization header being too long](#storage-access-issues-due-to-authorization-header-being-too-long)|Has workaround|
 |Azure Synapse serverless SQL pool|[Querying a view shows unexpected results](#querying-a-view-shows-unexpected-results)|Has workaround|
 |Azure Synapse serverless SQL pool|[Queries longer than 7,500 characters may not appear in Log Analytics](#queries-longer-than-7500-characters-may-not-appear-in-log-analytics)|Has workaround|
-|Azure Synapse serverless SQL pool|[Proxied connections can be affected by Gateway, resulting in connection failures](#proxied-connections-may-result-in-failure-due-to-gateway)|No workaround|
+| Azure Synapse serverless SQL pool | [Queries on external tables may take longer or not complete due to missing statistics](#queries-on-external-tables-may-take-longer-or-not-complete-due-to-missing-statistics) | Has workaround |
 |Azure Synapse Workspace|[Blob storage linked service with User Assigned Managed Identity (UAMI) isn't getting listed](#blob-storage-linked-service-with-user-assigned-managed-identity-uami-is-not-getting-listed)|Has workaround|
 |Azure Synapse Workspace|[Failed to delete Synapse workspace & Unable to delete virtual network](#failed-to-delete-synapse-workspace--unable-to-delete-virtual-network)|Has workaround|
 |Azure Synapse Workspace|[REST API PUT operations or ARM/Bicep templates to update network settings fail](#rest-api-put-operations-or-armbicep-templates-to-update-network-settings-fail)|Has workaround|
@@ -273,6 +274,44 @@ Suggested workarounds are:
 - Use the `sys.dm_exec_requests_history` view in your Synapse Serverless SQL pool to access historical query execution details.
 - Refactor the query to reduce its length below 7,500 characters, if feasible.
 
+### Queries on external tables may take longer or not complete due to missing statistics
+
+Serverless SQL pool automatically creates statistics for external tables. However, a known issue can prevent statistics from being created for certain columns, which may result in suboptimal query plans and degraded query performance.
+
+**Workaround**
+
+The recommended workaround is the following:
+
+- Run the following diagnostic query in the database hosting the external table to identify columns where `stats_name` or `stats_date` is NULL.
+- If statistics are missing, copy the text from the `cmd_create_stats` column and run it in a new session to create the statistics.
+- Make sure all relevant tables and columns have statistics and that the statistics are recent. If statistics are outdated, drop and recreate them to help the SQL optimizer generate more efficient query plans.
+- Consider implementing an automation to periodically drop and recreate statistics to help maintain consistent query performance.
+
+```sql
+SELECT 
+   schema_name(o.schema_id) AS [schema_name],
+   object_name(o.object_id) AS [table_name],
+   o.create_date AS [table_date_create],
+   c.name AS [column_name],
+   s.name as [stats_name],
+   STATS_DATE(s.object_id, s.stats_id) AS [stats_date],
+   'CREATE STATISTICS [' + 'Stats_' + c.name + '] ON [' + schema_name(o.schema_id) + '].[' + object_name(o.object_id) + '] ([' + c.name + ']) WITH FULLSCAN;' AS cmd_create_stats,
+   'DROP STATISTICS [' + schema_name(o.schema_id) + '].[' + object_name(o.object_id) + '].[' + 'Stats_' + c.name + '];' AS cmd_drop_stats,
+   'DROP STATISTICS [' + schema_name(o.schema_id) + '].[' + object_name(s.object_id) + '].[' + s.name + '];' AS cmd_drop_existing_stats
+FROM sys.objects AS o
+INNER JOIN sys.columns AS c 
+   ON o.object_id = c.object_id
+LEFT JOIN sys.stats_columns AS sc 
+   ON sc.object_id = c.object_id AND sc.column_id = c.column_id
+LEFT JOIN sys.stats AS s
+   ON s.object_id = sc.object_id AND s.stats_id = sc.stats_id
+WHERE o.type = 'U'
+ORDER BY [schema_name], [table_name], [column_name];
+```
+
+In addition to mitigating this issue, maintaining fresh statistics can improve overall query performance. 
+ 
+
 ## Azure Synapse Apache Spark pool active known issues summary
 
 ### Starting a Spark session (with custom python libraries) is taking longer than usual