Event Stream Query Editor in EventStream Definition
Hello,
Currently, the EventStream UI for defining EventStream flows is working perfectly. While most of us enjoy the no-code experience, it would be greatly appreciated if we can be allowed to edit or entirely define the query (similar to ASA Job queries).
Delta Live Table Equivalent for MS Fabric
While it is understandable that this is a proprietary technology from Databricks, it is a very desirable feature for Data Engineers who consume events (either from logs or systems like Shareplex).
Being able to simplify CDC changes or consume json events from Shareplex (Oracle -> EventHu...
Workspace-Level Parameters
The ability to have workspace-level parameters (similar to ADF) would be a nice way of standardizing naming conventions, share parameters between services, etc
EventStream to allow MultiJson messages as source
Azure Stream Analytics and KQL can ingest data from EventHub messages in a multi-json fashion. That is, messages do not necessarily have the same schema. This is common in some CDC and IoT scenarios.
ASA will even let us define UDF and custom queries to manage these messages.
Whil...
Allow partitions in EventStream Delta Sink
I have an event stream (reading from an event hub) dumping data periodically in a delta table. This delta table, due to the nature of the data it contains, can grow very large. For example, a small streaming dataset can produce around 100 million rows in two weeks. Since data is constantly bei...
Salesforce Connector for Pipelines
Hello,
The azure data factory is currently using a new version of the Salesforce connector that requires the creation of a connected app on the salesforce side. This has been working great. However, when migrating our pipelines from Azure Data Factory to Fabric we see that th...
Delta 3.2 Support
Hello,
Delta 3.2 brings in nice optimization features and most notable full support for Liquid Clustering. Would we be able to have delta 3.2 as a possible preview runtime in the near future?
Make procedures visible in Data Studio
Currently procedures are only automatically displayed in Management Studio.
It would be nice to have them displayed in Azure Data Studio as well.
Salesforce Connector | Registered Application Authentication (as in azure data factory)
Hello,
Would it be possible to get the ADF Salesforce connector to use the same authentication schema as the other Data Factory. Most of us have moved to this schema and it feels like a step backwards to use a regular user for the Data Factory in Fabric.
Lakehouse T-SQL endpoint on-demand refresh
There is an ofter large delay for data to become available via the T-SQL Endpoint. With barely 142 delta tables, some newly created delta tables wont even be available until we manually hit refresh next to the warehouse endpoint in the T-SQL view. While the data is readily available to spark, ...