Plinio Nunez on 25 May 2024 14:56:11
While it is understandable that this is a proprietary technology from Databricks, it is a very desirable feature for Data Engineers who consume events (either from logs or systems like Shareplex).
Being able to simplify CDC changes or consume json events from Shareplex (Oracle -> EventHub) would simplify replication from Oracle-to-Cloud and make fabric the go-to solution for data ingestion at this level.
The current work-around for this is an event stream -> delta-lake and then spin up a spark job to apply changes in order to delta tables.
- Comments (1)
RE: Delta Live Table Equivalent for MS Fabric
It would be good if you could define an event stream with a delta table change feed as the source. Currently, you can ingest streaming data from an external source but there's no easy way to process that through a series of discrete stages for a medallion architecture