Skip to main content

Power BI

Needs Votes

Dataflows - allow enterprise support to add "staging" step to existing dataflows

Vote (9) Share
Stef's profile image

Stef on 05 Mar 2019 21:48:32

As a data engineer responsible for our Enterprise data platform I would like to collaborate with business users using Data Flows. A problem we are facing is that a source system gets multiple request for more or less the same data since there is no central coordination. For some source systems this leads to serious issues e.g. instability

As our enterprise support I want to take existing dataflows and introduce a staging step which gets the data from the source and stores it in a raw'er format in our data lake. This raw'er format can then be linked to all dependent dataflows. The main benefit is that the source will only get one request and from that point all subsequent step are in an area I can control much better (data lake)

Comments (1)
Stef's profile image Profile Picture

NP Lima on 19 Feb 2021 12:23:57

RE: Dataflows - allow enterprise support to add "staging" step to existing dataflows

What is the advantage of the Datalake for your control, as opposed to having:

1 workspace with "Staging Dataflow" - with only the Data Engineers as authorised users

1 workspace with "Normal Dataflow" - This 2nd dataflow uses the one from Staging as the data source; your reporting builders are the authorised users in this workspace.