Power BI
Needs VotesDataflows - allow enterprise support to add "staging" step to existing dataflows
Stef on 05 Mar 2019 21:48:32
As a data engineer responsible for our Enterprise data platform I would like to collaborate with business users using Data Flows. A problem we are facing is that a source system gets multiple request for more or less the same data since there is no central coordination. For some source systems this leads to serious issues e.g. instability
As our enterprise support I want to take existing dataflows and introduce a staging step which gets the data from the source and stores it in a raw'er format in our data lake. This raw'er format can then be linked to all dependent dataflows. The main benefit is that the source will only get one request and from that point all subsequent step are in an area I can control much better (data lake)
- Comments (1)
RE: Dataflows - allow enterprise support to add "staging" step to existing dataflows
What is the advantage of the Datalake for your control, as opposed to having:
1 workspace with "Staging Dataflow" - with only the Data Engineers as authorised users
1 workspace with "Normal Dataflow" - This 2nd dataflow uses the one from Staging as the data source; your reporting builders are the authorised users in this workspace.