Imke Feldmann on 20 Aug 2021 08:00:28
Although we can use quite a couple of methods to speed up slow queries (I've collected some of them here: Speed/Performance aspects – The BIccountant ), one current limitation for query refresh speed lies in the fact that PQ will always evaluate all steps back to the source to determine the "query plan".
I've tested quite a couple of scenarios where I staged intermediate results into static csv files or sheets in Excel and the performance improvements have been enormous.
So what would be really great is that instead of having to create these workarounds, we could edit a property of a query step so its result would be evaluated as a first step and then stored in cache before continuing to evaluate the further steps.
Actually, maybe even opt for a "bring your own datalake"-version where we can store to Azure Blob if the data is too large for the cache. But in a way, that refresh is properly synchronized.
- Comments (2)
RE: Speed up Power Query by allowing to break the refresh chain
100% agree.I suggest the function be called "Query.Buffer"
RE: Speed up Power Query by allowing to break the refresh chain
If i could improve any 1 thing in PQ it would be this issue. As PQ spreads to other Msft products this becomes an even more important issue. My preference would be for Table.Buffer() and List.Buffer() to actually do what it seems they should do... buffer the results such that downstream query actions need to look no farther upstream than this buffered data. Read the source only once!