Nick VanLent on 08 Feb 2024 20:39:41
When we setup a dataflow with incremental refresh, to refresh the same number of partitions as what are being stored, the oldest partition is not being refreshed. For example, we set "Store rows from the past" to 36 Months, and also set "Refresh rows from the past" to 36 Months as well. The partition for 36 months before the current month is still left in the storage, but it is not included in the refresh. So this idea is to have the ability to refresh the same number of partitions that are being stored.
On a side note, I assumed this was an issue, so I submitted it in a support ticket, and was told that it is by design.