Power BI — Incremental refresh

When we deal with large datasets, data refreshing might take long time. No wonder, Power BI refreshes entire data set, even if any record hasn’t hanged. So, we can use incremental refresh functionality.

You can find how to configure incremental refresh in Microsoft Docs. It is pretty simple.

We campare time of refreshing dataset without incremental refresh and with incremental refresh.

My test dataset contains 6.5 milion rows and 3 columns.

Refershing dataset took around 40 seconds.

Next, I’ve set incremental refresh like bellow. Filter is set on Created column.

2 full years are stored and 2 complete months are refreshed.

I’ve inserted around 1 milion rows with Created column values set to ‘2020–11–01’ so it is in a range of incremental refresh rule.

Adding 1 minion rows took 1 minute. Next I added 300 rows. Process took 12 seconds. The difference is visible. Instead of refreshing entire dataset you can add only new records.

Here is a summary:

At the end, I’ve set Detect data changes and updated 300 rows.

It took 14 seconds.

As you see Incremental Refresh saves a lot of time and resources. In this scale the difference is small, several dozen seconds. But when you store houndreds million or bilions row it makes the difference.

BI Developer (Microsoft Stack)