Yechezkal Gutfreund on 02 Jan 2024 00:58:46
From what I have read, the PBI Storage engine represents TimeStamp values as a Double. Which means that it does not do any optimized storage (RLE, HashTable, value compression).
We are creating fairly large Fact Tables, that are index'ed by a Timestamp (We do IOT work, and this is network telemetry that has a timestamp and a few other values).
We don't need granularity more than an hour, but by my calculations, even if you go into Seconds since 1900, there is plenty room in a 64 bit integer to store all the days.
No, we don't want to create our own version of TimeStamp (we would lose the time functions). Sounds like a simple feature to test if the value is aligned to a minute or a second, the storage engine could optimize the storage for the value using an 64bit integer.
Right now, your TimeStamp, is ok for financial applications, they don't need granularity of better than a Day, but if you want to break into the big data market for Industrial IOT, Autonomous vehicles, and network monitoring this is going to be important, since we generate gobs of data that is timestamped.