- When performing a mailbox backup from Office 365, progress may slow down unexpectedly, especially when processing the “Calendar” folder
- Dirty data is the usual cause of this “stall”.
- It occurs because dirty data causes server-side errors that require special handling.
- BackupAssist 365 is engineered for performance and resilience, and deals with this with high efficiency – despite the appearance of a stall.
- It’s generally nothing to worry about.
Backing up Office 365 mailboxes to local PST files – how we engineered for performance with large batches
On the whole, BackupAssist 365 has been optimized for performance when downloading mailboxes from Office 365 or Exchange to PST file.
A key part of this is minimizing the number of requests to the server. Each request incurs latency, and therefore is expensive. By default, BackupAssist 365 will download mailbox items from Office 365 or Exchange in large batches, such as 400 items at a time.
There are occasional circumstances where it might appear that BackupAssist 365 gets “stuck” or slows down – especially with Calendar items. In the progress log, you might see a message like:
“Downloading items 1 to 400 from Calendar” – and this message may stay there for minutes.
Note: this problem can occur whether you use a single login to back up multiple mailboxes, or not.
Dirty data in the mailbox is not possible to back up
Almost always, the cause of the slowdown is dirty data. Here’s why.
When we issue the commands to backup a mailbox from Office 365 or Exchange, along the lines of “fetch items #1 to #400 from the Calendar folder”, the server will retrieve those items and return them in one operation.
However, occasionally we get an error message instead. If just one of those items is “dirty” – that means, if it does not confirm to the expected data format – then the Exchange server will throw an error, instead of returning any data.
(We’ll explain how dirty data gets into a mailbox a bit later.)
Why it can get slow (the price of resilient backups for Office 365 and Exchange)
In the example above, instead of skipping ignoring the error returned from the server and skipping all 400 items, the correct thing to do is to download every item that isn’t “dirty”.
But how do we know which one of the 400 items is the dirty one? Unfortunately, we don’t. We have to search for it… and the most efficient way to search for the dirty data is to do a kind of binary search: to break the request for 400 items into two requests for 200 items. Somewhere, one of those requests will fail. Then we break the next failed requests into two requests for 100 items. And then down to two requests for 50 items. And so on… until we have downloaded everything we can.
This means that the number of requests to the server will explode.
Here’s an example
In the example above, let’s say that item #2 is dirty but every other item is normal, then you’d see behaviour something like this:
1. Fetch 400 items (#1 - #400) – failed
2. Fetch 200 items (#1 - #200) – failed
3. Fetch 200 items (#201 - #400) – success
4. Fetch 100 items (#1 - #100) – failed
5. Fetch 100 items (#101 - #200) – success
6. Fetch 50 items (#1 - #50) – failed
7. Fetch 50 items (#51 - #100) – success
8. Fetch 25 items (#1 - #25) – failed
9. Fetch 25 items (#26 - #50) – success
10. Fetch 13 items (#1 - #13) – failed
11. Fetch 12 items (#14 - #25) – success
12. Fetch 7 items (#1 - #7) – failed
13. Fetch 6 items (#8 - #13) – success
14. Fetch 4 items (#1 - #4) – failed
15. Fetch 3 items (#5 - #7) – success
16. Fetch item #1 – success
17. Fetch item #2 – failed
18. Fetch item #3 – success
19. Fetch item #4 – success
So in the example above, there are 19 times the number of requests to Exchange caused by one dirty item.
In summary you can see,
- It takes 1 request to get 400 items when all items are clean, and
- It takes 19 requests to get 399 items when one item is dirty.
The more dirty items there are, the more number of server requests are needed. Because we don’t know how many dirty items there are in a batch, doing the binary search is going to be the most efficient way to ensure we back up all the valid items.
What causes the dirty data in Office 365 or Exchange mailboxes?
We are not exactly sure what the causes are. We have noticed that it is more prevalent with data that has been migrated one or more times. For example, it happened on our own on-premise Exchange Server when data had been migrated from Exchange 2003 to Exchange 2007 to Exchange 2010, and also included data added from PST files using Outlook.
Any fault with the EDB files in Exchange, or faulty copying form PST files, or any bug in Outlook clients could have caused the problem. Another possibility is 3rd party applications and mail clients that add items to a mailbox. If these clients introduced data that did not meet the required specifications, that 3rd party app might appear to work but cause problems with backups.
When designing BackupAssist 365, we had to engineer it for reliability and dependability. As explained above, that means sometimes performance appears to slow down when there is dirty data present. The server returns an error when trying to retrieve the dirty data.
Despite the appearance of a stall, BackupAssist 365 attempts to handle the presence of dirty data in the most efficient way possible. Therefore, the Office 365 mailbox backup PST files will contain all the valid items in it.
If you experience a slowdown or stall lasting more than an hour, please contact our technical support department and we’ll investigate it further.