"Checking for corrupted database..." Scan & "Backup..." Need To Be Faster

Frequently Asked Questions
1

The “Checking for corrupted database…” scan needs to be sped up or changed so it is much faster. My client who is using eM Client v9.0.1708 on Windows 10 sometimes has the program lock up and crash. He reopens it and sits through a 2+ hour scan. He has 3 email accounts: 1 Microsoft Exchange account with 30 GBs of emails and 2 IMAP accounts (one with ~9 GBs and the other with ~15 GBs of emails). He also has ~1,500 contacts. The eM Client folder in %APPDATA% is 54.0 GBs in size containing 320 files and 13 folders presently.

It is faster to copy/paste the %APPDATA%\eM Client folder from another source over to the computer doing the scan (after cancelling it and closing eM Client completely) to fix the issue than to wait for the scan. ~15 minutes to copy/paste the folder as opposed to 2+ hours of scanning.

This scan prevents him from doing his business which is bad in a business setting.

Another thing that takes too long is the Backup. Why have it go to a .ZIP file and be compressed? Why not instead have the Backup make a copy of the %APPDATA%\eM Client folder with the resulting folder having the same name as what is used on the .ZIP backup(s)? This would make it MUCH FASTER.

That may be the answer to replace the long database scan. Allow the user (who does backups) to cancel the scan, go to restore, have restore delete the %APPDATA%\eM Client folder, have restore copy/paste the latest backup folder to %APPDATA%, rename the copied backup folder to eM Client, and then reopen eM Client. Some sort of integrated countdown timer would be nice as well.

2

If there is some inconsistency with the database, maybe caused when the application was incorrectly terminated, the database check on start needs to run to ensure data integrity. It would be better to discover the cause of the corruption, and prevent that, rather than restore every time from a previous copy.

The backup does more than just zip a folder. It optimizes and checks the data for any errors as well, as there would be no point in making a backup that contains faulty data. But the time it takes shouldn’t be an issue as it runs in the background so doesn’t interfere with use of eM Client.

3

@Gary I am sorry, but in a business setting, it is now. It has to work NOW, or the work cannot be done. No one in business wants to wait 2-3 hours for a database scan which will not tell them what happened. They just want eM Client to open and work… NOW. I made a batch file to backup eM Client and another to restore eM Client. The backup batch file is neat as it does incremental backups. I would share the commands I am using if that is allowed here.

4

It is not recommended, but if that works for you . . .

Better to ensure eM Client is shut down correctly so there is no need for a database check on start.

5

There is no need to do a database check if the backup being restored is fine either.

I am happy to share the batch commands from the files I made if it is allowed.

6

There is no reason to restore a backup if the database is not corrupt.

Under normal use, there shouldn’t be any database check, so the dalay to restore the backup before every start is unacceptable. Better to address why it is being corrupted, than to restore a backup that may not have all the current data and preferences. And of course if the backup itself is corrupt, which can happen . . .

7

Hence the reason for restoring the backup, to correct a corrupt database the fast way with a database that is not corrupt. It literally takes about 15 minutes to restore 50+ GBs of email using the batch file I made as opposed to a 2-3 hour ordeal with watching a database scan run.

There is no need to do a backup of a corrupt database. Who would do that? “OH! My database is bad? I better back that up now!” That is nuts, but I guess a minority might do that.

And why did you change this to a FAQ instead of leaving it as a Feature Request? I am requesting eM Client do database scans WAY FASTER that it presently does them. There is no feasible logic that the eM Client developers could possibly think making a user wait for 2-3 hours or more is acceptable especially when that scan doesn’t tell the user what the issue was or what was fixed. There is not even a countdown timer to let the user know when it is going to finish.

It is easier to replace a corrupt database with a good one as opposed to waiting hours for a database scan to finish that tells users nothing other than what it is scanning assuming they don’t hit the cancel button to continue on with their lives.

If it takes longer than 30 minutes to run a database scan, it is taking too long.

Also, eM Client crashing doesn’t mean the database is bad. Yet eM Client assumes that is the case and runs a lengthy database check that may in fact not even be needed.

1 Like
8

There is no need to restore a backup on every start.

Please find out why the user is not closing eM Client correctly, then there is nothing else that needs to be done as eM Client will start without having to do a database check on start.

1 Like
10

I never said there was a need to restore on every start.

It is just that it takes 2-3 hours for “Checking for corrupted database…” to fully run for my client when it is triggered by some issue. I know it is important if it can fix a corrupted database if it is in fact bad. I am just saying it is easier and much faster for my client to restore a known good backup using the .bat file I made to do it with.

I also know one has to find out why the program is crashing. This is not always possible without shadowing my client waiting for this to happen if it ever does happen. The Window Event Viewer doesn’t always have the info.

One time with my client present, I did see eM Client lock up for some reason. The Windows “Program Not Responding” error popped up asking to either wait or close the program. We tried waiting which didn’t work so we had to close eM Client. Upon re-opening eM Client, the “Checking for corrupted database…” scan started. People at work cannot be waiting hours for this scan to finish. It was then I came up with the .bat file idea which has worked fine since.

I am just making a feature request to greatly speed up both “Checking for corrupted database…” and Backup (which also checks the database as it backs up) if possible. It would also be nice to have a countdown timer to know how long both of these will take to complete.

1 Like
11

Of course we do the backup and database check as quickly as possible. It just takes as long as it takes to do it properly.

You can do whatever you want though. Cancel the database check then restore a backup.

But please don’t recommend that as it could result in irreversible data loss because you are assuming that the backup is 100% safe. In some instances it may not be. Then you are really in deep water because the cancelled database repair may have rendered the database, which might have otherwise been repaired, now totally useless as well.

12

This is the first time I have ever ran eM Client’s built-in backup with the intention of trying to constantly monitoring it.

5:25 PM (00:00 elapsed) Started a backup of eM Client using its built-in backup function. There have been no database checks so assuming database is perfectly fine. 1 Microsoft Exchange account (primary) and 2 IMAP email accounts (secondary use) to back up. %APPDATA%\eM Client …

eM Client

10:40 PM (05:15 elapsed) Backup is still processing accounts. The progress bar is about 20-25% across.

12:10 AM (06:45 elapsed) No difference on progress bar.

1:10 AM (07:45 elapsed) Progress bar is almost halfway across. I was away so never saw it jump up to this.

2:40 AM (09:15 elapsed) FINALLY! The backup is compressing.

2:55 AM (09:30 elapsed) Progress bar is at ~75%.

3:30 AM (10:05 elapsed) Progress bar is at ~90%.

3:56 AM (10:31 elapsed) Backup completed.

The back is nearly 10 GBs smaller than the %APPDATA%\eM Client folder.

10:31 seems like an awful long time to me to do 55.3 GBs of email. Was there a database error that needed corrected during the backup? I have no idea as nothing was shown if there was. All that popped up was a notification window stating the backup was done. It would be nice if there was a countdown timer to let the user know how long it will take. Good thing this can be ran while eM Client is in use.

BUT… if the eM Client backup does do a database check and fixes errors, I could live with this. I could show my client how to do this once a week when leaving work for the day.

So for fun, I decided to do a restore using eM Client’s built-in restore…

3:59 AM (00:00 elapsed) Started eM Client restore using the backup file created above.

4:15 AM (00:16 elapsed) Progress bar is about halfway through.

4:29 AM (00:30 elapsed) Restore completed.

Now 30 minutes to do the restore is not bad at all with the amount of email my client has. It is also nice the restore closes eM Client and reopens it when completed. Nice work here.

So in conclusion, the backup is fine IF it is doing a database check and fixing corrupt files while backing up. I do wish it had a countdown timer and gave some details on what it was doing or what it fixed, but I guess that is not necessary if it is ran while away from work for the day. As for the "Checking for corrupted database…”, I will teach my client to cancel it, let eM Client open, and perform the restore operation. I will no longer have him use my .bat files to do this with. The whole idea with those was to let him quickly restore the database if the "Checking for corrupted database…” window appeared so he could cancel it.

13

Assuming the database was bad, backup fixed it, restoring the backup, and eM Client using a good database, I decided to run another backup today to see if it was any faster…

6:00 PM (00:00) Backup started.

6:45 PM (00:45) Progress bar is ~40% across. Much faster so far than yesterday.

7:00 PM (01:00) Progress bar is ~50% across. Still processing accounts.

7:12 PM (01:12) Backup is now compressing. This didn’t happen until 9:15 in last time. The database must have been corrupt.

8:30 PM (02:30) Backup completed!

If I had known that the reason it was taking eM Client so long to run a backup based on ~55 GBs of data was due to database corruption, I never would have complained about it. This is why it needs to show some details during the backup. 2:30 to do the backup with the same amount of data is perfectly acceptable.

Once again, I will teach my client to cancel the "Checking for corrupted database…”, let eM Client open, and perform the restore operation. I will also teach my client to do a backup once a week.

14

That is very dangerous. If you cancel the database check, you may not be able to open eM Client if the database is irreversibly corrupt. Then the user needs to manually delete the database folder before they can do a restore. And if you have not verified that the last backup is OK, you have lost everything. Please don’t recommend that.

That is good.

15

Then my client would have to wait hours and lose time for work waiting for that to complete. That is not an option in a business setting (or any setting IMO) if a recent backup exists. ~30 minutes to restore from a good backup is preferred to hours waiting for a database check/repair. Especially when it doesn’t let the user know how long it is going to take, what the problem is, what is being fixed, etc.

Isn’t that the eM Client backup’s job? To verify the backups since it apparently repairs a corrupt database as it backs up?

If the eM Client database (%APPDATA%\eM Client) gets irreversibly corrupt to where eM Client will not open, my client could:

  1. Rename the %APPDATA%\eM Client folder. Call it “eM Client.old”.

  2. Open a backup eM Client made with a .zip program like 7-Zip.

  3. Let it extract to a new %APPDATA%\eM Client folder.

  4. Open eM Client and verify it works fine.

  5. Delete the “eM Client.old” folder.

That is essentially what eM Client Restore does now. It renames %APPDATA%\eM Client to %APPDATA%\eM Client.old, extracts the .zip backup to a new %APPDATA%\eM Client folder, and deletes the.old folder prior to eM Client re-opening.

16

Please do not recommend cancelling the database check on start. If you want to do that, fine, it is your choice, but it is irresponsible to recommend that to others as a normal practice.

If the backup is not usable, and you cancel the database repair, you have lost everything. Rather let the repair complete. Then, recommend to the user that they exit eM Client correctly so that there is no need for a database repair on the next start.

17

How? You are not explaining anything. You seem to think it is OK to lose a workday so eM Client can do a database check. That is NOT OK. Does eM Client’s backup work well or not?

Please stop recommending people lose their entire workday for a database check to run that doesn’t let users know how long it is going to take, what the problem is, what is being fixed, etc.

18

Yes agree with @Gary that cancelling a database error check is "not normally a good idea"and best to find out the source of the problem with your friend or yourself “if happening to you too”.

If “you or your client haven’t already” as you both have made a backup in eM Client, I would suggest to try “removing all the IMAP and Exchange accounts and re-adding” them from scratch. Add one acct and then “once all synced” close and reopen eM Client and see if you get any errors which you normally shouldn’t.

If that then works, add the other accounts again one at a time and again close and open inbetween.

However if you still get database checking errors on startup and you have also tried uninstalling and reinstalling V9 on startup, then the problem is either in “one of those accounts”, or “some third party program installed” casing eM Client to close prematurely causing the database check.

19

Thankfully it has been a week or two since a database check has happened AFAIK. And hopefully it is rare for it to ever happen.

But, if it does happen, users need to know how long it is going to take, what the problem is, what is being fixed, etc.

The risk of losing an entire workday to this for business users is not acceptable.

Why won’t someone please explain to me why it is bad idea to cancel the database check, let eM Client open, and then immediately do a restore from a good backup made by eM Client? The eM Client backup fixes the database as it backs it up correct? So how can this possibly be a bad idea?

Instead, I get a PM threatening to ban me.

A fix that takes 30 minutes to do (restoring the backup) is vastly preferred over something that can take hours to accomplish the same thing (letting the database check run). The real world will always prefer simplicity or complexity.

20

I already gave the reason:

2 Likes
21