Offlineimap not syncing all folders/messages

[benjaminpreiss]

General informations

• system/distribution (with version): Debian GNU/Linux 10
• offlineimap version (offlineimap -V): offlineimap v7.2.3
• Python version: Python v2.7.16
• server name or domain: -
• CLI options nr1: -a Account1,Account2,Account3 -c offlineimap_from_server1.conf
• CLI options nr2: -a Account1,Account2,Account3 -c offlineimap_to_server2.conf

Configuration file offlineimaprc

nr1: configuration server1 -> local maildir
nr2: configuration local maildir -> server2

pythonfile (if any)

NO pythonfile

Step 1: Clean Slate

rm -r ~/.offlineimap/Account-Account1 ~/.offlineimap/Repository-RemoteAccount1 ~/.offlineimap/Repository-LocalAccount1 ~/.offlineimap/Account-Account2 ~/.offlineimap/Repository-RemoteAccount2 ~/.offlineimap/Repository-LocalAccount2 ~/.offlineimap/Account-Account3 ~/.offlineimap/Repository-RemoteAccount3 ~/.offlineimap/Repository-LocalAccount3

rm -r ~/linux-files-wsl/Account1 ~/linux-files-wsl/Account2 ~/linux-files-wsl/Account3

Step 2: Info Server1 -> local maildir

offlineimap -a Account1,Account2,Account3 -c offlineimap_from_server1.conf --info

Logs

Step 3: Dry Run Server1 -> local maildir

offlineimap -a Account1,Account2,Account3 -c offlineimap_from_server1.conf --dry-run

Logs

Step 4: Real Run Server1 -> local maildir

offlineimap -a Account1,Account2,Account3 -c offlineimap_from_server1.conf

Logs

Step 5: Info local Maildir -> Server2

offlineimap -a Account1,Account2,Account3 -c offlineimap_to_server2.conf --info

Logs

Step 6: Dry Run local Maildir -> Server2

offlineimap -a Account1,Account2,Account3 -c offlineimap_to_server2.conf --dry-run

Logs

Step 7: Clean offline imap cache (because of UID validity problems)

rm -r ~/.offlineimap/Account-Account1 ~/.offlineimap/Repository-RemoteAccount1 ~/.offlineimap/Repository-LocalAccount1 ~/.offlineimap/Account-Account2 ~/.offlineimap/Repository-RemoteAccount2 ~/.offlineimap/Repository-LocalAccount2 ~/.offlineimap/Account-Account3 ~/.offlineimap/Repository-RemoteAccount3 ~/.offlineimap/Repository-LocalAccount3

Step 8: Real Run local Maildir -> Server2

offlineimap -a Account1,Account2,Account3 -c offlineimap_to_server2.conf

Logs

Problemdescription

Sadly, some messages are simply not transferred to server 2.
E.g. RemoteAccount3:INBOX UID 3 and 4 are missing.

The same happens for all messages in the following folders:

• LocalAccount3 Sent Messages
• LocalAccount3 Deleted Messages
• LocalAccount3 Drafts

I suspect this has something to do with my nametrans rules... But I am not quite sure because it works mostly.

[ssclift]

I hit a similar problem transferring mail to a new server, but I think it was old data in the meta-files. I've not traced this completely, I suspect it has to do with with the way meta-data is stored by account name. I cleaned out the meta-data and it worked. That's not really a solution for everyday use, however.

[benjaminpreiss]

@ssclift Thanks mate, how do you clean out the meta-data?

[ssclift]

There's a *.meta directory. However, I don't really understand what I did there, it just worked...

I suspect if your configuration for server1->local and the configuration for local->server2 use completely different account names in the [Accounts] section of the offlineimap.conf, it might do the trick.

Removing the meta-data is a messy solution... I was in a hurry. Again, it was just a guess without really understanding the guts.

[ssclift]

Ah ha... I've just had to set up similarly to you after switching email servers.

I used separate meta-data directories, and that seems to work.

[general]
accounts = ...
metadata = ~/mailxfer/fetch_old.meta

then another config file with

[general]
accounts = ...
metadata = ~/mailxfer/update_new.meta

Works for me, your mileage may vary.

[benjaminpreiss]

Wow, thank you so much! Will try and get back to you!

[benjaminpreiss]

@ssclift Sadly, it didn't work :( Same folders and mails missing, exactly the same output as before...

I wonder whether directly syncing from imap to imap would do the trick?

Also, I read in the docs that nametrans mapping has to be unique, that means merging folders should not be possible?

[nicolas33]

Yes, merging folders is not possible. There's a one-to-one mapping between local and remote with the UIDs for the emails for each mailbox.

[nicolas33]

What you're attempting to do should be possible. HOWEVER, be sure to not mix the cache data between both syncs setups. Either use different conf file files with uniq metadatadir OR have one config file and two different accounts having uniq remote and local repository sections each.