Backups potentially missing FBE data

[GigabyteProductions]

Some unnecessary context: I had just signed up for Project Fi service, and was trying to use Project Fi without the Project Fi app installed. However, it is necessary to use the Project Fi app to activate the SIM card and give it my phone number. Since I didn't want Play Services running on a phone with all of my personal information on it, I decided I would backup my data, erase the phone, install Play Services and Project Fi, activate SIM, erase the phone, and restore my backup.

Since I could not decrypt my phone (bug in TWRP), there were no additional encrypting sdcardfs mounts in /data, so all of /data was what resided on my phone. tetherback made a tar of that. Upon restoration attempt (gzip -d < data.ext4.win | adb -d shell tar -vC /data -x on a clean /data), I found that the tar preserved ownership and SELinux contexts, but the phone would not boot successfully unless I erased these files from /data. Upon further investigation, I found that simple things like mv-ing all the /data files to another place in /data (no cross-filesystem movement), and them mv-ing them back would also break my ability to boot.

Regarding File-Based-Encryption, Android's Website (https://source.android.com/security/encryption/file-based#direct-boot) (https://web.archive.org/web/20171025210612/https://source.android.com/security/encryption/file-based#direct-boot) says:

A complete implementation of file-based encryption on an Ext4 file system is provided in the Android Open Source Project (AOSP) and needs only be enabled on devices that meet the requirements.

My theory is that information regarding FBE became a part of the file system structure rather than being transparent on top of a generic file system like ecryptfs, and that tar is not able to preserve this information.

I understand interest in maintaining this project is waning. However, I think this is important to document.

[dlenski]

This is a great report, and seems like a really frustrating issue…

A complete implementation of file-based encryption on an Ext4 file system is provided in the Android Open Source Project (AOSP) and needs only be enabled on devices that meet the requirements.

My theory is that information regarding FBE became a part of the file system structure rather than being transparent on top of a generic file system like ecryptfs, and that tar is not able to preserve this information.

Sounds like you're onto something. Reminds me of the bad ol' days of the MacOS classic resource fork where there was structure data attached(-ish) to what appeared to be a normal file, but not readable by standard cross-platform methods. ☹️

I understand interest in maintaining this project is waning. However, I think this is important to document.

Yeah, this does seem important to warn about to prevent data loss.

Do you know of any version of tar that knows how to copy the FBE metadata? What version of Android is your device running?

[GigabyteProductions]

Sounds like you're onto something. Reminds me of the bad ol' days of the MacOS classic resource fork where there was structure data attached(-ish) to what appeared to be a normal file, but not readable by standard cross-platform methods. ☹️

Resource forks aren't necessarily bad things; they just need to be made aware of. A backup system I'm working on for work needs to be aware of resource forks, so I am aware of how iffy they can be for backup systems.

Do you know of any version of tar that knows how to copy the FBE metadata? What version of Android is your device running?

I am not aware of what is necessary to copy the supposed metadata. I'm not really sure that there is metadata, as that was just a speculation. Regarding finding a way to copy the metadata: is it at all practical or even desirable to use something like e2image to copy the filesystem itself?

I am using LineageOS 14.1, compiled from 14.1 branches for marlin. It is based off of Android 7.1.