[HOWTO] Backup and Restore Data on Android Devices

[AnotherElk]

@beck

I had TWRP on my FP2 and I guess the installation was easier back then, because now with my FP3+ I am struggling to install it.

Why do you want to install it? Just because you had it on the Fairphone 2? For your consideration …

@AnotherElk

on the Fairphone 3 you usually don’t install it, because installing it results in a changed boot partition, as there’s no recovery partition on A/B devices.
A changed boot partition breaks OTA updates when using Fairphone OS, and in case of using /e/ it will just get overwritten by /e/ updates with the LineageOS recovery (or future /e/ recovery) … so there’s not much sense in installing TWRP on the Fairphone 3, so the Fairphone 3 usually doesn’t have it in recovery mode.

Booting TWRP with fastboot works

https://forum.fairphone.com/t/used-to-the-fairphone-2-dont-do-these-fairphone-2-things-on-the-fairphone-3/57721

[beck]

Thanks for your quick reply!

@AnotherElk

Why do you want to install it?

As I did with my FP2, I´d like to use it to create backups like every three months or so, to be prepaired for eventual phone loss or destruction.

Thanks for the link, I like the diclaimer which definately excludes me
“Don’t install TWRP on the Fairphone 3 before you really know what you are doing”

I also don´t feel comfortable with the whole fastbood/adb topic, so I guess I´ll just live without backups.

[AnotherElk]

@beck

I guess I´ll just live without backups.

Possible, if you’ve got no important data on the phone ever.
But before giving up you might want to have a look at regularly syncing things like messages, contacts, photos and the whole Internal Storage to a computer, e.g. with MyPhoneExplorer. That’s not difficult … How To and FAQ.

[Luca_Bissoli]

@smu44

Instead, I think we should try to complement it with a simple solution for backing up user data.

Hi everybody,

A little contribution from another newbie on /e/.

I think Smu44 touched a relly sensible keypoint. An easy user procedure for Data backup that allows to put your data wherever you want (cloud, pc,usb, etc) is one of the feature that make worthable the choice.

It could be spendable as well from a marketing point of view, remarking the security framework for degoogled OS.

I think a programming task than can be rewarded a lot in term of reputation.

[ljahn]

I think the devs are working on including seedvault into /e/ (see these issues on the gitlab).
This does full backups of all application data.
@Manoj any status updates?

[AnotherElk]

@ljahn

This does full backups of all application data.

It doesn’t, apparently.

@harvey186

The app is able to backup most apks and the data, but nit from all apps. But it works only on clean fresh install

@Ingo_FP_Angel

And to my knowledge it will not be possible to backup “everything”. App developers can specify if their app (data? not sure) can be backed up. When I tried it out some of the apps couldn’t be backed up indeed (sorry, don’t remember which).
But on the other hand, the homescreen was preserved and also several settings I checked (like volume/brightness level etc.)

My conclusion is: it’s as good as it gets for a “normal” user (i.e. when root or TWRP backup are not an option). Even if it’s not perfect.

[ljahn]

@AnotherElk

It doesn’t, apparently

Oh, that’s bad.
Do you know of any list of apps that work with seedvault?
I’d assume most apps from f-droid should work, because that’s Calyx OS’ main app store. This would still cover (hopefully ) most apps you use on your /e/ phone.

[smu44]

As this thread is an HowTo for TWRP, discussing additional software may be out of topic

In the HowTo I tried to provide an MTP solution (within TWRP), but it may not work on some devices.

One idea could be to provide users with a TWRP-installable zip, containing all necessary stuff (binaries, script) to help user backup their data as a complement to TWRP’s.

[sonyxa2]

Thank you for your feedback.
Personally, I would like to have a tool that is newbie-proof… you hit a button, you get the backup, and after the update of the OS, you restore, and that’s it … I would not mind if it is something to be done with adb, but I would really like one command only, without having to be afraid of having lost something or not to be able to recover…

[Grendel]

Thanks @smu44 for pointing out this problem with Shelter. On my OnePlus 5T I had it installed with Magisk after an upgrade to the Pie build. Under Oreo before I didn’t have this problem and Nandroid would backup with no problem but it appears the multiple user accounts feature frequently blocks TWRP backups which end with “error 255”. I had made the mistake of installing Shelter but when I realised this was the cause and so uninstalled it, the blocked Nandroid backup was still there. I tried several of the suggested solutions to no avail until I came across a tip from user Pannal on the github site of TWRP. He has created a flashable file - RMDAD_v0.0.1.zip which removes the instances of multiple accounts and once I had flashed it my codeworkx version of TWRP worked perfectly. It was only the /data partition that caused the problem as the others backed up with no problem.
I have since tested my Nandroid backup of /boot, /system & /data by doing a restore and it worked perfectly to my great relief, as I now have a reliable backup method. The presence of my Magisk mod to get root doesn’t seem to affect anything, which is just as well since I use Titanium Backup to safeguard my user installed apps and as you probably know it only works on a rooted device.
I hope my experience will be useful for those running into this error 255 problem as it seems to exist on other devices as well. Lesson learned: Don’t install Shelter or use multiple accounts!

[sonyxa2]

Thank you very much for sharing this hint. It seems very interesting to me …
I had not checked that before, but perhaps it could be useful (and easy) to have the backup of the data …

Any feedback from users?

Thank you in advance

[smu44]

Thanks @Grendel for your feedback

I’ll try again on my TWRP 3.4 devices with Shelter, last time the error didn’t show …
But I’ve seen it in the past, with Shelter and/or Magisk.

The data partition problem with multi-user enabled makes sense, but it could have been fixed in latest TWRP.

So we could assume that this error could be related to :

• device
• ROM
• TWRP device-specific build
• TWRP version
• Shelter and/or Magisk installed

Too much criteria to diagnose

I agree with your conclusion : better stay away from Shelter, and I’ll add : maybe from Magisk too !

[smu44]

Or, on some other hands : if your TWRP backup fails with error #255, remove additional profile and Shelter, and retry !

[smu44]

After some tests on 2 devices (Lenovo YogaTab 3 and Xiaomi Redmi 4X), I come back to propose some adjustments. I could restore from a corrupted ROM image using this method.

I’ll be very grateful if somebody with a testing device or similar experience could confirm, so we may evolve the HowTo

1/ Restore

• Unless necessary, only restore the specified partitions, even if more are included in backup. I tried to restore others (EFS, persistent, recovery, images), every time my device rebooted abruptly in fastboot and eventually I had to reinstall TWRP.
• It is safer to restore /data/media before first system reboot (see below for backup). If you used a tgz archive, something like cd / then tar -xzvf /path/to/backup/storage/data_media.tgz. Also don’t forget to check owners and use chown -R user:user * if necessary.
• Before first reboot to system, you may want to remove /data/system/locksettings.db (or similar). If your device was protected by pattern/pin/password, you may not be able to unlock while rebooted in system.
• Proposed sequence :
  • reboot to TWRP with backup files available
  • format data
  • restore boot/system/data
  • restore data/media from hand-made archive
  • delete /data/system/locksettings.db (or similar)
  • reboot system (Android may encrypt at first boot)
  • set lock (pattern/pin/password) again, and optionally activate Secure Boot
  • reboot again so system

2/ Backup

• As a matter of precaution, I strongly suggest to remove any lock in Security before backup (don’t forget to reboot to system to check). You will lose your fingerprints records.
• If your phone has secure Boot activated (prompting at boot with black background), it may persist even after removing lock (it happened to me …). You can try the following :
  • set lock to pattern, activate Secure Boot, reboot system
  • set lock to swipe, reboot system
  • set lock to none, reboot system
• Boot to system or TWRP should not require any code/pattern/password, Files app should show your files.
• As suggested in previous posts, uninstall Magisk and Shelter.
• I also advise to make a gzipped tar backup of /data/media while in TWRP :
  • do the TWRP backup, do not reboot
  • from an adb-enabled computer, launch an adb shell (as always, adb devices first, if no device shown you may have to play with MTP in TWRP Mount)
  • Alternatively, you can use the TWRP Terminal.
  • identify a target storage (for example, you can use the mount command to find path to your SDCard)
  • launch cd / then tar -czvf /path/to/backup/storage/data_media.tgz . (note : some TWRP may have zip, bzip2, GNU tar, … I’ll stick on ToyBox/BusyBox tar as it seems to be widely available)
  • also record owners using ls -l /data/media > /path/to/backup/storage/data_media_owners.txt
• On my Lenovo YT3, TWRP backup to SDCard never succeeded (stuck somewhere in middle of nowhere). I could use an USB OTG, but it was much slower as internal storage.
• In any case, take the time to copy your backups to another storage (PC). You may have to move files to /data/media/... if using internal storage, before rebooting to system.
• If enough space is available, feel free to backup more partitions. But be aware that only some can be restored without using any “magic”.
• Have the TWRP image file on hand, just in case …

[manjar]

The latest version of TWRP is:
twrp-3.5.0_9-0-FP3.img

[petefoth]

I currently use TWRP for backing up my /e/ devices, when of upgrading from one /e/ build to the next. I usually start with a ‘dirty flash’ - installing the new ROM without wiping the existing data used by the old ROM. If that doesn’t work, I wipe the data partition and do a clean install, then restore my apps and data from the TWRP backup. This works well, and has rescued me a couple of times when I flashed my device into a non-functioning state

It’s not so good for migrating from one device to a different device - same or different model. TWRP backups seem to be tied to a specific device. (Even if that’s not the case, I have never managed to restore data from one device to another, even of the same model).

So I have been looking at a project called Android Backup and Restore Tools

[TOOLS][ZIPS][SCRIPTS]Android Backup and Restore Tools[Multiple...

More information in the projects README.md on github

AndDiSa/android_backup_project

It doesn’t need a rooted device, but rooted debugging must be enabled in 'Developer Options`.

It works pretty well: I have been able to backup installed apps and data, and restore to

• the same device;
• different device of the same model;
• different model, running a ‘same version’ /e/ ROM

I haven’t tried, but I think it would work, to restore to a device running a newer (or older) version. It also works well at restoring a single app, with or without its data, which is a big improvement over TWRP backups.

Unfortunately, it currently seems to have one main weakness: it does not backup account information for apps where that information is handled by the Account Manager app (i.e. it’s listed in Settings | Accounts). I tried backing up using --system-apps, which does appear to backup Account Manager, and its data, but when I restored that app, there were still no accounts listed. The apps affected for me were :

• Twidere (Twitter client)
• NextCloud / e/Cloud 0- this is a biggie - I keep my contacts, calendar and notes in NextCloud, so I had to re-enter all account details in the NextCloud app and the WebDav accounts before my I could see my notes, contacts and calendars. (The Calendar app was populated, but the events were ‘device only’ and there was no synching).
• Ebay
• BBC Sounds
• Nationwide (Banking app)
• Bandcamp
• XDA Developers
• MyEG (England Golf app)
• Ring (Smart Doorbell app)
• Evernote
• OS Maps (did not work after restore and had to be re-installed. Stored offline maps were not restored and had to be downloaded again)
• OSMAnd (no login account is needed, but stored offline maps were not restored and had to be downloaded again)

Other apps which store their own account data did work OK after restore

• K-9 Mail
• Messenger
• WhatsApp
• Telegram
• igMember (Intelligent Golf client)
• Scottish Golf

The NHS Covid-19 (UK tracking app) did not work after restore and had to be re-installed.

All of that is a pretty big failing for my use cases, so I will carry on with TWRP backups until SeedVault migrates into /e/ user builds from the test builds. Roll on v0.15-q

Until then, this tool may be useful for some people’s use cases .

I didn’t fully test the backup / restore_apps --system-apps functionality, and I only quickly played with the full-backup/restore, which seemed to backup OK, but restoring - full or partial - either did nothing or stopped the device from booting There’s a good chance that was user error.

PS I know that Shelter can cause problems with TWRP backups. None of the devices I worked with used Shelter, so I have no idea whether this tool would work if Shelter` is installed and used.

[anil]

Just a note, on a A/B device such as the FairPhone 3, you don’t want to actually install TWRP as then your device won’t boot once you lock the bootloader:

FP3 latest update corrupt, doesnt work

Quite a number of users here have a Fairphone 3, so perhaps you should let us handle the Fairphone 3 stuff . Yes, TWRP is available for the Fairphone 3, I use it myself. But on the Fairphone 3 you usually don’t install it, because installing it results in a changed boot partition, as there’s no recovery partition on A/B devices. A changed boot partition breaks OTA updates when using Fairphone OS, and in case of using /e/ it will just get overwritten by /e/ updates with the LineageOS r…

So, every time you want to make a backup, you need to run fastboot boot twrp.img and directly launch it from your computer. (sigh)

[AnotherElk]

@anil

Just a note, on a A/B device such as the FairPhone 3, you don’t want to actually install TWRP as then your device won’t boot once you lock the bootloader:

@anil

So, every time you want to make a backup, you need to run adb boot twrp.img and directly launch it from your computer. (sigh)

It’s fastboot boot twrp.img .
And if you lock the bootloader, you can’t boot TWRP this way anyway.

[anil]

Sorry for the typo, s/adb/fastboot/
Edit: Don’t do this it causes your data to be deleted
Yes, my current workaround is:

1. unlock the bootloader
2. run fastboot boot twrp.img
3. Make a backup
4. lock the bootloader
5. reboot back into the OS

It’s an unfortunate number of steps

[AnotherElk]

@anil

unlock the bootloader

@anil

lock the bootloader

Both of these steps force a factory reset for security reasons.

So you are only doing a TWRP backup/ restore of the state of the OS without your data and Apps, and you have to somehow bring your data and Apps back after every TWRP use once the OS is booted again?