[HOWTO] Backup and Restore Data on Android Devices

eOSWiki HOWTOs
,
31

Thank you very much for sharing this hint. It seems very interesting to me …
I had not checked that before, but perhaps it could be useful (and easy) to have the backup of the data …

Any feedback from users?

Thank you in advance

32

Thanks @Grendel for your feedback :slight_smile:

I’ll try again on my TWRP 3.4 devices with Shelter, last time the error didn’t show …
But I’ve seen it in the past, with Shelter and/or Magisk.

The data partition problem with multi-user enabled makes sense, but it could have been fixed in latest TWRP.

So we could assume that this error could be related to :

  • device
  • ROM
  • TWRP device-specific build
  • TWRP version
  • Shelter and/or Magisk installed

Too much criteria to diagnose :frowning:

I agree with your conclusion : better stay away from Shelter, and I’ll add : maybe from Magisk too !

33

Or, on some other hands : if your TWRP backup fails with error #255, remove additional profile and Shelter, and retry ! :smiley:

34

After some tests on 2 devices (Lenovo YogaTab 3 and Xiaomi Redmi 4X), I come back to propose some adjustments. I could restore from a corrupted ROM image using this method.

I’ll be very grateful if somebody with a testing device or similar experience could confirm, so we may evolve the HowTo :slight_smile:

1/ Restore

  • Unless necessary, only restore the specified partitions, even if more are included in backup. I tried to restore others (EFS, persistent, recovery, images), every time my device rebooted abruptly in fastboot and eventually I had to reinstall TWRP.
  • It is safer to restore /data/media before first system reboot (see below for backup). If you used a tgz archive, something like cd / then tar -xzvf /path/to/backup/storage/data_media.tgz. Also don’t forget to check owners and use chown -R user:user * if necessary.
  • Before first reboot to system, you may want to remove /data/system/locksettings.db (or similar). If your device was protected by pattern/pin/password, you may not be able to unlock while rebooted in system.
  • Proposed sequence :
    • reboot to TWRP with backup files available
    • format data
    • restore boot/system/data
    • restore data/media from hand-made archive
    • delete /data/system/locksettings.db (or similar)
    • reboot system (Android may encrypt at first boot)
    • set lock (pattern/pin/password) again, and optionally activate Secure Boot
    • reboot again so system

2/ Backup

  • As a matter of precaution, I strongly suggest to remove any lock in Security before backup (don’t forget to reboot to system to check). You will lose your fingerprints records.
  • If your phone has secure Boot activated (prompting at boot with black background), it may persist even after removing lock (it happened to me …). You can try the following :
    • set lock to pattern, activate Secure Boot, reboot system
    • set lock to swipe, reboot system
    • set lock to none, reboot system
  • Boot to system or TWRP should not require any code/pattern/password, Files app should show your files.
  • As suggested in previous posts, uninstall Magisk and Shelter.
  • I also advise to make a gzipped tar backup of /data/media while in TWRP :
    • do the TWRP backup, do not reboot
    • from an adb-enabled computer, launch an adb shell (as always, adb devices first, if no device shown you may have to play with MTP in TWRP Mount)
    • Alternatively, you can use the TWRP Terminal.
    • identify a target storage (for example, you can use the mount command to find path to your SDCard)
    • launch cd / then tar -czvf /path/to/backup/storage/data_media.tgz . (note : some TWRP may have zip, bzip2, GNU tar, … I’ll stick on ToyBox/BusyBox tar as it seems to be widely available)
    • also record owners using ls -l /data/media > /path/to/backup/storage/data_media_owners.txt
  • On my Lenovo YT3, TWRP backup to SDCard never succeeded (stuck somewhere in middle of nowhere). I could use an USB OTG, but it was much slower as internal storage.
  • In any case, take the time to copy your backups to another storage (PC). You may have to move files to /data/media/... if using internal storage, before rebooting to system.
  • If enough space is available, feel free to backup more partitions. But be aware that only some can be restored without using any “magic”.
  • Have the TWRP image file on hand, just in case …
1 Like
35

The latest version of TWRP is:
twrp-3.5.0_9-0-FP3.img

36

I currently use TWRP for backing up my /e/ devices, when of upgrading from one /e/ build to the next. I usually start with a ‘dirty flash’ - installing the new ROM without wiping the existing data used by the old ROM. If that doesn’t work, I wipe the data partition and do a clean install, then restore my apps and data from the TWRP backup. This works well, and has rescued me a couple of times when I flashed my device into a non-functioning state :slight_smile:

It’s not so good for migrating from one device to a different device - same or different model. TWRP backups seem to be tied to a specific device. (Even if that’s not the case, I have never managed to restore data from one device to another, even of the same model).

So I have been looking at a project called Android Backup and Restore Tools

More information in the projects README.md on github

It doesn’t need a rooted device, but rooted debugging must be enabled in 'Developer Options`.

It works pretty well: I have been able to backup installed apps and data, and restore to

  • the same device;
  • different device of the same model;
  • different model, running a ‘same version’ /e/ ROM

I haven’t tried, but I think it would work, to restore to a device running a newer (or older) version. It also works well at restoring a single app, with or without its data, which is a big improvement over TWRP backups.

Unfortunately, it currently seems to have one main weakness: it does not backup account information for apps where that information is handled by the Account Manager app (i.e. it’s listed in Settings | Accounts). I tried backing up using --system-apps, which does appear to backup Account Manager, and its data, but when I restored that app, there were still no accounts listed. The apps affected for me were :

  • Twidere (Twitter client)
  • NextCloud / e/Cloud 0- this is a biggie - I keep my contacts, calendar and notes in NextCloud, so I had to re-enter all account details in the NextCloud app and the WebDav accounts before my I could see my notes, contacts and calendars. (The Calendar app was populated, but the events were ‘device only’ and there was no synching).
  • Ebay
  • BBC Sounds
  • Nationwide (Banking app)
  • Bandcamp
  • XDA Developers
  • MyEG (England Golf app)
  • Ring (Smart Doorbell app)
  • Evernote
  • OS Maps (did not work after restore and had to be re-installed. Stored offline maps were not restored and had to be downloaded again)
  • OSMAnd (no login account is needed, but stored offline maps were not restored and had to be downloaded again)

Other apps which store their own account data did work OK after restore

  • K-9 Mail
  • Messenger
  • WhatsApp
  • Telegram
  • igMember (Intelligent Golf client)
  • Scottish Golf

The NHS Covid-19 (UK tracking app) did not work after restore and had to be re-installed.

All of that is a pretty big failing for my use cases, so I will carry on with TWRP backups until SeedVault migrates into /e/ user builds from the test builds. Roll on v0.15-q :slight_smile:

Until then, this tool may be useful for some people’s use cases .

I didn’t fully test the backup / restore_apps --system-apps functionality, and I only quickly played with the full-backup/restore, which seemed to backup OK, but restoring - full or partial - either did nothing or stopped the device from booting :slight_smile: There’s a good chance that was user error.

PS I know that Shelter can cause problems with TWRP backups. None of the devices I worked with used Shelter, so I have no idea whether this tool would work if Shelter` is installed and used.

9 Likes
37

Just a note, on a A/B device such as the FairPhone 3, you don’t want to actually install TWRP as then your device won’t boot once you lock the bootloader:

So, every time you want to make a backup, you need to run fastboot boot twrp.img and directly launch it from your computer. (sigh)

1 Like
38

It’s fastboot boot twrp.img :wink: .
And if you lock the bootloader, you can’t boot TWRP this way anyway.

2 Likes
39

Sorry for the typo, s/adb/fastboot/
Edit: Don’t do this it causes your data to be deleted
Yes, my current workaround is:

  1. unlock the bootloader
  2. run fastboot boot twrp.img
  3. Make a backup
  4. lock the bootloader
  5. reboot back into the OS

It’s an unfortunate number of steps :frowning:

40

Both of these steps force a factory reset for security reasons.

So you are only doing a TWRP backup/ restore of the state of the OS without your data and Apps, and you have to somehow bring your data and Apps back after every TWRP use once the OS is booted again?

41

:cry emoji: well that explains why my data was disappearing LOL - I thought it was a continuation of my SD card woes and was continuing to debug it from that angle

42

Should be tar -czvf /path/to/backup/storage/data_media.tgz /data/media/

43

Can this news (probably for me only) have in impact on user backup easy usability?

1 Like
44

It’s coming

Though it looks like it’s may not be in v0.15. It’ll arrive one day :slight_smile:

4 Likes
45

Hello.

Quick question: with two ‘sistership’ devices (Moto E 2015), same TWRP and /e/OS (last official, both /e/ from OTA update) can I hope clone one to the other, just restoring n°1’s backup in n°2 ? 2 is a few month older… but has a good battery.

Of corse, I will save first ‘ordinary’ datas (photos, .vcf and so on) on PC, remove SDcards and SIMs and use a dedicated SDcard as ‘external storage’ for this operation.

Thanks for your advice.

2 Likes
46

I believe that TWRP backups are tied to a single device, and cannot be restored to a different device. Backups on SD Card can be found in a TWRP directory, with a sub-directory named by the Serial number of the device (e.g BHS9898XX4F). So you could try copying n°1’s backup from the n°1’s serial number directory to the n°2’s serial number directory.

I would be interested to know if that works if you are able to try it.

3 Likes
47

OK thanks.

Will try and share…later (work)

48

I’ll try to :

  • manually install all apps on the source phone, to the target phone.
  • extract all /data apps directories from TWRP backup
  • restore them on target phone

I don’t think a whole restore of /data will work, as it may contains device-specific data.
But it still worth a try :wink:

1 Like
49

Works :sunglasses: I’m with the clone right now. Have to check details but seems to be OK…

  • backuped first the ‘clone’ > has got his TWRP folder and ‘serial numbered’ subfolder,
  • backuped the ‘master’,
  • dragged its backup to PC then droped into clone’s TWRP subfolder,
  • no problem with Restore nor Reboot…

Note that clone’s TWRP went on first page (‘Read only’): due to wipe (cache + dalvik) during backup ?

3 Likes
50

Yeah, I’ve done that before on my LG G3 D851. I’ve been through a few of those devices. After copying TWRP backups to a new ‘serial numbered’ subfolder all restores and bootups worked fine for me also.

As long as two devices are of the exact same variant, sharing backups should be no problem.

2 Likes