[HOWTO] Backup and Restore Data on Android Devices

[anil]

@anil

Just a note, on a A/B device such as the FairPhone 3, you don’t want to actually install TWRP as then your device won’t boot once you lock the bootloader:

:cry emoji: well that explains why my data was disappearing LOL - I thought it was a continuation of my SD card woes and was continuing to debug it from that angle

[smu44]

@smu44

tar -czvf /path/to/backup/storage/data_media.tgz

Should be tar -czvf /path/to/backup/storage/data_media.tgz /data/media/

[Luca_Bissoli]

Can this news (probably for me only) have in impact on user backup easy usability?

LineageOS adopts SeedVault as its open source backup solution

[petefoth]

It’s coming

@petefoth

All of that is a pretty big failing for my use cases, so I will carry on with TWRP backups until SeedVault migrates into /e/ user builds from the test builds . Roll on v0.15-q

Though it looks like it’s may not be in v0.15. It’ll arrive one day

[trefix]

Hello.

Quick question: with two ‘sistership’ devices (Moto E 2015), same TWRP and /e/OS (last official, both /e/ from OTA update) can I hope clone one to the other, just restoring n°1’s backup in n°2 ? 2 is a few month older… but has a good battery.

Of corse, I will save first ‘ordinary’ datas (photos, .vcf and so on) on PC, remove SDcards and SIMs and use a dedicated SDcard as ‘external storage’ for this operation.

Thanks for your advice.

[petefoth]

I believe that TWRP backups are tied to a single device, and cannot be restored to a different device. Backups on SD Card can be found in a TWRP directory, with a sub-directory named by the Serial number of the device (e.g BHS9898XX4F). So you could try copying n°1’s backup from the n°1’s serial number directory to the n°2’s serial number directory.

I would be interested to know if that works if you are able to try it.

[trefix]

OK thanks.

Will try and share…later (work)

[smu44]

I’ll try to :

• manually install all apps on the source phone, to the target phone.
• extract all /data apps directories from TWRP backup
• restore them on target phone

I don’t think a whole restore of /data will work, as it may contains device-specific data.
But it still worth a try

[trefix]

Works I’m with the clone right now. Have to check details but seems to be OK…

• backuped first the ‘clone’ > has got his TWRP folder and ‘serial numbered’ subfolder,
• backuped the ‘master’,
• dragged its backup to PC then droped into clone’s TWRP subfolder,
• no problem with Restore nor Reboot…

Note that clone’s TWRP went on first page (‘Read only’): due to wipe (cache + dalvik) during backup ?

[marcdw]

Yeah, I’ve done that before on my LG G3 D851. I’ve been through a few of those devices. After copying TWRP backups to a new ‘serial numbered’ subfolder all restores and bootups worked fine for me also.

As long as two devices are of the exact same variant, sharing backups should be no problem.

[smu44]

If anyone is facing error #255 while backup, it’s worth a try to examine recovery.log in TWRP’s backup folder
In my case, it failed on an empty /data/per_boot dir, and succeeded after removing this dir

[petefoth]

I have had another look at the ‘Android Backup and Restore Tools’ project which uses shell scripts on a Linux PC to backup and restore apps and data. I looked at it a while ago, but then I started using TWRP backup / restore instead. I decided to revisit this project as a possible alternative to TWRP for backup and restore, to see if it can offer a more flexible and fine-grained backup and restore, specifically for using a known set of user apps and data on different devices and phone models.

After a couple of days experimenting, it is looking pretty good. It will - with some exceptions - restore the same set of apps and data across different phone models. I have only tried devices running the same /e/ and Android version: using it across /e/ and /Android version updates is the next experiment.

It still does not seem to restore accounts, even if you restore the Account Manager app and data, so to use it on a fresh install you need to boot the system, complete the first time setup wizard (including adding an /e/ account), then enable adb root debugging, before you can restore the data. Have fun!

After a bit more experimenting:

• restore seems to be architecture dependent: I started with a Sony Xperia XZ1 Compact lilac. Apps backed up from there mostly restore fine onto a Z5 Compact suzuran (which was the previous Xperia generation). I then tried on Z3 and Z3 Compact, which were an even earlier generation, and many apps began to fail, when installing apps with multiple apk files in the archive (e.g. app_com.DibsemSoftware.HowManyStrokes.tar.gz, contains the following split apks: split_config.en.apk split_config.arm64_v8a.apk base.apk split_config.xhdpi.apk ). Restoring that archive fails with the error Failure [INSTALL_FAILED_NO_MATCHING_ABIS: Failed to extract native libraries, res=-113. These archives restored OK on Z5 Compact, but not on Z3/Z3C.

• All phones involved were running e-0.19-q-20211029

• So backups from one model may be restorable on some other models, but not all

• ‘failed’ apps in this scenario include: Facebook Messenger, Telegram, Cluster, HowManyStrokes, Ebay, Evernote, Garmin Connect & Golf

• these can sometimes be fixed, by replacing the split apks in the archive with a single APK, where one is available (e.g. for F-Droid apps)

• Some restored apps keep stopping when run. These can be fixed by deleting app storage, meaning the apps can be restored, but the user data is lost

I’ll add more information here as I discover it.

[chrisrg]

I gave this a first trial to backup user apps from one device to another and yes, it’s looking quite good, as you say.

I had a little bit of trouble understanding how to get things running at the start, but most of my questions were solved by reading the xda thread.

Had to install pv (sudo apt install pv) on my ubuntu pc for the script to run as intended.

Backing up on Essential PH-1 (mata) to Xperia XA2 (pioneer). Both devices with usb debugging and rooted debugging enabled; and running e-0.19-r-20211030.

The backup script backed up about 17 apps and their data putting them into a folder it created.

I tried the restore function with only one app and it’s data in a folder to start with, Aurora store, and it worked well. The app opened and ran a session straight off. I was quite suprised with that!

I’ll experiment more with this tomorrow; thanks for the heads-up.

[chrisrg]

There seems to be a live edit option for your post!

Experimented with restoring more apps today (mata to pioneer) and have to say this seems to be a great tool.

K9 all set up a working, Nekogram all good; accounts added, Bromite complete with bookmarks and passwords- that was a surprise! Fdroid with the additional repos I had and a couple of other apps.

Normally when I do any sort of backup it’s a tedious manual process, but manageable for the small amount of apps I use; but I can see this method easing the process considerably. Transferring phone history is one thing I would like to be able to do this way but I haven’t managed so far. I took another backup_apps with system (which created a much larger folder of 1.2GB) just to get the phone history.

From the backup folder I selected the app and data for com.android.dialer and the same for message as well. After another restore message was all present and correct, but no history in the dialler. I’m keeping logs at the moment but I can’t understand a lot of the content.

[petefoth]

@chrisrg

Transferring phone history is one thing I would like to be able to do this way but I haven’t managed so far.

I backup call history (along with SMS Messages) using the 'SMS Backup and Restore` app from Aurora Store. I have it set to do daily backups to a local SD card. So if I need it on a fresh install I just install the app (or restore it with this tool ) then restore the backed up messages and call logs.

@chrisrg

I took another backup_apps with system (which created a much larger folder of 1.2GB) just to get the phone history.

I have had a bit of a play with the --system backup, but not had much joy with it. I wanted to get accounts backed up by backing up and restoring the `Account manager’, but the accounts don’t get backed up so I’ve given up on it.

Also my main use case for this tool is to have a common set of user apps and data which I can use for testing with different phone models, as I am official maintainer for two devices, and I also test for a third official device which I happen to own, and another unofficial device. I had real problems getting backed up system apps from one device restored on a different model, so I gave up because that use case isn’t so important to me.

The other important use case is to back up my ‘daily driver’ and my wife’s (which runs LineageOS with GApps), as a “belt and braces” strategy in addition to backing up with TWRP. Again, I don’t really need system apps and their data for that. Worst case is I have to input my /e/ and OwnDrive NextCloud accounts, because they are held in Account Manager. I have a backup of my email account settings from K-9 Mail, and all my other data (Contacts, Calendar, Notes) lives in various clouds.

So good luck with trying to get what you want out of --system, but for me the small amount of stuff held there isn’t worth the extra hassle. I’ve now got a set of user apps and data that I can flash to a freshly installed device in a single operation which take 10-15 minutes, and which - unlike TWRP - can be used to restore just a single app at a time. So I am a happy ROM maintainer and /e/ user

[chrisrg]

Thanks Pete, learn something new all the time. I’d not cottoned on to the export settings capability of K9 lol, 2 seconds and you’re sorted. What a great app.

SMS Backup and Restore is my ‘normal’ route although I don’t like it’s tracker count; but I guess I’ll stick with it.

@petefoth

So good luck with trying to get what you want out of --system

Yup, I’ll be following your lead on that; probably won’t bother with it. It’s good to try things out though and see what you can get with the tools available. I don’t use twrp much as it doesn’t work well on android 11 and the devices I have. This’ll be good for helping set up devices at testing time though and I’ll certainly be using it next time I need to restore my daily driver (mata)

I’ve recently acquired an xz2c. Wanted one for a long time. It’s a great little phone. I think that’s one of your’s, no?

[petefoth]

@chrisrg

Thanks Pete, learn something new all the time. I’d not cottoned on to the export settings capability of K9 lol, 2 seconds and you’re sorted. What a great app.

Agreed. I’ve been using it for many years an wouldn’t be without it.

@chrisrg

I’ve recently acquired an xz2c. Wanted one for a long time. It’s a great little phone. I think that’s one of your’s, no?

No. My daily driver is the XZ1C, for which I make unofficial and custom builds. When I was looking for a replacement for the Z5C Iused previously, I looked at the XZ2C, but rejected it because it’s bigger, it has no headphone jack socket, and - at the time - it was significantly more expensive.

[marcdw]

Alrighty folks, I have some good news.
For backing up SMS/MMS I have been using SMS Backup & Restore (Pro) for years. Has never failed plus there really hasn’t been anything that comes close.

For some time, on F-Droid there has been an app called SMS Import/Export that uses JSON file format. Ignored because it didn’t backup MMS. With the latest version that has changed. MMS is now supported.

Test drive on my daily driver Nougat setup (not /e/OS).
First, backed up using SMS Backup & Restore (just in case). About 1100 messages.
Then exported messages with SMS I/E. No errors.
Deleted all messages and rebooted for good measure.
Set SMS I/E as the default SMS app. Imported the messages. No errors.
Reset my default SMS app and reboot.

All messages, media, proper order/threading was intact. It went perfect.
When exporting you can choose where to save.
Now folks have a usable FLOSS alternative. Sure, it doesn’t have all the extra features but for most, backing up and restoring is all that matters.

SMS Import / Export | F-Droid

[petefoth]

@marcdw

For backing up SMS/MMS I have been using SMS Backup & Restore (Pro) for years. Has never failed plus there really hasn’t been anything that comes close.

For some time, on F-Droid there has been an app called SMS Import/Export that uses JSON file format. Ignored because it didn’t backup MMS. With the latest version that has changed. MMS is now supported.

Does SMS Import/Export support scheduled backups? That is mainly how I use SMS Backup and Restore, to ensure I always have an up-to-date backup (without having to remember to run a backup before I flash a new ROM version)

[marcdw]

No, not yet. I went browsing around the issue tracker and the topic of scheduling has come up. I believe the dev is looking into it.
Read a post from the dev where he didn’t think anyone would be using the app. Something about being a proof-of-concept and learning thing. Folks requested MMS support so he obliged. Hopefully he can add scheduling or auto backups. An issue for that was opened just a day ago.

Issue # 7 ([feature] Periodic backup support) at tmo1/sms-ie