Backups, bootloader, root

As the second /e/ build for the Fairphone 3 is out, I took the opportunity to check whether my theoretical backup and restore procedure with an unlocked bootloader works with /e/ in practice, too. (It did work with the stock Fairphone OS, but better safe than sorry.)

  • Made sure I kept the install file for the initial 20200428 build.
  • Made a copy of the whole Internal Storage (because).
  • Made a TWRP backup of my installation of the 20200428 build, including the data partition.
  • Updated /e/ successfully to the 20200514 build.

Had a look which slot was active (Fairphone 3 is an A/B device), just to be sure to only mess up one slot if in doubt :slight_smile: .

  • Installed the /e/ 20200428 build to the same slot where the 20200514 build was installed before, including wiping the data partition according to the install instructions.
  • Booted /e/ and set up the screen lock (= the decryption method) the same way it was before.
  • Restored the data partition with TWRP.
  • Copied the whole Internal Storage back to the phone.

This worked, the 20200428 build was back with all my stuff as if nothing happened.
The only cleanup left to do was to set my own custom ringtone again.
The ringtone gets set back to a fallback generic one by the OS because the OS is up and running earlier than when my custom ringtone comes back later with the Internal Storage copy.

  • Updated successfully to the 20200514 build again, so this restore didn’t break OTA updates.

(If somebody finds out how to simply restore everything including the OS out of a TWRP backup, please do tell. I ran into a bootloop trying this, just as I did with the stock OS.)

5 Likes

Good to hear that this works! So before trying stuff or updating the OS, with an unlocked bootloader, with the FP3 we can do it just like with any other phone and custom ROM before - apart from needing a laptop to boot TWRP.

Just to be clear: What I wanted to discuss here ultimately goes further - a comprehensive backup solution that doesn’t need you to turn off the device, is independet from OS version and device type as far as possible, can be automated, doens’t need root and/or an unlocked bootloader,…

I will have to track back from my general conviction to run an unlocked bootloader, there’s a legitimate concern about temporary access to the phone with malicious intent …

(Source)

… while I’m still pretty certain the data would be safe if the phone simply got lost or stolen, but we’ll have to see, it seems I didn’t drill down on this enough. Sorry.

2 Likes

Thanks for the update! That’s bad, but still… Running an OS without a backup solution is not an alternative.

1 Like

I am also looking for a way to fully backup my FP3, so that I can fully recover everything of the moment of backup (potentially even on a new device). Is there any update on this goal?

Is device unlocking also required for “adb backup”? When I tried “adb backup”, the “okay” button to start the process was disabled on the phone.

Thank you for the reply! Yes, I already read about it but which backup solution is more reliable?

And a more concrete question: Is TWRP + syncopoli (for the internal storage) reliable? (I saw it worked for you but “adb backup” is also working for some people :wink:.)

It is working for some people :slight_smile: .
There can be no 100% reliable method, because there is no 100% reliability in the way most software is made. This also applies to TWRP (just feed your favourite internet search engine with “createtarfork() process ended with error 255”.)

So the real question is … what’s your Plan B if TWRP fails?

You got the Internal Storage covered as long as you confirm it got successfully copied to your computer or wherever.
You can install the OS from scratch (better download and keep the OS install files matching your backups).

Apart from the /data/media folder (= Internal Storage) that leaves the rest of the data partition. This holds e.g. your contacts, your SMS/MMS messages, your call history, data saved there by Apps.

There are tools like MyPhoneExplorer to sync contacts, messages and call history to a computer.
Google is constantly tightening some screws, so it isn’t as convenient to restore messages as it was some time ago, and I don’t know whether in recent Android the call history can be restored at all to a phone, but at least this data is not lost.

Apps are covered by their availability in Stores, of course.
If you want to quicken the install or if you need a particular version of an App for some feature or compatibility issue, you could extract the respective App APKs (with e.g. SAI, since there are split APKs around which standard APK extractors don’t handle correctly).

App data in the data partition but not in Internal Storage is a problem.
You would have to rely on sync/backup/export functionality of every single App which you consider important. Well … let’s hope TWRP works :slight_smile: .

Yes, I would like reliability similar to copied files. So that I can, in the worst case, manually restore.

(just feed your favourite internet search engine with “createtarfork() process ended with error 255”.)

Thanks for the pointer. This is very scary: Such error numbers without additional descriptions were already bad practice 20 years ago and even worse: they seem to use for every problem the same error code 255 (additional users, profiles, dual apps and/or their data, encryption and compression problems).

So this backup program is really considered to be best practice to backup android phones in the year 2020?

Google is constantly tightening some screws

Isn’t /e/ in a position to change this situation again?

Apps are covered by their availability in Stores, of course.

It is not only that. Installing all apps and configuring them takes for me days, I would like to avoid wasting my time like that.

App data in the data partition but not in Internal Storage is a problem.

Why? Wouldn’t it be possible for /e/ to make all partitions “normal”. Why do they work so differently?

Background: With Linux I am able to do a full backup+restore with “cp” or “rsync” for more than 10 years. Coming back to a working installation from any situation is simply creating encrypted partitions and copy the data to it, and then installing the bootloader. I do not understand why Android, sharing the same kernel, made the situation so much worse?

I won’t be the judge of that.
But TWRP worked great for me on the Fairphone 2 and 3 up until now.

Everybody would like to avoid that, and usually it shouldn’t be necessary (even if advisable when upgrading Android to a new major version).
We are talking about a Plan B here. It’s good to have one, even if it’s a pain.

It’s always amazing what people can achieve if they try hard enough.

/e/ is Android, and Android is what it is. I don’t have much technical insight, I’m just dealing with reality :wink: .

I backup system and data partitions using TWRP, which takes care of installed apps and their settings, as well as settings and data for system apps. I am not bothered about internal storage, since most of my documents and data - at least all of the stuff I care about - lives in (and is synced with) the cloud. My external SD card mostly contains music, which I copy over as required from my home PC.

Thanks for raising the subject: you reminded me it’s time to take an up-to-date backup :slight_smile:

To their defense: it is a common problem in the Android eco-system, e.g. Nextcloud Talk simply shows a black screen on non-working video calls and gives absolute no clue about what went wrong. And I better do not start talking about DAVx⁵.

But TWRP worked great for me on the Fairphone 2 and 3 up until now.

Good to know. Have you even been able to migrate from Fairphone 2 to 3?

Are there some good tutorials how to set it up for Fairphone 3? (I already have /e/ installed but the device is not rooted.)

It’s always amazing what people can achieve if they try hard enough.

I sign that :wink:

Hello. I’m pretty new to my FP3, but already had the great “fun” to lose all my data with the present /e/ update. Having read this thread I tried to install TWRP (I found s.th. like twrp original and a twrp manager). None of them worked (probably because the FP3 does not allow the apps to geht root access?). So how do i manage to get a complete backup? Beside the system i have an SD card for pics,music etc. Funny that the Android folder from the main storage seems to get mirrored to the SD (but didn’t help me, when the system broke down w the last /e/-update)

Hello, I know there is a way to boot TWRP for FP3 without installing it, so you could go into this direction.

But does the backup/restore feature works on the FP3, no idea.

Works like a charm. This is exactly what OP wanted (transferring the whole thing verbatim into a collection of partition images) However (please read on)

Word of advice for anyone reading this: I come from a time where we used things like CyanogenMod, ClockWorkMod and SuperSU. A/B slots didn’t exist and recovery partitions were a thing. Due to my own ignorance on how things work nowadays I shot myself in the foot a few times over the last few days. I lost all my user data once, and at some point I even managed to brick both slots at the same time.

Since it was pretty painful for me, some hints on things to be careful with:

  • If you begin tinkering with TWRP, at some point you’ll probably want to unlock your bootloader. Make sure you have a working, full backup first because unlocking your bootloader will wipe your data partition clean (yes I learned this the hard way…)
  • read about A/B slots and make sure you understand the basic principles before doing anything at all. When you backup and restore from TWRP you should know which slot you’re backing up or restoring to!
  • It’s not really a good idea to install TWRP permanently before you have a working backup to fallback to and a working alternate slot to boot, so you can “adb push” the files from it. TBH I don’t really understand how this works now that there’s no recovery partition, but whatever tricks it’s doing to blend it into boot partition they seem dangerous…
  • I recommend reading this thread too from the developer of TWRP/FP3 port, lots of useful info in it.
  • If when trying to recover you think of going back to FairphoneOS, keep in mind this procedure will install it to both slots simultaneously. If at that point your data partition was being managed by /e/OS, then both slots will complain about data being corrupt and demand that you perform factory reset :smirk:
  • Edit: I almost forgot. Do not recover every partition! This and this recommends only boot, system, vendor and data. However on some of my attempts this wasn’t enough, so I also included in my restore dtbo, product and vbmeta which are the ones listed in /e/OS install instructions. I’ve restored firmware and EFS at least once and nothing bad happened, but I really don’t recommend it. If I understand correctly what “firmware” means here, I guess this could permanently brick your device. And I’ve read somewhere EFS is where IMEI data is stored so better not play with that, unless you actively want to tamper with GSM settings (is that even legal?).
6 Likes

BTW does anyone know what “firmware” even means nowadays? I thought everything except data partition was considered firmware. And referring as “firmware” to a partition that’s not firm and can be overwritten seems contradictory. Is “firmware” where the fastboot environment lives?

@robertb: Thank you for your good summary of information and in-depth-links. But I have one question:

I want to create a full backup of my FP3 partitions. But I still have the bootloader locked.

When I try to fastboot boot twrp.img, I get:

FAILED (remote: unlock device to use this command)

So the first thing for me in order to boot into TWRP seems to be: I have to unlock the bootloader.

The question: Do I miss anything?

I do not really want to unlock the bootloader, since it is a FP3 with preinstalled /e/OS, so I am worried about loosing warranty. On the other hand I really would like to have partition backups in case of OTA updates going wrong. (I suffered from this on other devices with other operation systems.)

No, you are not missing anything.
If you want to boot TWRP, you’ll have to unlock the bootloader.

And be aware that TWRP doesn’t do a full backup of the data partition, as TWRP doesn’t backup Internal Storage, you might want to copy that to somewhere safe additionally (can be done via USB when TWRP is running, for instance).

1 Like

I suspect we might be referring to different things.

“unlock the bootloader” could mean 1 but also 2. Note that 1 is the first step of 2.

When I used this phrase I was referring to the full steps in 2. Maybe the error you’re getting means you have to do 1 (which is only the first step in 2 and doesn’t wipe your data partition).