[HOWTO] End-to-end Encrypted Full Backup and Restore Solution for /e/OS using Termux and restic

In the wake of the upcoming /e/OS T release, I thought it would be handy to share a new Android backup solution I recently discovered and which is still fairly unknown, I guess :smile:

About this backup solution

  • We make use of restic here, a standalone command-line backup tool which generally works on all major platforms (Linux, BSD, Mac, Windows WSL, and Android)
  • restic is free Open Source software and backups are end-to-end encrypted per default
  • it allows you to store your encrypted backups both locally or on a remote server (in my guide below I will show you how to backup to sd card, own server, and Murena Cloud / Nextcloud, but there are even more storage types)
  • backups are de-duplicated, incremental and compressed, which saves you a lot of time, space, and data usage
  • as restic is a independent tool this backup solution does NOT depend on adb, adb backup, adb root, root debugging or custom shell scripts
  • it allows you to backup all the data on your device if you have root, so you don’t have to deal with backups being incomplete as in the case of SeedVault when certain apps have set a “do not backup” flag
  • restoring data works in two ways: specifying a certain path on Android to restore to or you mount a backup snapshot on your desktop file system and just copy files out of there (there is also a open source restic file explorer software to simplify this, see Restic Browser in honorable mentions below)

Downsides of this backup solution

  • Without root you are only able to backup internal storage (of personal and work profiles) and sd card
  • We have to utilize the command line (Termux), if you are fine with that, this guide should be pretty straight foward for you :slight_smile:
  • restic consumes more RAM the bigger your backup repository gets (I guess we are talking about TBs here, my FP4 has 8 GB RAM, and worked perfectly fine so far)
  • <Your feedback is needed here>

I successfully tested this method on a FP4 with /e/OS R (Android 11) and S (Android 12).

Please note that the app data directories given in the examples here could differ depending on your specific device. The given sd card and internal storage paths should match accross most devices, I hope.

For more details on the usage of restic please refer to the restic documentation. I won’t go into topics like tags, backup rotation, key management, etc. here.

Honorable mentions

  • This guide uses restic natively via command line. If you prefer to use a GUI, there is a unofficial experimental Restic Backup Android App on F-Droid (I won’t cover this app here as it is quite limited as of now and therefore doesn’t offer the same capabilites on which we rely in this guide)
  • Restic Browser provides you with a GUI to browse and restore your restic backup repositories on your Windows/Linux/MacOS desktop machine
  • For those ppl who prefer a similar encrypted FLOSS backup solution: Instead of restic you can also use BorgBackup via Termux. It can be installed using the command: pkg install borgbackup

Prerequisites

  • Termux installed (install via F-Droid)
  • Grant Termux file access permissions in app settings (else you will run into permission denied errors when running restic)
  • For full backups: root (e.g. Magisk), Termux needs to be able to run as Superuser when running restic with sudo

1 Initial Setup: Install restic via Termux

Update packages

apt update
apt upgrade

→ If there are any prompts just hit ENTER all the time

Update all termux packages

pkg upgrade

Install sudo (ROOT ONLY)

pkg install tsu

Install restic

pkg install restic

Check if restic was installed successfully

restic version

restic 0.16.0 compiled with go1.20.6 on android/arm64

2 Backing up your data

2.1 OPTION 1: Backup to sd card

:exclamation: Replace “/storage/XXXX-XXXX/” with your sd card path :exclamation:

Create backups folder on sd card

mkdir /storage/XXXX-XXXX/backups/

Initialize restic backup repository

:exclamation: This step is only needed once at first setup :exclamation:

restic init -r /storage/XXXX-XXXX/backups/

enter password for new repository:
enter password again:
created restic repository 36fbfa9fb8 at /storage/XXXX-XXXX/backups/

Please note that knowledge of your password is required to access the repository. Losing your password means that your data is irrecoverably lost.

Backup internal storage (without root) (default/personal profile)

restic -r /storage/XXXX-XXXX/backups backup /storage/emulated/0/ --verbose

Backup app data (with root) (default/personal profile)

sudo restic -r /storage/XXXX-XXXX/backups backup /data/data/ /data/user/0/ /data/user_de/0/ --verbose

Show all backups

restic -r /storage/XXXX-XXXX/backups snapshots

Restore backup (latest backup)

restic -r /storage/XXXX-XXXX/backups restore --target /storage/emulated/0/my-restore-directory latest

Browse your backups / Mount backup repository to file system (for example on your desktop machine)

:exclamation: In order to use this restic needs also to be installed on your desktop machine :exclamation:

mkdir /mnt/restic
restic -r /your/sdcard/path/backups mount /mnt/restic

If you prefer a GUI for browsing your backups on your desktop machine: Use Restic Browser.

2.2 OPTION 2: Backup to Murena Cloud / your own Nextcloud via WebDAV

:exclamation: restic doesn’t support backing up to Murena Cloud / Nextcloud instances (WebDAV) out of the box, it needs rclone for that :exclamation:

Update packages

apt update
apt upgrade

→ If there are any prompts just hit ENTER all the time

Update all termux packages

pkg upgrade

Install rclone

pkg install rclone

Configure rclone

rclone config

Create new remote

2023/09/23 11:00:04 NOTICE: Config file “/data/data/com.termux/files/home/.config/rclone/rclone.conf” not found - using defaults
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q>

→ n

Specify name of your backup destination

name>

→ MurenaWebDAV / NextcloudWebDAV (or whatever name you like)

Specify type of your backup server

Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.

→ Choose number corresponding to WebDAV

Specify backup-server URL

Option url.
URL of http host to connect to.
E.g. https://example.com.
Enter a value.
url>

→ Log into Murena Cloud / Nextcloud, navigate to Files → open menu → Files settings → copy WebDAV path from there (alternatively, use the URL below and replace USERNAME with your username)

https://murena.io/remote.php/dav/files/USERNAME/

Choose WebDAV Type

Option vendor.
Name of the WebDAV site/service/software you are using.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
1 / Fastmail Files
\ (fastmail)
2 / Nextcloud
\ (nextcloud)
3 / Owncloud
\ (owncloud)
4 / Sharepoint Online, authenticated by Microsoft account
\ (sharepoint)
5 / Sharepoint with NTLM authentication, usually self-hosted or on-premises
\ (sharepoint-ntlm)
6 / Other site/service or software
\ (other)
vendor>

→ Choose number corresponding to Nextcloud

Specify Murena Cloud / Nextcloud username

Option user.
User name.
In case NTLM authentication is used, the username should be in the format ‘Domain\User’.
Enter a value. Press Enter to leave empty.
user>

→ Enter your Murena Cloud / Nextcloud USERNAME

Specify Murena Cloud / Nextcloud password

:exclamation: For security reasons it is recommended to use a dedicated App password :exclamation:

→ Log into Murena Cloud / Nextcloud, navigate to Settings → Security → scroll down to Devices & sessions → Fill in a app name of your liking, e.g. “restic-Android” → Create new app password → copy to clipboard

Option pass.
Password.
Choose an alternative below. Press Enter for the default (n).
y) Yes, type in my own password
g) Generate random password
n) No, leave this optional password blank (default)
y/g/n>

→ y

Enter the password:
password:
Confirm the password:
password:

→ Enter your Murena Cloud / Nextcloud PASSWORD

Skip Bearer token process

Option bearer_token.
Bearer token instead of user/pass (e.g. a Macaroon).
Enter a value. Press Enter to leave empty.
bearer_token>

→ ENTER (to skip)

Save configuration

Edit advanced config?
y) Yes
n) No (default)
y/n>

→ n

Confirm your configuration

Configuration complete.
Options:

→ y

Your added cloud server will be listed as current remote

Current remotes:

Name Type
==== ====
MurenaWebDAV webdav

Finally, you can quit rclone

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>

→ q

Test WebDAV connection to cloud via rclone

:exclamation: Don’t miss the colon “:” after the name) :exclamation:

rclone lsd MurenaWebDAV:

→ Should display files and folders you have inside your Murena Cloud / Nextcloud:

2023/09/23 11:19:58 NOTICE: Chunks temporary upload directory: https://murena.io/remote.php/dav/uploads/USERNAME/
-1 2023-09-09 10:33:55 -1 Documents
-1 2023-09-09 10:33:54 -1 Images
-1 2023-09-09 10:33:56 -1 Templates

Initialize restic backup repository

:exclamation: This step is only needed once at first setup :exclamation:

restic -r rclone:MurenaWebDAV:/backups init

Backup internal storage (without root) (default/personal profile)

restic -r rclone:MurenaWebDAV:/backups backup /storage/emulated/0/ --verbose

Backup app data (with root) (default/personal profile)

sudo restic -r rclone:MurenaWebDAV:/backups backup /data/data/ /data/user/0/ /data/user_de/0/ --verbose

Show all backups

restic -r rclone:MurenaWebDAV:/backups snapshots

Restore backup (latest backup)

restic -r rclone:MurenaWebDAV:/backups restore --target /storage/emulated/0/my-restore-directory latest

Browse your backups / Mount backup repository to file system (for example on your desktop machine)

:exclamation: In order to use this restic needs also to be installed on your desktop machine :exclamation:

mkdir /mnt/restic
restic -r rclone:MurenaWebDAV:/backups mount /mnt/restic

If you prefer a GUI for browsing your backups on your desktop machine: Use Restic Browser.

2.3 OPTION 3: Backup to own server via SFTP/SSH

:exclamation: needs restic and OpenSSH to be installed on both sides (Android & server) :exclamation:

Install OpenSSH

pkg install openssh

Create a dedicated user for restic on your server

:exclamation: Replace HOST and USER below with the ones of your server :exclamation:

Initialize restic backup repository

:exclamation: This step is only needed once at first setup :exclamation:

restic -r sftp:USER@HOST:backups init

On first connection you will be prompted once to verify the key fingerprint of your server

The authenticity of host ‘XXX.XXX.X.XX (XXX.XXX.X.XX)’ can’t be established. ED25519 key fingerprint is SHA256:XXXXXXXXXX…
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

→ yes

Backup internal storage (without root) (default/personal profile)

restic -r sftp:USER@HOST:backups backup /storage/emulated/0/ --verbose

Backup app data (with root) (default/personal profile)

sudo restic -r sftp:USER@HOST:backups backup /data/data/ /data/user/0/ /data/user_de/0/ --verbose

Show all backups

restic -r sftp:USER@HOST:backups snapshots

Restore backup (latest backup)

restic -r sftp:USER@HOST:backups restore --target /storage/emulated/0/my-restore-directory latest

Browse your backups / Mount backup repository to file system (for example on your desktop machine)

:exclamation: In order to use this restic needs also to be installed on your desktop machine :exclamation:

mkdir /mnt/restic
restic -r sftp:USER@HOST:backups mount /mnt/restic

If you prefer a GUI for browsing your backups on your desktop machine: Use Restic Browser.

3 Advanced usage

Backup work profile (without root)

Install Termux in work profile or clone to work profile using Shelter/Island/Insular

Run following commands from Termux in work profile:

Backup internal storage of work profile to sd card

restic -r /storage/XXXX-XXXX/backups backup /storage/emulated/10/ --verbose

Backup internal storage of work profile to Murena Cloud / Nextcloud

:exclamation: To make this work first repeat rclone configuration steps from guide above with Termux in work profile :exclamation:

restic -r rclone:MurenaWebDAV:/backups backup /storage/emulated/10/ --verbose

Backup internal storage of work profile to server via SFTP/SSH

restic -r sftp:USER@HOST:backups backup /storage/emulated/10/ --verbose

Backup work profile (with root)

No need to install Termux to work profile, just use sudo and specify work profile path (/data/media/10/)

Backup internal storage of work profile to sd card (root)

sudo restic -r /storage/XXXX-XXXX/backups backup /data/media/10/ --verbose

Backup internal storage of work profile to Murena Cloud / Nextcloud (root)

:exclamation: to make this work first repeat rclone configuration steps from guide above using: sudo rclone config :exclamation:

sudo restic -r rclone:MurenaWebDAV:/backups backup /data/media/10/ --verbose

Backup internal storage of work profile to SSH server (root)

sudo restic -r sftp:USER@HOST:backups backup /data/media/10/ --verbose
7 Likes

That’s really interesting and I just tried it (not on /e/ though), but on my non-rooted phone I get a “permission denied” in termix if I try /storage/emulated/0/. As a consequence, the backup doesn’t really contain anything.

Edit: my bad, I hadn’t granted “file access” to termux yet and it didn’t ask, so I enabled in the the app settings.

Thx, I just added that bit to the guide, so that we hopefully got that permission requirement covered now :+1:

1 Like

I’m stuck at the step for initializing the backup repository. It says:
Fatal: create key in repository at /storage/F01A5CD41A5C9984/System-Backups/RealmeGT/restic-backup failed: sync /storage/F01A5CD41A5C9984/System-Backups/RealmeGT/restic-backup/keys/3525ec52c31e3a12622d289628508ab9ebc4dfbdfefb0bfdbfaeacbdee609aa9-tmp-2782534080: operation not permitted

I have no idea why, I’m rooted, I used tsu, and I granted Termux all permissions

Without having some background info this is kinda hard to solve.

  • On what Android, Magisk, Termux, and restic version are you?
  • Did you enable Termux as Superuser in Magisk?

From what I can see you try to backup on SD card. If you are rooted anyway, does it work if you run the initial commands with sudo?:

# Create backups folder on sd card
sudo mkdir -p /storage/F01A5CD41A5C9984/System-Backups/RealmeGT/restic-backup/

# Initialize restic backup repository (NOTE: This step is only needed once at first setup)
sudo restic init -r /storage/F01A5CD41A5C9984/System-Backups/RealmeGT/restic-backup/

It could be on certain devices that the path to SD card differs from the given /storage/ example. To check if you can access your SD card from Termux, just try to run ls -l /path/to/your/sdcard/ there. If Termux returns the list of the content on your SD card you should be also able to initalize a restic backup repository there. Of course you should also make sure that the backup folder exists first and then initialize the restic repo in it.

Hi, many thanks for your reply

  • Android 13, RMX2202_13.1.0.1630(EX01)
  • Magisk 26.3
  • Termux 0.118.0
  • Restic: 0.16.2
    Termux has Superuser access

I don’t know if it’s important, but this is not an SD card but an external hard drive connected via the OTG port.

Adding sudo to the commands makes no difference. Although Magisk notifies me Termux has been given superuser permissions, it still fails with the same error. ls -l works normally and lists everything. I have full access to everything on the drive from my phone and can edit and delete stuff as I please, both in Termux and in managers like Root Explorer. But it always fails with the same error message