This is good news
Fingers crossed !
Here you are : Jaguar Network - Share
MD5 : 05412f6737e627e74b6ffcbaaa03e6df
This is good news
Fingers crossed !
Here you are : Jaguar Network - Share
MD5 : 05412f6737e627e74b6ffcbaaa03e6df
très élégant ce jaguar…
e_a3xelte-userdebug 10 QQ3A.200805.001 eng.root.20210812.020744 UNOFFICIAL-release,test-keys
Runs very well. Even Magic Earth found me straight away, tested very briefly in Bliss and Lawnchair. Great.
You are a star. Thank you.
Please be aware that 0.18 /e/ is still in beta testing phase
My thanks, Sylvain, for your help and support.
As you guessed, it seems my 373G partition is just too small. If there is a way to limit the size of the download in Docker with a local_manifest, it has eluded me.
During the final hour of the session is seen at 15:38 hours
Filesystem Size Used Avail Use% Mounted on
/dev/sda4 373G 373G 0 100% /srv
A search for the last files to sync, shows this file downloaded at 16:12 hours.
/srv/src/Q/.repo/project-objects/platform/prebuilts/clang/host/linux-x86.git/objects/pack:
-r--r--r-- 1 root root 4244258257 Aug 17 16:12 tmp_pack_OR8ZKm
While this shows that the “build folders” start to be created in Q at 16.15 hours.
ls -alt /srv/src/Q
drwxr-xr-x 90 root root 4096 Aug 17 16:22 external
drwxr-xr-x 8 root root 4096 Aug 17 16:19 device
drwxr-xr-x 20 root root 4096 Aug 17 16:18 development
drwxr-xr-x 3 root root 4096 Aug 17 16:16 developers
drwxr-xr-x 10 root root 4096 Aug 17 16:16 dalvik
drwxr-xr-x 3 root root 4096 Aug 17 16:15 cts
lrwxrwxrwx 1 root root 19 Aug 17 16:15 Android.bp -> build/soong/root.bp
lrwxrwxrwx 1 root root 26 Aug 17 16:15 bootstrap.bash -> build/soong/bootstrap.bash
drwxr-xr-x 6 root root 4096 Aug 17 16:15 build
-r--r--r-- 1 root root 92 Aug 17 16:15 Makefile
drwxr-xr-x 3 root root 4096 Aug 17 16:15 bootable
drwxr-xr-x 13 root root 4096 Aug 17 16:15 bionic
drwxr-xr-x 31 root root 4096 Aug 17 16:15 art
drwxr-xr-x 4 root root 4096 Aug 17 16:15 android
But Q folders are not fully populated:
/srv/src/Q$ du -sBG *
1G android
0G Android.bp
1G art
1G bionic
1G bootable
0G bootstrap.bash
1G build
1G cts
1G dalvik
1G developers
1G development
1G device
3G external
1G Makefile
Docker fails with Exit Code 1. No specific out of space error or final time stamp appears in the repo-log.
One assumes the sync completed ok as the repo-log shows 3,020 lines saying “Updating files”, speculation, populating the above directory, Q/
then, speculation, the “out of space error” is 44 lines similar to:
error: unable to write file examples/apache-httpd/corpus_http2 2511e2df990fb53afa7d65fd0d5438d6.00003b8b.honggfuzz.cov
which even seems to end “out of breath” with
error: unable to write file examples/apache-httpd/corpus_http2/952e916795345b2e386ba354a6a93306.0001f1bb.honggfuzz.cov
error: unable to write file examples/apache-httpd/corpus_http2/953b268a0acd5479f833925e0b039241.0000c26e.honggfuzz.cov
error: unable to write file examples/apache-httpd/corpus_http2/95926929821cc8a6dd0660d477e8a348.00000
I think I will try a non-docker build GitHub - kikislater/builde: Script to build /e/ next!
You’re welcome
Your guessing seems correct to me : you’re out of disk space.
Some program may output a clear error message, some other won’t …
“unable to write file” looks like one of them !
I don’t think that trying to build out of Docker will solve the problem : Docker is only using a few GB (comparing to src), more likely located outside of your build filesystem.
This being said, 373GB of source tree is way too much !
I don’t know any way to reduce it’s size, it would imply to know if some repo/branches are useless …
Did you purge your source tree before download ? If you have some previous versions of some repo, they may have been kept, aside the new ones.
I will try to build for 0.18 Q a3xelte after reducing my storage to 373GB, we’ll see what happen.
But not before this evening.
I’ll try to record some sizes during the whole operation …
Yes, I formatted the partition (ext4). I also chose to clear /overlay2/ thoroughly. That is very good of you to offer to try to reproduce my problem, thanks but no hurry!
Hi !
Got some news …
I reduced my build filesystem to 369GB (sorry it’s a SAN disk, very difficult to get precise size because of allocation schemes …).
And my build failed, but not like yours
Repo sync was OK, and build failed with a classic “No space left on device”.
I’ve put a “du” in cron (every 5 minutes), but like a dumbass I forgot to output timestamp, so it’s unusable
If you wish to compare I can upload the repo log or retry the whole build, with timestamps this time !
Thank you so much for the time spent on this!
I think we established one needs more than 370G to build /e/ with docker.
This is my feeling from the numbers I collected, I omitted any speculation about the way that ccache is accommodated, and how one estimates an “overhead” for building.
I will now be looking for a more space solution!
Thank you again!
Seems like we have no choice here …
About CCACHE : you can manage it’s size by using the variable CCACHE_SIZE, i.e. by adding
-e “CCACHE_SIZE=10G” \
to your Docker command (default is 50G).
Remember, you can list variables with docker inspect <imageid>
command
But decreasing CCACHE size will slow down the build process …
I’ll retry the build with a filesystem size around 400GB.
Why would a guy take the hard road when there is an easy way ?
So I attached a 512GB volume, improved the du followup with timestamps, and launched a new build.
This way, we’ll know how much space it will need !
Build done
The most disk space used was :
Aug 22 13:51:02 344G /srv
Aug 22 13:55:09 353G /srv
Aug 22 14:00:05 303G /srv
Corresponding entries in log (Docker is 2 hours behind my local clock) :
2021-08-22 11:54:52 - common.py - INFO : using prebuilt recovery.img from BOOTABLE_IMAGES…
2021-08-22 11:54:52 - common.py - INFO : Running: “brotli --quality=6 --output=/srv/src/Q/out/soong/.temp/tmpw7Htqs/system.new.dat.br /srv/src/Q/out/soong/.temp/tmpw7Htqs/system.new.dat”
2021-08-22 11:57:07 - common.py - INFO :
2021-08-22 11:57:09 - common.py - INFO : using prebuilt boot.img from BOOTABLE_IMAGES…
So, the most usage was at the end of the build process, when the system image was assembled.
That make sense
NB 1 : my CCACHE setting was the default 50G, after build the size of ccache directory is 11G …
NB 2 : the du command ran every 5 minutes, so real max size can be a little higher …
I’m following along your disk space shortages to be prepared for my own build.
I want to link to [HOWTO] How much space do I need to build for /e/? as it already speaks of needing 390G combined, android official says 400G and quotes some numbers on machine size vs build time
Please be aware that it was a 0.18 Q build for a3xelte; sizes may change for another device !
And obviously, the space needed increases with the versions of Android …
4 weeks later…
The larger hard drive did not provide the solution directly, but it did allow me to work on building a LineageOS ROM with breakfast and brunch side by side with Docker!
I eventually produced a Lineage ROM that ran nicely. For /e/ I followed the same method and overnight this ROM was delivered 0.19-q-20210919-UNOFFICIAL-a3xelte and it has just booted successfully.
I read the repo script and in my imagination I thought my chaotic results could possibly result from repo being able to flip-flop between python2 and python 3.
I added this symlink
sudo ln -s /usr/bin/python3.8 /usr/bin/python
and most of the chaotic traces in my Lineage build disappeared and I could quite easily (during 3 weeks) resolve the issues.
Now that I search the errors I received I find that this Android page mentions the python2 python3 weakness and was updated only 5 days ago!
I will firm up these numbers but I have a complete Lineage build occupying about 154G and ‘lean’ /e/ of 118G. (Detailed below)
If I went back to Docker I would check out
A new version of repo (2.15) is available.
You should upgrade soon
As @Smu44 pointed out in post #15 that change would have to be made within docker; and
Edit. An up to date repo command does exist as mentioned in post #15. It may/might be used during the build, but during repo init and repo sync, the “pre existing” repo will be used.
Nice work !
The repo warning is to be ignored, as it is working fine.
However, one could build an updated Docker image, using the Docker file in there : https://gitlab.e.foundation/e/os/docker-lineage-cicd
The Docker build script manages the Python 2-3 switch, please see lines #200 & #303 : https://gitlab.e.foundation/e/os/docker-lineage-cicd/-/blob/master/src/build.sh
I did only see this warning in my above Android link, this morning:
Warning: Python 2 support was sunset on Jan 1, 2020 as detailed in Sunsetting Python 2. All major Linux distributions are deprecating support for the Python 2 package. Google strongly recommends that you migrate all your scripts over to Python 3.
Perhaps in my euphoria, I jumped to the wrong conclusion - lol.
Edit. My symlink of python to python3.8 seemed contrary (the late night I did it) knowing the previously_understood method included the use of python2 and python3.
[Edit to clarify]. The result that surprised me was that the Lineage build immediately resolved [having implemented this change] while the docker /e/ build [running its own unaltered system] continued with a repo sync which would not finish - which in my case seems to be the thing that breaks Docker on this machine.
That’s the weird problem
Unfortunately, it’s difficult to know if it’s related to the Docker image, your host system, …
As you could download LOS source trees, your network may be out of cause.
You may want to try this method …
Hi @smu44 thanks for your reply, I think also part responsible that my machine (Intel i3 @ 2.40GHz, nproc=4, 8G RAM + 12G Swap) is probably at the bottom end of capable!
Another weird thing is the space used. I have 3 builds on two hard drives, here measuring with ncdu.
ncdu 1.13 /srv
372.1 GiB [##########] /src
2.3 MiB [ ] /logs
20.0 KiB [ ] /local_manifests
…Total disk usage: 372.1 GiB Apparent size: 370.9 GiB Items: 356359
repo init -u https://gitlab.e.foundation/e/os/android.git -b v1-q --depth 1 --no-clone-bundle
repo sync -j 1 -c --fail-fast --no-clone-bundle, breakfast, brunch
ncdu 1.13 /media/iain/ebuild
50.6 GiB [##########] /out
32.9 GiB [###### ] /prebuilts
18.0 GiB [### ] /.repo
8.0 GiB [# ] /external
1.9 GiB [ ] /frameworks
…Total disk usage: 118.5 GiB Apparent size: 118.0 GiB Items: 1415020
ncdu 1.13 /media/iain/android/lineage --------
54.6 GiB [##########] /.repo
47.3 GiB [######## ] /out
32.2 GiB [##### ] /prebuilts
8.0 GiB [# ] /external
3.5 GiB [ ] /vendor
…Total disk usage: 154.6 GiB Apparent size: 153.9 GiB Items: 1792990
Yes, that link is very nicely worded and has formed part of my education