>> [Fri Feb 23 19:11:39 UTC 2024] Setting "UNOFFICIAL" as release type
>> [Fri Feb 23 19:11:39 UTC 2024] Adding OTA URL overlay (for custom URL )
>> [Fri Feb 23 19:11:39 UTC 2024] Preparing build environment
>> [Fri Feb 23 19:11:39 UTC 2024] Starting build for FP4, v1.19.1-s branch
>> [Fri Feb 23 19:11:39 UTC 2024] ANDROID_JACK_VM_ARGS=-Dfile.encoding=UTF-8 -XX:+TieredCompilation -Xmx4G
>> [Fri Feb 23 19:11:39 UTC 2024] Switch to Python2
>> [Fri Feb 23 19:13:35 UTC 2024] Failed build for FP4
>> [Fri Feb 23 19:13:35 UTC 2024] Finishing build for FP4
>> [Fri Feb 23 19:13:35 UTC 2024] Cleaning source dir for device FP4
>> [Fri Feb 23 19:13:37 UTC 2024] Switch back to Python3
It temporarly freezes at “Swtich to Python2” and eventually fails the build. This is the log file of the build:
[Fri Feb 23 19:11:39 UTC 2024] Starting build for FP4, v1.19.1-s branch
19:11:41 Build sandboxing disabled due to nsjail error.
Trying dependencies-only mode on a non-existing device tree?
19:11:43 Build sandboxing disabled due to nsjail error.
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=12
LINEAGE_VERSION=1.19.1""-s-20240223-UNOFFICIAL-FP4
TARGET_PRODUCT=lineage_FP4
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_ARCH=arm64
TARGET_ARCH_VARIANT=armv8-a
TARGET_CPU_VARIANT=cortex-a76
TARGET_2ND_ARCH=arm
TARGET_2ND_ARCH_VARIANT=armv8-a
TARGET_2ND_CPU_VARIANT=cortex-a76
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-6.5.0-18-generic-x86_64-Ubuntu-20.04.6-LTS
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=SQ3A.220705.004
OUT_DIR=out
PRODUCT_SOONG_NAMESPACES=vendor/fairphone/FP4 device/fairphone/FP4 hardware/qcom-caf/sm8250 vendor/qcom/opensource/commonsys/display vendor/qcom/opensource/commonsys-intf/display vendor/qcom/opensource/display vendor/qcom/opensource/data-ipa-cfg-mgr vendor/qcom/opensource/dataservices packages/apps/Bluetooth
WITH_SU=false
============================================
19:11:44 **********************************************************
19:11:44 You are building on a machine with 15.4GB of RAM
19:11:44
19:11:44 The minimum required amount of free memory is around 16GB,
19:11:44 and even with that, some configurations may not work.
19:11:44
19:11:44 If you run into segfaults or other errors, try reducing your
19:11:44 -j value.
19:11:44 **********************************************************
19:11:44 Build sandboxing disabled due to nsjail error.
e[1mbuild/make/core/soong_config.mk:199: e[35mwarning: e[0me[1mBOARD_PLAT_PUBLIC_SEPOLICY_DIR has been deprecated. Use SYSTEM_EXT_PUBLIC_SEPOLICY_DIRS instead.e[0m
e[1mbuild/make/core/soong_config.mk:200: e[35mwarning: e[0me[1mBOARD_PLAT_PRIVATE_SEPOLICY_DIR has been deprecated. Use SYSTEM_EXT_PRIVATE_SEPOLICY_DIRS instead.e[0m
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=12
LINEAGE_VERSION=1.19.1""-s-20240223-UNOFFICIAL-FP4
TARGET_PRODUCT=lineage_FP4
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_ARCH=arm64
TARGET_ARCH_VARIANT=armv8-a
TARGET_CPU_VARIANT=cortex-a76
TARGET_2ND_ARCH=arm
TARGET_2ND_ARCH_VARIANT=armv8-a
TARGET_2ND_CPU_VARIANT=cortex-a76
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-6.5.0-18-generic-x86_64-Ubuntu-20.04.6-LTS
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=SQ3A.220705.004
OUT_DIR=out
PRODUCT_SOONG_NAMESPACES=vendor/fairphone/FP4 device/fairphone/FP4 hardware/qcom-caf/sm8250 vendor/qcom/opensource/commonsys/display vendor/qcom/opensource/commonsys-intf/display vendor/qcom/opensource/display vendor/qcom/opensource/data-ipa-cfg-mgr vendor/qcom/opensource/dataservices packages/apps/Bluetooth
WITH_SU=false
============================================
[100% 1/1] out/soong/.bootstrap/bin/soong_build out/soong/build.ninja
FAILED: out/soong/build.ninja
cd "$(dirname "out/soong/.bootstrap/bin/soong_build")" && BUILDER="$PWD/$(basename "out/soong/.bootstrap/bin/soong_build")" && cd / && env -i "$BUILDER" --top "$TOP" --out "out/soong" -n "out" -d "out/soong/build.ninja.d" -t -l out/.module_paths/Android.bp.list -globFile out/soong/.bootstrap/build-globs.ninja -o out/soong/build.ninja --available_env out/soong/soong.environment.available --used_env out/soong/soong.environment.used Android.bp
Killed
19:13:35 soong bootstrap failed with: exit status 1
ninja: build stopped: subcommand failed.
#### failed to build some targets (01:52 (mm:ss)) ####
> [Fri Feb 23 19:13:35 UTC 2024] Failed build for FP4
> [Fri Feb 23 19:13:35 UTC 2024] Finishing build for FP4
> [Fri Feb 23 19:13:35 UTC 2024] Cleaning source dir for device FP4
19:13:37 Entire build directory removed.
#### build completed successfully (2 seconds) ####
It seems that the issue has to do with not enough RAM. Is their any solution without having to get more RAM. I currently have sixteen.
I won’t try to answer your real question (other than 16G should be ok in a fairly modern 64bit machine) but I think it worth considering the number of processors in your machine (but you might consider adding 4- 8G swap space, can be done a bit like this for an experiment https://linuxize.com/post/how-to-add-swap-space-on-debian-10/).
Just run the command nproc. The response is your number of processors.
The -j flag is part of repo. The -j flag determines how many jobs will run simultaneously. When building with repo, especially in a new environment with an undeveloped ccache and 16G RAM I would run first with a significant -j flag. The command for nproc=8 might be:
repo sync -j 4 -c --force-sync --no-clone-bundle
In the event of certain problems repo might suggest
Edit
Is this a true Ubuntu environment ? I am uncertain of the significance of
Second edit I wonder if you have built a ROM on this machine before. If the aim was to test the machine I would follow these instructions to build standard LineageOS https://wiki.lineageos.org/devices/FP4/build/. A standard Lineage build seems a bit less demanding of resources. (Personally I do not favour Docker for testing as it is so scripted that error messages can be unclear. A Lineage build from these instructions would be more educational.)
Update as I had not actually seen the 15.4GB of RAM error verbatim. Even adding 1G Swap should suppress this warning.
A little more research in a current repo sync build environment shows this …
// Build the tree. The 'what' argument can be used to chose which components of
// the build to run.
func Build(ctx Context, config Config, what int) {
ctx.Verboseln("Starting build with args:", config.Arguments())
ctx.Verboseln("Environment:", config.Environment().Environ())
if totalRAM := config.TotalRAM(); totalRAM != 0 {
ram := float32(totalRAM) / (1024 * 1024 * 1024)
ctx.Verbosef("Total RAM: %.3vGB", ram)
if ram <= 16 {
ctx.Println("************************************************************")
ctx.Printf("You are building on a machine with %.3vGB of RAM\n", ram)
ctx.Println("")
ctx.Println("The minimum required amount of free memory is around 16GB,")
ctx.Println("and even with that, some configurations may not work.")
ctx.Println("")
ctx.Println("If you run into segfaults or other errors, try reducing your")
ctx.Println("-j value.")
ctx.Println("************************************************************")
} else if ram <= float32(config.Parallel()) {
ctx.Printf("Warning: high -j%d count compared to %.3vGB of RAM", config.Parallel(), ram)
ctx.Println("If you run into segfaults or other errors, try a lower -j value")
… occurs in /build/soong/ui/build/build.go … but I do not know really how significant the warning is with this small margin of diff.
Beyond my understanding … my search indicates a tentative link also to /external/swiftshader/third_party/llvm-7.0/llvm/, which again might point to the enquiry “Is this a true Ubuntu environment”.
I think looking more carefully, @maximebellis, your virtual disk limit is set way too low – even the max 467G is borderline. Please can you try again on max.
Can I double check, via Dual boot or VM ?
Edit
Interesting from your image, looks like your machine has nproc=8, but docker seems to allow you to set a limit of say 4 (or even 1 later, as this would further help eliminate some segfault). This would seem worth a test, worth also comparing / noting “build time” with last 03:48 (mm:ss) On reflection this is probably not a way of manipulating the -j flag.
So I tried to completly restart. Removed the folder and create the directories again. Also maxed out everything in docker. When I first type repo sync, it adds the device files for the FP4 in the device folder. But when I than run
>> [Sat Feb 24 12:18:46 UTC 2024] Starting build for FP4, v1.19.1-s branch
12:18:52 Build sandboxing disabled due to nsjail error.
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
12:18:52 dumpvars failed with: exit status 1
File "/usr/local/bin/repo", line 51
def print(self, *args, **kwargs):
^
SyntaxError: invalid syntax
Device FP4 not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Found repository: android_device_fairphone_FP4
Default revision: lineage-19.1
Checking branch info
Checking if device/fairphone/FP4 is fetched from android_device_fairphone_FP4
Adding dependency: LineageOS/android_device_fairphone_FP4 -> device/fairphone/FP4
Using default branch for android_device_fairphone_FP4
Syncing repository to retrieve project.
Repository synced!
Looking for dependencies in device/fairphone/FP4
device/fairphone/FP4 has no additional dependencies.
Done
12:18:54 Build sandboxing disabled due to nsjail error.
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
12:18:54 dumpvars failed with: exit status 1
12:18:55 Build sandboxing disabled due to nsjail error.
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
12:18:55 dumpvars failed with: exit status 1
** Don't have a product spec for: 'lineage_FP4'
** Do you have the right repo manifest?
No such item in brunch menu. Try 'breakfast'
>> [Sat Feb 24 12:18:55 UTC 2024] Failed build for FP4
>> [Sat Feb 24 12:18:55 UTC 2024] Finishing build for FP4
>> [Sat Feb 24 12:18:55 UTC 2024] Cleaning source dir for device FP4
12:18:56 Entire build directory removed.
#### build completed successfully (1 seconds) ####
And I look at the device folder and the folder for fairphone is removed.
This is not the “book method” – if you had a clean environment then repo init was not run before repo sync — repo sync will not know what to do – may make it more difficult to see if / where docker fails and to interpret logs!
I imagine you never saw the message “repo sync has finished successfully.”
At this stage I keep notes of disk space used. The above full job might have added 120G used space.
Now to docker …
Repository synced!
Looking for dependencies in device/fairphone/FP4
device/fairphone/FP4 has no additional dependencies.
Done
The above looks good … but … extracting “e[31m” type bits which are warnings in colour we see …
build/make/core/product_config.mk:160: error: Can not locate config makefile for product "lineage_FP4"
12:18:55 dumpvars failed with: exit status 1
>> [Wed Feb 28 08:45:07 UTC 2024] Starting build for FP4, v1.19.1-s branch
08:45:14 Build sandboxing disabled due to nsjail error.
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
08:45:14 dumpvars failed with: exit status 1
File "/usr/local/bin/repo", line 51
def print(self, *args, **kwargs):
^
SyntaxError: invalid syntax
Device FP4 not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Found repository: android_device_fairphone_FP4
Default revision: lineage-19.1
Checking branch info
Checking if device/fairphone/FP4 is fetched from android_device_fairphone_FP4
Adding dependency: LineageOS/android_device_fairphone_FP4 -> device/fairphone/FP4
Using default branch for android_device_fairphone_FP4
Syncing repository to retrieve project.
Repository synced!
Looking for dependencies in device/fairphone/FP4
device/fairphone/FP4 has no additional dependencies.
Done
08:45:15 Build sandboxing disabled due to nsjail error.
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
08:45:15 dumpvars failed with: exit status 1
08:45:16 Build sandboxing disabled due to nsjail error.
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
08:45:16 dumpvars failed with: exit status 1
** Don't have a product spec for: 'lineage_FP4'
** Do you have the right repo manifest?
No such item in brunch menu. Try 'breakfast'
>> [Wed Feb 28 08:45:16 UTC 2024] Failed build for FP4
>> [Wed Feb 28 08:45:16 UTC 2024] Finishing build for FP4
>> [Wed Feb 28 08:45:16 UTC 2024] Cleaning source dir for device FP4
08:45:17 Entire build directory removed.
#### build completed successfully (1 seconds) ####
The following local_manifests files exist: /src/S/.repo/local_manifests/proprietary_gitlab.xml, /src/S/.repo/local_manifests/proprietary.xml and /src/S/.repo/local_manifests/roomservice.xml which reads:
The one line roomservice.xml is unexpected. At least a part of the problem here is that docker has failed to pull in a kernel line in .repo/local_manifests/roomservice.xml
This must be backed into the docker image, I see this, too
[quote="maximebellis, post:6, topic:55806"]
```
:18:52 dumpvars failed with: exit status 1
File "/usr/local/bin/repo", line 51
def print(self, *args, **kwargs):
^
SyntaxError: invalid syntax
```
[/quote]
I was getting the above errors starting build with the late Oktober/November version of the /e docker image.
Tried fixing a ton of things and I even reformatted the ssd and installed new.
[quote="maximebellis, post:4, topic:55806"]
`08:49:03 You are building on a machine with 15.4GB of RAM`
[/quote]
16Gb RAM and just a single 500Gb SSD; minimal Ubuntu 22.04 Server and Docker from snaps
Did work for me on my first ever build:
I might spin up my workstation on the weekend and run your device in docker just see.
>> [Thu Feb 29 12:25:57 UTC 2024] Starting build for FP4, v1.19.1-s branch
[W][2024-02-29T12:26:02+0000][27603] void cmdline::logParams(nsjconf_t *)():250 Process will be UID/EUID=0 in the global user namespace, and will have user root-level access to files
[W][2024-02-29T12:26:02+0000][27603] void cmdline::logParams(nsjconf_t *)():260 Process will be GID/EGID=0 in the global user namespace, and will have group root-level access to files
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
12:26:02 dumpvars failed with: exit status 1
File "/usr/local/bin/repo", line 51
def print(self, *args, **kwargs):
^
SyntaxError: invalid syntax
Device FP4 not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Found repository: android_device_fairphone_FP4
Default revision: lineage-19.1
Checking branch info
Checking if device/fairphone/FP4 is fetched from android_device_fairphone_FP4
Adding dependency: LineageOS/android_device_fairphone_FP4 -> device/fairphone/FP4
Using default branch for android_device_fairphone_FP4
Syncing repository to retrieve project.
Repository synced!
Looking for dependencies in device/fairphone/FP4
device/fairphone/FP4 has no additional dependencies.
Done
[W][2024-02-29T12:26:04+0000][27673] void cmdline::logParams(nsjconf_t *)():250 Process will be UID/EUID=0 in the global user namespace, and will have user root-level access to files
[W][2024-02-29T12:26:04+0000][27673] void cmdline::logParams(nsjconf_t *)():260 Process will be GID/EGID=0 in the global user namespace, and will have group root-level access to files
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
12:26:04 dumpvars failed with: exit status 1
[W][2024-02-29T12:26:05+0000][27742] void cmdline::logParams(nsjconf_t *)():250 Process will be UID/EUID=0 in the global user namespace, and will have user root-level access to files
[W][2024-02-29T12:26:05+0000][27742] void cmdline::logParams(nsjconf_t *)():260 Process will be GID/EGID=0 in the global user namespace, and will have group root-level access to files
In file included from build/make/core/config.mk:313:
In file included from build/make/core/envsetup.mk:312:
e[1mbuild/make/core/product_config.mk:160: e[31merror: e[0me[1mCan not locate config makefile for product "lineage_FP4".e[0m
12:26:05 dumpvars failed with: exit status 1
** Don't have a product spec for: 'lineage_FP4'
** Do you have the right repo manifest?
No such item in brunch menu. Try 'breakfast'
>> [Thu Feb 29 12:26:05 UTC 2024] Failed build for FP4
>> [Thu Feb 29 12:26:05 UTC 2024] Finishing build for FP4
>> [Thu Feb 29 12:26:05 UTC 2024] Cleaning source dir for device FP4
12:26:06 Entire build directory removed.
#### build completed successfully (1 seconds) ####
… ish in that the build went to 99%. But not tested in docker.
First thing to check – has docker put your FP4.xml into /.repo/local_manifests ?
… and / or has a wrong roomservice.xml been readded ?
Edit
Ah ! I think this is the problem … I do not see
-v "/srv/local_manifests:/srv/local_manifests"
in your new docker command.
Side issue -- fix as above first
I also noticed that while I wrote
You may need to adjust your paths to match your environment. Elsewhere you used a structure /srv/e/src – it may be that you should be careful to keep paths consistent, though I don’t expect this to be a problem
Great! If you were to do so perhaps you could checkout this detail within a docker environment (as, having abandoned docker, I cannot be sure what was present in my docker test).
We have both seen a single line roomservice.xml which reads
However using it (with repo sync) I am currently failing at the “writing build rules …” stage with
FAILED:
build/make/core/Makefile:61: error: overriding commands for target `out/target/product/FP4/vendor/bin/hw/android.hardware.biometrics.fingerprint@2.1-service',
previously defined at build/make/core/base_rules.mk:525
11:38:23 ckati failed with: exit status 1
I’ve done one more complete formatting of my SSD and started from scratch, with ~15h of repo download – but I can’t build anything anymore, neither any of my previous Samsung, nor Fairphone. So, sorry I can’t be of any help.
Will probably abandon the /e Docker, as it is lacking some basic information, like release notes, updated documentation, updated howto…
If you were to do so perhaps you could checkout this detail within a docker environment (as, having abandoned docker, I cannot be sure what was present in my docker test).