I have tried for the past few days to build the /e/ ROM.
I must miss something because until step 5 I have no problem, but at step 5 extract-files.sh doesn’t exist.
I have tried to download the code on efoundation’s gitlab: /e/devices/android_device_samsung_universal9810-common (I have a Samsung S9). It said the extract_utils.sh was missing so I downloaded this: /e/os/android_vendor_lineage/-/tree/v1-q.
I put them both in srv/e/src/version (to respect the paths in step 5). Now I have an error “$DEVICE must be set before including this script!”
I don’t know if I did what I was supposed to do, if I have to download other things, or if there is something else I missed…
It would be great if someone could help me understand what the problem is please.
As your device was officially supported by LineageOS up to Android 10/ Q lineage-17.1, the short answer is that you don’t need to run the extract-file.sh script.
All you need to do is include the following text in your docker run command: -e "INCLUDE_PROPRIETARY=true" . This will pickup the proprietary files needed to create the build by, downloading them from lineage-17.1 branch of TheMuppets proprietary_vendor_samsung GitHub repo . The files for this device are in the universal9810-common directory
The LineageOS for microGDocker has this enabled by default so you don’t need to set the INCLUDE_PROPRIETARY variable when building for devices supported by LineageOS and TheMuppets.
The /e/ Docker used also to have it enabled. However, /e/ decided a while ago
to make their own fork of The Muppets vendor repos, and use those forks in their /e/OS builds
not to make those forks available - or even visible - to anyone outside the /e/ dev team.
Doing this broke their public, community Docker build process and instructions. Instead of fixing that, (e.g. by adding the advice to set the INCLUDE_PROPRIETARY variable, or by making their forked repos visible / available for building), they chose to add the same ‘Run the extract-files.sh script’ text that LineageOS uses, despite the facts that
extracting vendor files in this way is tricky, buggy and often doesn’t work, in hard-to-diagnose ways.
they don’t do this when they make builds themselves. They use repos containing either extracted files or files copied from other organisations’ repos
Both LOS and /e/ do this for “legal reasons”, ignoring the fact that if using these files is breaking copyright law ( a big “if” - I don’t know of any organisation that has faced legal action for making copies of extracted vendor files), then they are breaking that law by using them in their own builds.
Anyway, sorry for the rant, but it annoys me when /e/ say that they support their community in making unofficial builds, at the same time as making it harder for the community to do just that. Good luck with your build!
You can increase the cache size limit by adding the following text to your docker run command: -e "CCACHE_SIZE=200G". You can experiment with sizes, but 50G is definitely too small. 200G or 300G work for me
Thank you for your answer.
I have been able to download completely the code (I think), but I still have an issue.
Here is what I have in the terminal and the logs:
The reason why the directory does not have the ./extract-files.sh script is because the directory is empty. It is fresh after having been created in Step 4. Create directories and then Step 5. Extract proprietary blobs directs the user to run this script, which is obviously not there.
It looks to me like the published procedure may be missing a step regarding populating that fresh directory with the necessary files.
The necessary files are usually found in the device tree source code for your device. That won’t be present on your build machine until EITHER
repo sync has been called, either manually or by the Docker instance OR
you manually git clone the repository containing the device tree code
The more usual way of getting proprietary blobs is to get them from a git repository such as TheMuppets github repos. That used to be done automatically by using the USE_PROPRIETARY flag in the docker run command. But /e/ chose to break this functionality when they decided to use their own forks of ‘TheMuppets’ repos, and then decided to make those forked repos accessible only to /e/ developers. This broke a build method that had previously worked for people making unofficial builds.
Instead of reverting that change, they changed the official 'HowTo Build` document to say, in effect “you’re on your own when it comes to the proprietary blobs”. And they did it in such a way that - as you have found - means that following the instructions does not work!
If there is a 'TheMuppets` repo for your manufaturer / device, then you can include it in your local manifest, and there will be no need to extract the files manually. Which is good because the instructions of extracting proprietary blobs (which were copied from the LineageOS Wiki pages) are out of date, and don’t actually work for many (most?) devices.
ah yes, as it sounds at “5.” to be without alternative to device- or even image-extract makes new builders struggle. A blob repo include is so much easier. To include their own muppets repo, users would need to understand (and supply) their own manifests though - that’s the way I do it
The README.md file from the /lineageos4microg/ Ci:CD repo is pretty good. The follwing lines can be lifted directly
Proprietary files
Some proprietary files are needed to create a LineageOS build, but they’re not included in the LineageOS repo for legal reasons. You can obtain these blobs in three ways:
by pulling them from a running LineageOS
by extracting them from a LineageOS ZIP
by downloading them from TheMuppets GitHub and GitLab repositories (unofficial)
Then we need to add some text about creating a manifest pointing to the relevant github (or gitlab) repo. The following should do the trick.
To use these files, you must have a local manifest that points to the TheMuppets repo containing the blobs for your device or manufacturer. Te following will include all the vendor blobs for all Sony devices (which will be very large, and will take a long time to sync)
Better is to find the repo that contains only the blobs for your device and include that in a manifest: the following will pull in the proprietary blobs for the Sony Xperia XZ3 (akatsuki) device
For some devices. particularly Xiaomi devices, the blobs are in gitlab not in github
If you are building for a device not officially supported by LineageOS, then they will not be in the TheMuppets repos. You will need to find the repo used by the developer of the unoffcial LineageOS ROM for that device. Not all ROM maintainers publish the proprietary blobs though. In which case yoy may have to try the first two methods listed above - ggod luck!
Maybe. They could easily have done it when the broke the build for community developers, but they chose not to bother That was the first time I got really annoyed with /e/'s attitude to “the community” that they claim to value!
that’s good, will you make it a merge request? A small disclaimer on the unofficial nature and potential risk of the repo and I think it will help first time builders alot.
One can give the xml any name and copy it over like any device / “roomservice” xml. Doesn’t need the buildscript env variable if the target path fits the device code name.
Putting clone-depth="1" in the project xml node will make it a shallow clone and only fetch binaries referenced by the ref / branch given.
Sorry, but no. I have already spent too much of my time trying to improve /e/'s documentation. I don’t have the time, patience or motivation to jump though the bureaucratic hoops involved in getting changes made. Feel free to use my words in a MR of your own
