/e/ OS "easy" installer: how to help packaging it to MacOS and Windows

Cause: org/openjfx/gradle/JavaFXPlugin has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

In fact I get the same error on a fresh copy of the installer.
And my suspicion was right: the default “Gradle JVM” is the problem. The one bundled with IntelliJ is used and for some reason that is only Java 8.
I’ll add a note to the troubleshooting howto.

For that mavenrepository.com problem: I get a 404 Not Found instead of 403 Forbidden when I try curl --head https://mvnrepository.com/org/yaml/snakeyaml/1.26/snakeyaml-1.26.pom
The Resource seems to be managed by Cloudflare which might explain it behaves differently for different users (regions).
But I do not see that error when running the installer in IntelliJ (with correct Gradle JVM). Maybe a 404 is handled “properly” while a 403 is not (just guessing, though).

I can build no problem from the command line. I’m investigating if I can debug that build using IntelliJ

I have some hints for you:

Error:

To solve:

Right click on project (in my case ei easy installer) > Open module settings

Project > Project SDK

I choose this JDK (Not sure if this is ok, but it works…)
4

Go to preferences, select gradle and select the j9-11 JVM.

Now i could build and run easy installer. Good luck.

1 Like

Today I learned (or remembered):

  1. IDEs are a waste of time, effort and motivation :slight_smile:
  2. Doing something else - e.g. go for a run - gives a fresh perspective on frustrating problems

I can build the installer using ./gradlew dist

If I run the built binary from the build directory, build/image/easy-installer-mac/bin/easy-installer it works as expected, detects a connected device, and reports that the device is not supported :slight_smile: . The interesting bits of the log are

    19:04:50.772 [JavaFX Application Thread] DEBUG ecorp.easy.installer.EasyInstaller -
    OS name = Mac OS X
    Java Home = /Users/pete/easy-installer/build/image/easy-installer-mac
    Current working dir = /Users/pete/easy-installer
    ADB folder path = /Users/pete/easy-installer/build/image/easy-installer-mac/bin/adb/
    ...
    19:05:10.608 [Thread-5] INFO  e.e.i.tasks.DeviceDetectionTask - runADBDevicesCmd(/Users/pete/easy-installer/build/image/easy-installer-mac/bin/adb/adb)
    19:05:10.610 [Thread-5] DEBUG ecorp.easy.installer.models.Command - getFinalCmd(), Splitted command =  /Users/pete/easy-installer/build/image/easy-installer-mac/bin/adb/adb devices -l
    19:05:10.618 [Thread-5] INFO  ecorp.easy.installer.models.Command - Command's Process started
    19:05:10.624 [Thread-5] DEBUG ecorp.easy.installer.models.Command -
      (debug)List of devices attached

Then try with the build zip file

 cp build/distributions/easyInstaller-mac.zip ~/temp/
 cd ~/temp
 unzip easyInstaller-mac.zip
 easy-installer-mac/bin/easy-installer

this hangs as before

 ...
 OS name = Mac OS X
Java Home = /Users/pete/temp/easy-installer-mac
Current working dir = /Users/pete/temp
ADB folder path = /Users/pete/temp/easy-installer-mac/bin/adb/
...
18:36:44.903 [JavaFX Application Thread] INFO  e.e.i.c.s.DeviceDetectedController - startDetection()
18:36:44.922 [Thread-4] INFO  e.e.i.tasks.DeviceDetectionTask - runADBDevicesCmd(/Users/pete/temp/easy-installer-mac/bin/adb/adb)
18:36:44.928 [Thread-4] DEBUG ecorp.easy.installer.models.Command - getFinalCmd(), Splitted command =  /Users/pete/temp/easy-installer-mac/bin/adb/adb devices -l

Next I try and run using using gradelw ./gradlew run. This also fails at the same point

18:43:40.852 [Thread-4] INFO  e.e.i.tasks.DeviceDetectionTask - runADBDevicesCmd(/Users/pete/easy-installer/buildSrc//adb/adb)
18:43:40.858 [Thread-4] DEBUG ecorp.easy.installer.models.Command - getFinalCmd(), Splitted command =  /Users/pete/easy-installer/buildSrc//adb/adb devices -l

(note in that instance there are two\ characters in the adb path, but that isn’t always the case)

In the cases where it fails, it is in the Command.execAndReadOutput() method, somewhere in the following lines (after the call to getFinalCmd(), before the logger call

             pb = new ProcessBuilder(getFinalCmd());
        }
        pb.redirectErrorStream(true);
        pc= pb.start();
        logger.info("Command's Process started");

I can add some logger commands to determine exactly where, but I suspect it will be in the pb.start() call and that it’s starting the adb process that never returns.

Not sure how much further I can take this because a: it sometimes works and b: once beyond this point, I need a supported phone otherwise I won’t be able to test the rest of the functionality. So I’ll have another quick look at IDEA in the light of your most recent post - thanks - and then I’ll explore the How to support a new device topic

1 Like

Thanks but no joy. When I build, it succeeds. When I run or debug if fails building EasyInstaller.Main. I don’t understand what the IDE is trying to do, or how it works out the project structure, or why one type of build works and another fails. And debugging IDE configurations is not a class of problem that interests me :slight_smile: |'ll let someone who has some idea of how it is supposed to work take it forward from here.

As I said in my earlier post, I’ll carry on by looking at how to support a new device

Did you run it here:
afbeelding

No. I hadn’t spotted that. I had been trying to run and debug in some other way. I’m setting breakpoints and looking at data as we speak :slight_smile:

Thanks again

Did you try this:

It was all it took for me …

The easy installer did not work, i found an issue with finding the OS, on my mac it was Mac with uppercase, and thats why it could not find scripts.
Made a merge request to fix that here

I ran the installer up to the point it wants to flash my FP3, but i cant do that because i’m on dev build and flashing stable would cause to have reinstall my phone which i don’t have time for at the moment… I think flashing would also work.

Yes. The build task runs, but nothing gets built, because the compileJava task has no source.

22:29:52: Executing tasks ':classes :testClasses'...

Starting Gradle Daemon...
Gradle Daemon started in 3 s 512 ms
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy NO-SOURCE
> Task :buildSrc:processResources NO-SOURCE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE

> Configure project :
Found module name 'ecorp.easy.installer'

> Task :wrapper

BUILD SUCCESSFUL in 22s
1 actionable task: 1 executed
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy NO-SOURCE
> Task :buildSrc:processResources NO-SOURCE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE

> Configure project :
Found module name 'ecorp.easy.installer'

> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava NO-SOURCE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE

BUILD SUCCESSFUL in 4s
2 actionable tasks: 2 up-to-date
22:30:24: Tasks execution finished ':classes :testClasses'.

And I can’t see any way of running whatever it thinks it has built successfully. But I can build and run from the command line using gradlew without changing any source code, but debugging is difficult with no IDE.

Like I said, I have no idea what the IDE is trying to do. I’m glad you are able to build and debug, and that you are able to help the developers build and package the installer for Mac. I’m not helping, so I’ll find something else to do :slight_smile:

In the debugger, this seems to be the problem: this exception is being thrown

th (slot_1) = {IOException@3504} 
 detailMessage = "Cannot run program "/Users/pete/IdeaProjects/easy-installer/buildSrc//adb/adb": error=2, No such file or directory"
 walkback = {long[9]@3521} [861223099, 861222757, 140655803440855, 140655803411529, 140655803411391, 140655803412321, 140655803428845, 857097629, 853157001]
 cause = {IOException@3522} 
 stackTrace = null
 suppressedExceptions = {Collections$EmptyList@3523} 
 disableWritableStackTrace = false

In the debugger, this seems to be the problem: this exception is being thrown

Out of curiosity: how did you run the debugger (I think I understood it doesn’t work in the IDE for you)?

detailMessage = "Cannot run program "/Users/pete/IdeaProjects/easy-installer/buildSrc//adb/adb"

There you have it :slight_smile:
It should be /Users/pete/IdeaProjects/easy-installer/buildSrc/OSX/adb/adb

As Andre said, it should work with his fix at https://gitlab.e.foundation/e/tools/easy-installer/-/merge_requests/85/diffs

Last night I hadn’t spotted the tiny box that lets you run and debug via gradle, despite @andrelam’s helpful hint

So two days for me to learn how to drive the IDE, ten minutes to track down a problem that’s already been found and fixed. Time for me to get outside and do something less frustrating (i.e. a game of golf) :slight_smile:

This doesn’t explain why the program hangs when I run the easy-installer file extracted from the built zip file. In that case we see that

OS name = Mac OS X
Java Home = /Users/pete/temp/easy-installer-mac
Current working dir = /Users/pete/temp
ADB folder path = /Users/pete/temp/easy-installer-mac/bin/adb/

which all looks correct. The command that hangs also looks correct
/Users/pete/temp/easy-installer-mac/bin/adb/adb devices -l
and if I run that command from the command line, outside of the app, it works as described.

I’m not sure how to go about debugging this one: I’ll try to investigate attaching the IDE debugger to a running process, and maybe some extra debug logging to find out more about what’s going on

1 Like

I have the same issue when creating the package with ./gradlew dist But when i run from within the IDE it does detect the phone, and seem to work… These issues are very annoying. Nothing to do with the actual work that needs to be done…

The first thing that comes to mind if the installer works when running from within the IDE but doesn’t work after ./gradlew dist is that in the IDE it is run with the JDK configured in the project settings (Adopt JDK from Andre’s screenshot) while in the distribution package it is run with the bundled JDK from buildSrc/OSX/jdk-11.0.2
But then I see no obvious reason why/how that would explain the observed behaviour.

Indeed. I don’t plan to spend much time on it as it is not clear what is the ‘official’ way of making production builds. No point debugging the gradlew builds if production builds are made a different way. I plan to raise a defect and let the dev team handle it. Before I do, I will just check whether the easy-installer file in the easyInstaller-linux-x64.zip built by gradlew works OK on a linux machine. If I has a Windows machine, I’d check the easy-installer.bat file too.

If the dev team make available a ‘production’ version of the installer for Mac, built the official way, I’ll happily spend some time checking that.

Do you know where the production builds of the installer are made available for download? I assume that Mac versions aren’t there, but it might be worth a quick look.

1 Like

Up to now, I have been focussing on building (and testing) the installer on/for MacOS, rather than packaging. Currently, building with gradlew seems to work, though there are some problems with the built installer, which are being debugged, and defects raised and fixed.

There hasn’t been much discussion about packaging and delivering the built installer for Mac.

What currently happens with gradlew is that zip files are produced for Mac, Windows and Linux. If we look at the GitLab CI configuration for building the easy installer, we can see that

  • for linux, it seems that gradlew, and the linux zip file its builds are not used (I may be wrong about that) and the CI packages the built installer and its dependencies into a snap.
  • for Windows, the CI uses gradlew and makensis to build an executable installer Easy-installer-setup.exe

What needs to be decided for MacOS, is how the installer should be packaged for delivery and installation. I think the following are both possible, and there may be other ways that I haven’t thought of :slight_smile: :

  1. Deliver the easyInstaller-mac.zip file that gradlew currently produces. Users will need to unpack the zip file, and run the extracted bin\easy-installer
  2. Package the installer and its dependencies into a MacOS App. User runs the app, and the app runs the ‘easy-installer’. The app could be delivered either
  • as a disk image (.dmg) file, that the user mounts, and then either drags the App into the Applications directory and runs it from there, or runs the app from the mounted disk image

  • as a Package (.pkg) file that is run by the package installer.

    Both these methods are standard was of delivering Mac apps, and Mac users will be familiar with them both.And both can be automated and included in the CI process. However, whatever method is used for delivering the app, creating an app and the means of delivering it, involves more work (specification, design, implementation and testing, if it’s going to be done properly :wink: ) than delivering the zip file created by gradlew.

Given that /e/ don’t seem to be in a position to allocate any development time or resources to delivering a Mac installer, then the simplest course - i.e. delivering the zip file built by gradlew - seems to be the best bet. Even with that solution, there needs to be ‘official’ involvement from /e/ in:

  • approving (or otherwise) the decision on how to package and deliver the installe
  • deciding what level of protection and integrity checking is necessary so that the user knows that the zip file and its contents they are unpacking and running (or the app they are installing and running) are the same as those built and delivered by /e/;
  • including the Mac installer in their CI process, and making it available on the /e/ website.

Neither of these things can realistically (in my opinion at least) be done by ‘the community’.

Thoughts?

1 Like

Thanks for your hard work and now hoping /e/ dev team helps out.

1 Like

See issue #2886. I haven’t tried the linux version, ad production builds aren’t delivered this way. I may try that tomorrow if it rains :slight_smile:

I have spent some time looking into this, and have added a note to the gitlab issue.

To move forward, I could really do with some help and advice, particularly with using gradle about which I know absolutely nothing very little. If you can offer any advice, please respond in gitlab.

Thanks

1 Like