Jiotty on Raspberry Pi about jiotty-photos-uploader

[ylexus]

Hi. Last time I checked, the google photos library I am depending upon uses pre-built native binaries that they only ship for 64 bit Intel architecture. This is the reason Jiotty can’t run on anything else.

having said that, I have not checked their latest versions. Maybe they already support other architectures. I do have a Raspberry PI 4 at hand for my home automation, so can easily check. will do in the nearest future.

As for your run - this task runs UI, so nothing is printed to the console. All the logs to go ~/.jiottyphotosuploader/logs - check this dir. I am guessing it will contain the error related to the missing binary for arm.

[ylexus]

Just realised there was a typo in README.md. Plus --no-daemon is useful so that the build does not start the extra daemon process. So the proper command to run the UI is
./gradlew --no-daemon -DCLIENT_SECRET_PATH=/path/to/google-API-client-secret.json run

Just tried on RPi - it fails because there are no arm javafx libraries. javafx is the java UI library:

2021-06-21T23:18:46,142 ERROR [QuantumRenderer-0] u.o.l.s.c.LogLevel$5 Loading library prism_es2 from resource failed: java.lang.UnsatisfiedLinkError: /home/pi/.openjfx/cache/16/libprism_es2.so: /home/pi/.openjfx/cache/16/libprism_es2.so: wrong ELF class: ELFCLASS64 (Possible cause: can't load AMD 64 .so on a ARM platform)
2021-06-21T23:18:46,144 ERROR [QuantumRenderer-0] u.o.l.s.c.LogLevel$5 java.lang.UnsatisfiedLinkError: /home/pi/.openjfx/cache/16/libprism_es2.so: /home/pi/.openjfx/cache/16/libprism_es2.so: wrong ELF class: ELFCLASS64 (Possible cause: can't load AMD 64 .so on a ARM platform)

OK, we don't really need the UI on RPi, although it'll be a little more difficult to specify configuration without the UI. So I've tried building the binary (BTW just added command line to build it to README.md). Problem with this is that by default a build on linux tries to build both rpm and deb packages. So I temporarily patched app/build.gradle by adding
installerType='deb'
after
if (org.gradle.internal.os.OperatingSystem.current().linux) {

The build failed, but that's because I hard code the architecture in the name of the .deb file (can be fixed later)

Execution failed for task ':app:renameDistribution'.
> Warning: Could not find file /home/pi/java/jiotty-photos-uploader/app/build/jpackage/jiotty-photos-uploader_2.0.1-1_amd64.deb to copy.

The key point is that the binary is now ready in app/build/jpackage/JiottyPhotosUploader/bin/JiottyPhotosUploader. However, when I run it in the upload mode it seemed to be just hanging. It's getting very late here - will continue investigating later.

I had a few times where gradle daemon crashed during the build and a single occurrence of an integration test failing, probably because it timed out on a slower machine. So I had to retry a few times. An oh boy how slow the builds are one RPi compared to my MacBook Pro 2019 :)

[vicmortelmans]

Hi,

I hope it will work :)

Perhaps you had crashes because of lack of swap space? I have increased mine from 100MG to 1GB.

You refer to a log directory, but I don't see any? Why would that be?

Best regards,
Vic

[ylexus]

git pull and try again - I made a few fixes there. But I'd ignore the UI for now - looks like the official javafx libraries do not support armv7l. Googling finds https://webtechie.be/post/2020-04-08-installing-java-and-javafx-on-raspberry-pi/, which has a link to https://bell-sw.com/pages/downloads-embedded/ - they claim it contains javafx that works on RPi. However its latest version is 13. It may still work, but at least it will require detecting that we're on RPi and removing explicit dependency on official javafx in this case.

Anyway, my main priority now is to make CLI work.

[ylexus]

Alright, if you git pull now you'll be able to use instructions in README.md to build it on RPi. However, the UI still does not work, only command-line mode. The disadvantage is that the CLI will be using all the default settings (which may be fine with you). You can still update the config ~/.jiottyphotosuploader/data.json manually to customise settings, but I do not have documentation on it (and by default it's just empty). I'd prefer to try to make UI working first, so that one can use it to update the config. Maintaining the documentation on the config is something I'd prefer to avoid for now.

[ylexus]

Alright, UI now works too on RPi, was just a matter of using a proper JDK with RPi-compatible javafx (Full distribution of Liberica OpenJDK 16) - follow the updated steps in README.md.

While the binary works, I am not sure whether the resulting .deb installer will be self-sufficient to work without the installed JDK at all. I'll test that later.

[vicmortelmans]

Hi,

I still got a problem.

Here's the output for building a native binary:

$ ./gradlew --no-daemon -DCLIENT_SECRET_PATH=/home/pi/client_secret.json -PraspberryPi fullPackage -DVERSION=1.9.1

./gradlew: 39: cd: can't cd to "./
To honour the JVM settings for this build a single-use Daemon process will be forked. See https://docs.gradle.org/7.0.2/userguide/gradle_daemon.html#sec:disabling_the_daemon.
Daemon will be stopped at the end of the build 

> Configure project :app
Project :app => no module-info.java found
project ':app': Plugin name.remal.check-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...
project ':app': Plugin name.remal.check-dependency-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...
project ':app': Plugin name.remal.check-gradle-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...

> Task :app:ensureVersionReleaseReady
> Task :app:compileJava
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BaseItemState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BaseItemState.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BasePreferences[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BasePreferences.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BaseUploadState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BaseUploadState.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BasePathState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BasePathState.java
4 warnings

> Task :app:copyClientSecretIfDefined
> Task :app:processResources
> Task :app:classes
> Task :app:compileTestJava
> Task :app:processTestResources
> Task :app:testClasses
> Task :app:test

IntegrationTest > mergesPreExistingNonEmptyAlbumsWithSameNameAndReusesResultingAlbum() FAILED
    java.lang.AssertionError at IntegrationTest.java:919

53 tests completed, 1 failed

> Task :app:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:test'.
> There were failing tests. See the report at: file:///home/pi/jiotty-photos-uploader/app/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/7.0.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 7m 48s
7 actionable tasks: 7 executed

Here's the error report from app/build/reports/tests/text/index.html:

mergesPreExistingNonEmptyAlbumsWithSameNameAndReusesResultingAlbum()
java.lang.AssertionError: 
Expected: is <true>
     but: was <false>
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
	at net.yudichev.googlephotosupload.core.IntegrationTest.doExecuteUpload(IntegrationTest.java:919)
	at net.yudichev.googlephotosupload.core.IntegrationTest.mergesPreExistingNonEmptyAlbumsWithSameNameAndReusesResultingAlbum(IntegrationTest.java:263)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:135)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.base/java.lang.Thread.run(Thread.java:831)

If I do the run command, there's following output, up till a point where the process seems to stall, without any GUI being displayed:

$ ./gradlew --no-daemon -DCLIENT_SECRET_PATH=/home/pi/client_secret.json -PraspberryPi run
./gradlew: 39: cd: can't cd to "./
To honour the JVM settings for this build a single-use Daemon process will be forked. See https://docs.gradle.org/7.0.2/userguide/gradle_daemon.html#sec:disabling_the_daemon.
Daemon will be stopped at the end of the build 

> Configure project :app
Project :app => no module-info.java found
project ':app': Plugin name.remal.check-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...
project ':app': Plugin name.remal.check-dependency-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...
project ':app': Plugin name.remal.check-gradle-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...

> Task :app:compileJava
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BaseItemState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BaseItemState.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BasePreferences[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BasePreferences.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BaseUploadState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BaseUploadState.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BasePathState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BasePathState.java
4 warnings

> Task :app:copyClientSecretIfDefined
> Task :app:processResources UP-TO-DATE
> Task :app:classes
> Task :app:configJavafxRun

> Task :app:run
OpenJDK Server VM warning: You have loaded library /home/pi/.openjfx/cache/16/libprism_es2.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

[ylexus]

Did you do git pull before building and installed Full Liberica OpenJDK 16 - current URL is https://download.bell-sw.com/java/16.0.1+9/bellsoft-jdk16.0.1+9-linux-arm32-vfp-hflt-full.deb?
The test failure from your first build is most likely due to RPi being slow - I just pushed higher timeouts, try again.
RE the failure in your second run - can you please look at ~/.jiottyphotosuploader/log/main.log (I mentioned an incorrect path before - this one is correct) - what errors are there? Or attach it here for me to have a look. I used to get that warning about libprism_es2.so on when running against AdoptOpenJDK 14. What's the output of this command on your RPi? Should be as below:

pi@raspberrypi:~ $ ls -al /etc/alternatives/java
lrwxrwxrwx 1 root root 57 Jun 23 00:49 /etc/alternatives/java -> /usr/lib/jvm/bellsoft-java16-full-arm32-vfp-hflt/bin/java

[vicmortelmans]

Hi,

before my previous message, I had done a git pull. That was 6 hours ago. When I do another pull now, there's nothing new.

I reinstalled the jdk using the download link in your previous message. Originally I had installed it using these instructions, which is perhaps not taking the full jdk: https://bell-sw.com/pages/repositories/#apt

Just to make sure, here's the installation path:

$ ls -al /etc/alternatives/java
lrwxrwxrwx 1 root root 57 Jun 23 21:38 /etc/alternatives/java -> /usr/lib/jvm/bellsoft-java16-full-arm32-vfp-hflt/bin/java

When I do the binary build, I still get the test failure:

IntegrationTest > mergesPreExistingNonEmptyAlbumsWithSameNameAndReusesResultingAlbum() FAILED
    java.lang.AssertionError at IntegrationTest.java:919

When I do the run command, there seems to be progress. I very briefly see the GUI popping up, but then it crashes. The output:

$ ./gradlew --no-daemon -DCLIENT_SECRET_PATH=/home/pi/client_secret_545696670419-ovroj9d1s6o4tr9mvsg1amq5ekajdavd.apps.googleusercontent.com.json -PraspberryPi run
./gradlew: 39: cd: can't cd to "./
To honour the JVM settings for this build a single-use Daemon process will be forked. See https://docs.gradle.org/7.0.2/userguide/gradle_daemon.html#sec:disabling_the_daemon.
Daemon will be stopped at the end of the build 

> Configure project :app
Project :app => no module-info.java found
project ':app': Plugin name.remal.check-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...
project ':app': Plugin name.remal.check-dependency-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...
project ':app': Plugin name.remal.check-gradle-updates is deprecated: Use automatic dependency updates software like Renovate, Dependabot, etc...

> Task :app:compileJava
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BaseItemState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BaseItemState.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BasePreferences[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BasePreferences.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BaseUploadState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BaseUploadState.java
warning: Was unable to read source file for net.yudichev.googlephotosupload.core.BasePathState[com.sun.tools.javac.code.Symbol$ClassSymbol.class]: java.io.FileNotFoundException: net/yudichev/googlephotosupload/core/BasePathState.java
4 warnings

> Task :app:copyClientSecretIfDefined
> Task :app:processResources UP-TO-DATE
> Task :app:classes
> Task :app:configJavafxRun
> Task :app:run
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x61fe6f68, pid=8047, tid=8070
#
# JRE version: OpenJDK Runtime Environment (16.0.1+9) (build 16.0.1+9)
# Java VM: OpenJDK Server VM (16.0.1+9, mixed mode, serial gc, linux-arm)
# Problematic frame:
# C  [libglass.so+0x11f68]  destroy_and_delete_ctx(WindowContext*)+0x14
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/pi/jiotty-photos-uploader/app/hs_err_pid8047.log
#
# If you would like to submit a bug report, please visit:
#   https://bell-sw.com/support
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

> Task :app:run FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:run'.
> Process 'command '/usr/lib/jvm/bellsoft-java16-full-arm32-vfp-hflt/bin/java'' finished with non-zero exit value 134

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/7.0.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 5m 7s
5 actionable tasks: 4 executed, 1 up-to-date

When I look into the log, this seems the error that throws it off:

2021-06-23T20:42:00,905 INFO [main] n.y.g.u.UserInterface Waiting for 10 seconds until UI is initialized
2021-06-23T20:42:10,912 ERROR [main] n.y.j.c.a.Application Unable to initialize
java.lang.IllegalStateException: UI was not initialized in 10 seconds
        at com.google.common.base.Preconditions.checkState(Preconditions.java:510) ~[guava-30.1.1-jre.jar:?]
        at net.yudichev.googlephotosupload.ui.UserInterface.doStart(UserInterface.java:65) ~[main/:?]
        at net.yudichev.jiotty.common.inject.BaseLifecycleComponent.lambda$start$0(BaseLifecycleComponent.java:19) ~[jiotty-common-2.0.3-SNAPSHOT.jar:?]
        at net.yudichev.jiotty.common.lang.Locks.inLock(Locks.java:13) ~[jiotty-common-2.0.3-SNAPSHOT.jar:?]
        at net.yudichev.jiotty.common.inject.BaseLifecycleComponent.start(BaseLifecycleComponent.java:17) ~[jiotty-common-2.0.3-SNAPSHOT.jar:?]
        at net.yudichev.jiotty.common.app.Application.start(Application.java:172) ~[jiotty-common-2.0.3-SNAPSHOT.jar:?]
        at net.yudichev.jiotty.common.app.Application.start(Application.java:90) ~[jiotty-common-2.0.3-SNAPSHOT.jar:?]
        at net.yudichev.jiotty.common.app.Application.run(Application.java:141) [jiotty-common-2.0.3-SNAPSHOT.jar:?]
        at net.yudichev.googlephotosupload.ui.UiMain.main(UiMain.java:52) [main/:?]
        at net.yudichev.googlephotosupload.Main.main(Main.java:13) [main/:?]

When I then run again, the same happens, but now on the console the build is reported successful. Yet the same error is in the log.

[ylexus]

1. Integration test - apologies, the push got stuck in my IDE. Now pushed. Please try again. Is it RPi 4 or RPi 3 that you're running this on? I am running on RPi 4. If it's still failing, could you run ./gradlew --no-daemon --info -DCLIENT_SECRET_PATH=<whatever-your-secret-path-is> fullPackage -DVERSION=2.0.1 >build.txt 2>&1, wait until it fails again and attach build.txt here please? There will be no personal data there, it's just a build log.
2. run - the cause could be somewhere before the log extract that you posted. Could you attach the whole main.log please? Otherwise email it to me on [email protected].

[ylexus]

Made a few more fixes - it now builds the .deb fine, and that .deb installs and runs fine on my RPi. I'll soon release 2.0.0 with the RPi binary. The binary will not require and JDK installed, obviously, as it packages it all together. It even adds a shortcut to "Accessories"! :)
Would be interesting to know if you have any more issues. You can ignore the binary building one, as I'll release the binary soon anyway, but let's investigate why "run" does not work on your environment.

[ylexus]

On my RPi, I sometimes see an error in the log like in a submitted issue #115 when started via gradle run. If this is what you see to, then starting from binary is the only way.

[vicmortelmans]

Hi,

I have a Raspberry Pi 3.

Again making progress, but not there yet. The fullpackage build succeeded, after burning one of my cores for 30 minutes. The 150MB deb could be installed. Running jiotty just flashes a window that disappears immediately and writes nothing on the console, but I attach the main.log in this message.

main.log

I repeated the build while storing the output to build.txt, as well attached.

build.txt

Then I also did the 'run' again, same result: GUI flashing briefly, then nothing. Output is in run.txt.

run.txt

I hope you can make something out of it...

Best regards,
Vic

[ylexus]

The log just shows that the 10 seconds the app gives to initialise the UI is just not enough for the slow RPi3. I've just increased it to 30 and pushed - you'll have to rebuild again or try the faster "run" task again.

BTW if you add --build-cache to command line, it will increase the speed of subsequent builds.

Just an observation that fullpackage on RPi4 takes 9.5 mins.

Another hint: later, when you've done with building Jiotty for good, you can remove ~/.gradle directory with all the builds caches and downloaded dependencies.

[ylexus]

Released 2.0.0 with the RPi deb in it, so you don't have to build. If we discover any more issues, I'll do a patch release 2.0.1.

[vicmortelmans]

Hi,

I've rebuilt the deb and now Jiotty GUI is showing and interactive, but after a few moments it's gone again.

Same behaviour when I install from the released build.

main.log attached for both runs.

main.log_release_build.log
main.log_my_last_build.log

[ylexus]

Took me a while to figure this out. Th login dialog is supposed to show an integrated browser, using the native WebKit javafx library. That library is not included in Liberica JDK. https://bell-sw.com/pages/liberica-release-notes-16/ even says "Java FX Webkit - does not work." It worked on my RPi because I logged in from the command line. You can do this too - just run the command line version specifying the directory to upload.

Since we can't use integrated browser on RPi, I've made a small improvement to use the system browser in this scenario, so the UI login will work now too.

Released 2.0.1 - please try.

[vicmortelmans]

Hi,

Me again :)

The login request is now coming up in my browser, but it won't succeed. When I enter my credentials, the browser returns a screen saying "Error 401 (bad request) - The server cannot process the request because it is malformed. It should not be retried. That's all we know." and the app is frozen.

I tried this on the GUI and on the command-line executable. I also tried logging in to some other Google website on the Raspberry Pi, and that worked.

The console output of the command-line is not consistent. Here's the console output for three subsequent tries, each time with the above behaviour in the browser:

1209 pi:~ () $ /opt/jiotty-photos-uploader/bin/JiottyPhotosUploader -r /home/pi/Downloads/images
Please open the following address in your browser:
  https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=641898159424-0tmk9ngs1aog13ef0v4bg1njtnndj1c3.apps.googleusercontent.com&redirect_uri=http://localhost:8888/Callback&response_type=code&scope=https://www.googleapis.com/auth/drive.appdata%20https://www.googleapis.com/auth/photoslibrary
Attempting to open that address in the default browser now...
libEGL warning: DRI2: failed to authenticate
[1480:1480:0626/211000.220795:ERROR:viz_main_impl.cc(229)] Exiting GPU process due to errors during initialization
[1432:1598:0626/211001.883332:ERROR:object_proxy.cc(621)] Failed to call method: org.freedesktop.DBus.Properties.Get: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
[1432:1598:0626/211001.887363:ERROR:object_proxy.cc(621)] Failed to call method: org.freedesktop.UPower.GetDisplayDevice: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
[1432:1598:0626/211001.890538:ERROR:object_proxy.cc(621)] Failed to call method: org.freedesktop.UPower.EnumerateDevices: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
^CERROR: Unable to initialize java.lang.RuntimeException: com.google.inject.ProvisionException: Unable to provision, see the following errors:
1210 pi:~ () $ 
1210 pi:~ () $ /opt/jiotty-photos-uploader/bin/JiottyPhotosUploader -r /home/pi/Downloads/images
Please open the following address in your browser:
  https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=641898159424-0tmk9ngs1aog13ef0v4bg1njtnndj1c3.apps.googleusercontent.com&redirect_uri=http://localhost:8888/Callback&response_type=code&scope=https://www.googleapis.com/auth/drive.appdata%20https://www.googleapis.com/auth/photoslibrary
Attempting to open that address in the default browser now...
libEGL warning: DRI2: failed to authenticate
Opening in existing browser session.

^CERROR: Unable to initialize java.lang.RuntimeException: com.google.inject.ProvisionException: Unable to provision, see the following errors:
1210 pi:~ () $ /opt/jiotty-photos-uploader/bin/JiottyPhotosUploader -r /home/pi/Downloads/images
Please open the following address in your browser:
  https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=641898159424-0tmk9ngs1aog13ef0v4bg1njtnndj1c3.apps.googleusercontent.com&redirect_uri=http://localhost:8888/Callback&response_type=code&scope=https://www.googleapis.com/auth/drive.appdata%20https://www.googleapis.com/auth/photoslibrary
Attempting to open that address in the default browser now...
Opening in existing browser session.
^CERROR: Unable to initialize java.lang.RuntimeException: com.google.inject.ProvisionException: Unable to provision, see the following errors:

I've attached today's logfiles, which cover a number of tries in the GUI as well as on the command-line.

main.log.2021-06-26-1.log
main.log

[ylexus]

I am at loss with this one. I presume that when you run the console app, it opens the browser too? Can you please, instead of using that browser window, close the browser completely, copy the URL from the console (the one after "Please open the following address in your browser:") and paste it into a fresh browser instance?

If it still does not work, can you try going to Chromium browser settings, privacy and security, clear browsing data, time range=all time, and click "clear data" (I hope you don't mind it forgetting all your previous browsing data), then try launching Jiotty again?

Finally, if that does not work either, can you copy the URL from the browser URL field immediately after Jiotty opens the login URL but before you enter your credentials, and paste it here? All this Google OAuth thing does is an HTTP GET to google, and google calling the "redirect_uri" bad when the login is successful.

While typing this, I remembered having the same issue on RPi Chromium earlier, with another app of mine that uses similar login style with manual copy-pasting of the URL into a browser and a subsequent callback to localhost. I was not too concerned with this at that time as I was the only user of this app and also because it worked with another browsr. At some point it just started working again with Chromium. I did do a few RPi package updates in between. Can you check your Chromium version? Mine is Version 88.0.4324.187 (Official Build) Built on Raspbian , running on Raspbian 10 (32-bit).

[ylexus]

I understand that logging in to other Google services works, but this callback style OAuth is not really the same in terms of code path as logging in to web-based apps.

[vicmortelmans]

Hi,

none of the above worked. This was the browser version:

Version 86.0.4240.197 (Official Build) Built on Raspbian , running on Raspbian 10 (32-bit)

I upgraded to the same version as yours:

Version 88.0.4324.187 (Official Build) Built on Raspbian , running on Raspbian 10 (32-bit)

And now it works!

It's getting late now :) Tomorrow I'll check if I can configure Jiotty to use my own client secret and then I'll close this issue.

Thanks for all your support!

Best regards,
Vic

[ylexus]

Great! Seems to have been a bug in Chromium.

[vicmortelmans]

Hi,

configuring my own client secret seems to work as well.

I did a couple more tests with the command-line tool and although the pictures and albums are uploaded to Photos without problem, this error is displayed on the console before the program exits. Should I worry about this? Could it be related to the Raspberry Pi setup?

Best regards,
Vic

1227 pi:~ () $ /opt/jiotty-photos-uploader/bin/JiottyPhotosUploader -r /home/pi/Downloads/images
All uploads will use ORIGINAL quality and will count towards storage in your Google Account
Loading albums in cloud (may take several minutes)...
Scanning file system starting at roots [/home/pi/Downloads/images]...
Looking for files: completed 1
... done, 1 directories found that will be used as albums
Looking for files: completed; 4 succeeded, 0 failed
Loading albums in Google Photos: completed 26
Loading albums in Google Photos: completed 76
Loading albums in Google Photos: completed 126
Loading albums in Google Photos: completed 226
Loading albums in Google Photos: completed 261
Loading albums in Google Photos: completed 315
Loading albums in Google Photos: completed 386
... loaded 445 album(s) in cloud
Loading albums in Google Photos: completed; 445 succeeded, 0 failed
Reconciling 0 albums(s) with Google Photos, may take a bit of time...
Creating album [test-album] for path [/home/pi/Downloads/images/test-album]
Reconciling albums with Google Photos: completed; 1 succeeded, 0 failed
Scheduling upload of /home/pi/Downloads/images/test-album/551px-Tomb_of_Nakht_(7).jpg
Scheduling upload of /home/pi/Downloads/images/test-album/770px-Gato_enervado_pola_presencia_dun_can.jpg
Scheduling upload of /home/pi/Downloads/images/test-album/Cat_tongue_macro.jpg
Uploaded file /home/pi/Downloads/images/test-album/Cat_tongue_macro.jpg
Uploading media files: progress 25%
Scheduling upload of /home/pi/Downloads/images/test-album/TapetumLucidum.jpeg
Uploaded file /home/pi/Downloads/images/test-album/TapetumLucidum.jpeg
Uploading media files: progress 50%
Uploaded file /home/pi/Downloads/images/test-album/551px-Tomb_of_Nakht_(7).jpg
Uploading media files: progress 75%
Uploaded file /home/pi/Downloads/images/test-album/770px-Gato_enervado_pola_presencia_dun_can.jpg
Uploading media files: progress 100%
Adding to albums: completed; 1 succeeded, 0 failed
Uploading media files: completed; 4 succeeded, 0 failed
All done without fatal errors
ERROR: Exception in thread "upload-pool-0" java.lang.NullPointerException: Cannot invoke "java.util.Optional.map(java.util.function.Function)" because "this.limit" is null
ERROR: 	at net.yudichev.googlephotosupload.core.DriveSpaceTrackerImpl.refreshStatusDescription(DriveSpaceTrackerImpl.java:102)
ERROR: 	at net.yudichev.googlephotosupload.core.DriveSpaceTrackerImpl.lambda$afterUpload$6(DriveSpaceTrackerImpl.java:87)
ERROR: 	at net.yudichev.jiotty.common.lang.Locks.inLock(Locks.java:13)
ERROR: 	at net.yudichev.googlephotosupload.core.DriveSpaceTrackerImpl.afterUpload(DriveSpaceTrackerImpl.java:84)
ERROR: 	at net.yudichev.googlephotosupload.core.GooglePhotosUploaderImpl.lambda$createMediaItemsExecutor$36(GooglePhotosUploaderImpl.java:323)
ERROR: 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
ERROR: 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
ERROR: 	at java.base/java.lang.Thread.run(Unknown Source)
1228 pi:~ () $ 

[ylexus]

Is this error reproducible or random? Can you please try doing it in the UI and then send me the log? Hope is that it will also happen in the UI and then I’ll have a lot more context from the log. Thanks.

[vicmortelmans]

Hi,
it seems to happen every time.
The logfile is attached.
main.log

[ylexus]

OK so I found a bug (introduced together with #105) where any errors during the free google account space refresh are “swallowed” (instead of being displayed to the user). I’ve fixed the bug, which will allow you to see the root cause. I’ll push soon. The error is related to Google Drive API requests. Do you remember, when logging in, whether you allowed access to “app data on your google drive” or denied this access? If you denied, this would cause the issue. The permission only allows the app to read and write to its own data folder (the so-called Application Data folder: https://developers.google.com/drive/api/v3/about-auth) - the app can’t see any of your Drive files, so it’s safe.

[vicmortelmans]

As far as I remember, I just confirmed any permissions that were requested. 28 Jun 2021 12:15:53 Alexey Yudichev ***@***.***>:

…

OK so I found a bug (introduced together with #105[#105]) where any errors during the free google account space refresh are “swallowed” (instead of being displayed to the user). I’ve fixed the bug, which will allow you to see the root cause. I’ll push soon. The error is related to Google Drive API requests. Do you remember, when logging in, whether you allowed access to “app data on your google drive” or denied this access? If you denied, this would cause the issue. The permission only allows the app to read and write to its own data folder (the so-called Application Data folder: https://developers.google.com/drive/api/v3/about-auth) - the app can’t see any of your Drive files, so it’s safe. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub[#113 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AABHZDFLJKC7T3OJOTARYA3TVBDVHANCNFSM47COI6HA].

[ylexus]

OK then we need to see what exactly the problem is. I've just pushed the aforementioned improvement. Could you git pull latest sources and run the UI via the run task (as per README), and try uploading? it should fail now and display the summary of the error in the UI, and, most importantly, the full details of the error will be in the log. Please attach the log here.

We're getting there!

[vicmortelmans]

Now I get an error even before it attempts to upload pictures. Is that what you're looking for?

Logfile attached.
main_run.log

[ylexus]

Yes - it's now clear what the reason is. You're using your own Google project & client secret. You need to enable Google Drive API in your project - Google were helpful enough to even include the URL to go to in the error message!

This reminds me to update my instructions on the wiki to mention enabling the Google Drive API.

[ylexus]

Wiki updated. Let me know if there are any more issues or not. If not, I'll release a new version.

[vicmortelmans]

Added Drive API and the error is gone!

[ylexus]

Great collaboration, thanks! Will release 2.0.2 soon.

[ylexus]

2.0.2 is released with all the latest bugfixes