tormap / tormap

World map of Tor relays. Travel in time, filter and search.

TorMap

This project visualizes current and past public Tor relays on a world map. The backend regularly downloads descriptors from TorProject Archive and saves a processed version in a PostgreSQL database. The frontend displays the data on a world map and allows to filter and search for relays.

Development

The project is split into a backend and frontend part. The backend is written in Kotlin and the frontend in TypeScript. You can work on either part independently:

• Backend
• Frontend

Releases

We use Semantic Versioning and try to keep the frontend, backend, GitHub and Docker tags consistent. Releases can be found at:

• https://github.com/TorMap/tormap/releases
• https://hub.docker.com/u/tormap

tormap's Issues

Selection in Safari browser

These bugs occur in Safari browser version 15.2 on macOS. They cannot be reproduced using Chrome.

Reproduce family selection bug:

1. Open https://tormap.org/ in Safari and group by family
2. Select a family
3. Try deselecting the family
4. Now either a new family is falsely selected or the FamilySelectionDialog pops up.

Reproduce country selection bug:

1. Open https://tormap.org/ in Safari and group by country
2. Try selecting a country
3. Nothing happens

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): update dependency gradle to v8.10.1
• fix(deps): update material-ui monorepo (major) (@mui/icons-material, @mui/material, @mui/styles, @mui/x-date-pickers)

Pending Status Checks

These updates await pending status checks. To force their creation now, click the checkbox below.

• chore(deps): update dependency vite-plugin-checker to v0.8.0
• chore(deps): update plugin org.jetbrains.kotlin.jvm to v2
• chore(deps): update plugin org.jetbrains.kotlin.kapt to v2
• chore(deps): update plugin org.jetbrains.kotlin.plugin.allopen to v2
• chore(deps): update plugin org.jetbrains.kotlin.plugin.jpa to v2
• chore(deps): update plugin org.jetbrains.kotlin.plugin.spring to v2

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update frontend-patch (@emotion/react, @mui/icons-material, @mui/material, @mui/styles, eslint-plugin-react, jotai, nginx, typescript)
• chore(deps): update dependency eslint to v9
• chore(deps): update typescript-eslint monorepo to v8 (major) (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
• fix(deps): update dependency org.flywaydb:flyway-core to v10
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update plugin org.springframework.boot to v3
• fix(deps): update dependency date-fns to v3

Detected dependencies

dockerfile

frontend/Dockerfile

• node 20-alpine
• nginx 1.27.0-alpine

github-actions

.github/workflows/backend-docker.yml

• actions/checkout v4
• docker/login-action v3
• actions/setup-java v4
• gradle/gradle-build-action v3
• gradle/wrapper-validation-action v3

.github/workflows/backend-test.yml

• actions/checkout v4
• actions/setup-java v4
• gradle/gradle-build-action v3
• gradle/wrapper-validation-action v3

.github/workflows/frontend-build-deploy.yml

• actions/checkout v4
• FirebaseExtended/action-hosting-deploy v0
• FirebaseExtended/action-hosting-deploy v0
• FirebaseExtended/action-hosting-deploy v0

.github/workflows/frontend-docker.yml

• actions/checkout v4
• docker/setup-qemu-action v3
• docker/setup-buildx-action v3
• docker/login-action v3
• docker/metadata-action v5
• docker/build-push-action v6

gradle

backend/settings.gradle.kts

backend/build.gradle.kts

• org.jetbrains.kotlin.jvm 1.9.23
• org.jetbrains.kotlin.kapt 1.9.23
• org.jetbrains.kotlin.plugin.spring 1.9.23
• org.jetbrains.kotlin.plugin.allopen 1.9.23
• org.jetbrains.kotlin.plugin.jpa 1.9.23
• org.springframework.boot 2.7.18
• io.spring.dependency-management 1.1.4
• com.google.cloud.tools.jib 3.4.2
• org.springdoc:springdoc-openapi-ui 1.8.0
• org.springdoc:springdoc-openapi-kotlin 1.8.0
• com.fasterxml.jackson.module:jackson-module-kotlin 2.17.2
• org.postgresql:postgresql 42.7.4
• org.ehcache:ehcache 3.10.8
• org.flywaydb:flyway-core 8.5.13
• com.maxmind.geoip2:geoip2 4.2.0
• com.newrelic.telemetry:micrometer-registry-new-relic 0.10.0
• io.kotest:kotest-runner-junit5 5.9.1
• io.kotest:kotest-assertions-core 5.9.1
• io.kotest.extensions:kotest-extensions-spring 1.3.0
• org.testcontainers:testcontainers 1.20.1
• org.testcontainers:junit-jupiter 1.20.1
• org.testcontainers:postgresql 1.20.1
• io.mockk:mockk 1.13.12

gradle-wrapper

backend/gradle/wrapper/gradle-wrapper.properties

• gradle 8.10

npm

frontend/package.json

• @emotion/react 11.13.0
• @emotion/styled 11.13.0
• @mui/icons-material 5.16.0
• @mui/material 5.16.0
• @mui/styles 5.16.0
• @mui/x-date-pickers 5.0.20
• axios 1.7.4
• date-fns 2.30.0
• jotai 2.9.1
• leaflet 1.9.4
• leaflet.heat 0.2.0
• notistack 3.0.1
• react-infinite-scroller 1.2.6
• react-leaflet 4.2.1
• react-sliding-pane 7.3.0
• @types/geojson 7946.0.14
• @types/leaflet.heat 0.2.4
• @types/node 20.16.5
• @types/react-infinite-scroller 1.2.5
• @typescript-eslint/eslint-plugin 7.18.0
• @typescript-eslint/parser 7.18.0
• @vitejs/plugin-react-swc 3.7.0
• eslint 8.57.0
• eslint-plugin-jsx-a11y 6.10.0
• eslint-plugin-react 7.35.0
• typescript 5.5.3
• vite 5.4.3
• vite-plugin-checker 0.7.2
• node ^20
• yarn 4.4.1

---

• Check this box to trigger a request for Renovate to run again on this repository

Define config with application.yml instead of application.properties

• Replace application.properties with application.yml
• Use environment specific config (production/development)
• Disable H2 console in production

Add FamilySelectionTable sorting

https://mui.com/components/tables/

Fix redownload of descriptor files from archive

On Windows the descriptor files (33 GB) from the TorProject are currently always redownloaded on backend startup.

Rework Frontend to use custom hooks for handling settings

Create new Custom Hooks:

useSettings for geting an object with
current Settings, setSettings for changing current Settings and an eventhandler for changing current Settings

Investigate memory and disk usage

Run the backend over multiple hours, investigate and look for ways to optimize memory and disk usage.

Process recent descriptors hourly

https://metrics.torproject.org/collector/recent/relay-descriptors/consensuses/
https://metrics.torproject.org/collector/recent/relay-descriptors/server-descriptors/

Too many characters for DB field

Example error:

2023-04-26 09:15:03,782 WARN org.tormap.service.IpLookupService [http-nio-8080-exec-2] Location lookup for IP 10.0.0.2 with provider dbip failed! class com.maxmind.geoip2.exception.AddressNotFoundException: The address 10.0.0.2 is not in the database.
2023-04-26 09:15:03,783 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper [http-nio-8080-exec-2] SQL Error: 0, SQLState: 22001
2023-04-26 09:15:03,783 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [http-nio-8080-exec-2] ERROR: value too long for type character varying(255)
2023-04-26 09:15:03,783 INFO org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl [http-nio-8080-exec-2] HHH000010: On release of batch it still contained JDBC statements
2023-04-26 09:15:03,784 ERROR org.apache.juli.logging.DirectJDKLog [http-nio-8080-exec-2] Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception
org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:280)
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:233)
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:566)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:174)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.data.repository.core.support.MethodInvocationValidator.invoke(MethodInvocationValidator.java:99)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at com.sun.proxy.$Proxy150.save(Unknown Source)
	at org.tormap.adapter.HttpTraceRepository.saveTraces(HttpTracingRepository.kt:63)
	at org.tormap.adapter.HttpTraceRepository.add(HttpTracingRepository.kt:40)
	at jdk.internal.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at com.sun.proxy.$Proxy153.add(Unknown Source)
	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:96)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.authentication.ui.DefaultLogoutPageGeneratingFilter.doFilterInternal(DefaultLogoutPageGeneratingFilter.java:58)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:237)
	at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:223)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:223)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:217)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:112)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:82)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:891)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1784)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.hibernate.exception.DataException: could not execute statement
	at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:115)
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:37)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:200)
	at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:46)
	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3375)
	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3937)
	at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:107)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:604)
	at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:478)
	at java.base/java.util.LinkedHashMap.forEach(Unknown Source)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:475)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:344)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:40)
	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:107)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1407)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:489)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3303)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2438)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:449)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:183)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:40)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:281)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:562)
	... 107 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2676)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2366)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:356)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:496)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:413)
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190)
	at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:152)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197)
	... 128 common frames omitted

Log message about AS determination

Log messages about AS determination seem to have all a wrong count of 0.

E.g.
Determined the AS of 0 / 230 relays for month 2017-03

Add autogenerated id column to ApiTrace

Prevent unique index violations

Fix scheduled family updates

The relay families on the (live version) can currently only be viewed until 2021-10-02 although they should be updated regularly.

Typo in the infobox

with a view on technical-, social- and philosophical since.

Should be science

Check if it is worth upgrading to MUI v5.0

https://mui.com/blog/mui-core-v5/
https://www.npmjs.com/package/@mui/material

Add color picker for relay types

https://www.npmjs.com/package/material-ui-color-picker
https://mui.com/components/material-icons/?query=colo&selected=ColorLens

Do not lookup reserved IP adresses

Example log:
Location lookup for IP 10.0.0.2 with provider dbip failed! class com.maxmind.geoip2.exception.AddressNotFoundException: The address 10.0.0.2 is not in the database.

Add family actions

When a family is selected the user should have these actions available:

• Show list of relays belonging to this family (button) (maybe this could be always shown and when pressed, pull up all relays currently highlighted on screen)
• Show all families (button)

Use a HTTP library like Axios

• Evaluate which HTTP library to use
• Replace all fetch() calls with new HTTP library calls

Make frontend a Progressive Web App

https://create-react-app.dev/docs/making-a-progressive-web-app/

Refactor drawing of world map layers

• Move some code from useEffect() of world-map.tsx component to util
• Fix dependencies of useEffect() hook

Group relays by autonomous system

Rework about and donation page

SQL transaction deadlock

2024-05-20 04:10:41,677 INFO org.tormap.service.DescriptorCoordinationService [task-73] Finished 2024-05-19-12-05-00-server-descriptors with 0 errors
2024-05-20 04:10:41,853 INFO org.tormap.service.RelayDetailsUpdateService [task-62] ... Updating relay families for months: 2024-05
2024-05-20 04:10:41,859 INFO org.tormap.service.RelayDetailsUpdateService [task-73] ... Updating relay families for months: 2024-05
2024-05-20 04:25:39,397 INFO org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl [task-62] HHH000010: On release of batch it still contained JDBC statements
2024-05-20 04:25:39,425 ERROR org.hibernate.engine.jdbc.batch.internal.BatchingBatch [task-62] HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 1 update relay_details set address=('204.8.96.111'), allow_single_hop_exits=('FALSE'::boolean), autonomous_system_name=('QUINTEX'), autonomous_system_number=('62744'::int4), bandwidth_burst=('1073741824'::int4), bandwidth_observed=('14206093'::int4), bandwidth_rate=('1073741824'::int4), caches_extra_info=('FALSE'::boolean), circuit_protocol_versions=(NULL), contact=('email:john[]quintex.com url:https://www.quintex.com proof:uri-rsa abuse:john[]quintex.com keybase:aquintex twitter:aquintex hoster:Quintex Alliance Consulting uplinkbw:10000 btc:1Ab93nZzqBhhoTetUfzjXxpNjUoEQ84Jrt dnslocation:sameas dnsqname:y dnssec:y ...'), day=('2024-05-19 +00'), family_entries=('$0077BCBA7244DB3E6A5ED2746E86170066684887, $00DCAEAE3E54C32809E7F7CC4BF2A6FC68FC552F, $00EB3A12C77E70F19F66A7BBD094D62969AF350D, $041646640AB306EA74B001966E86169B04CC88D2, $139C86C4C9BC94E89BAF79B15EBFDF9396DD5BB0, $155D6F57425F16C0624D77777641E4EB1B47C6F0, $18E753214CD827244280BBD36AAC4F8E70B3EE8D, $1AE949967F82BBE7534A3D6BA77A7EBE1CED4369, $1BBCAEB763CA46DEB565A7251FAA6AC79A388ADD, $1DB25DF59DAA01B5BE3D3CEB8AFED115940EBE8B, $1E5136DDC52FAE1219208F0A6BADB0BA62587EE6, $21187DDD552605957B2EE5ABCBDA0E19251293ED, $2ED4D25766973713EB8C56A290BF07E06B85BF12, $2F81DA22A649EBB0CCD20A97527207E2E841F5F6, $30C55C496F5C7B9868E0BC6349A1CD5623F0B75F, $320D73AF6CC78987E710789847BFB8D61C31BD4B, $3268B4C7DC8D31C30FF14480409DD6D6B0702C9E, $3687FEC7E73F61AC66F7AE251E7DEE6BBD8C0252, $36D68478366CB8627866757EBCE7FB3C17FC1CB8, $3A1BC65DF03ECD50FDF7CFF9C5A4E049FCB9C1AF, $3CA0D15567024D2E0B557DC0CF3E962B37999A79, $3F91F384ABF6D115080638E52381231D1BFBFC84, $40E7D6CE5085E4CDDA31D51A29D1457EB53F12AD, $4316866D578B06DC79728883C8E5C9609F8311D8, $43209F6D50C657A56FE79AF01CA69F9EF19BD338, $4673A9741D2B7CF367CE3A6CE90C30E1820AAF9E, $4EBF459880000DAB6FDDAF0AE5122D0149097559, $4EE488AC0742BC6B747BB637A5635CE14E877F39, $501F5C4AACABEBFDC6A23543379F19B066396946, $521C30A968743B13E17E22E39BE52DA020FB92E1, $54A4820B46E65509BF3E2B892E66930A41759DE9, $5649CB2158DA94FB747415F26628BEC07FA57616, $5F4CD12099AF20FAF9ADFDCEC65316A376D0201C, $60D3667F56AEC5C69CF7E8F557DB21DDF6C36060, $654D634FC4281B16FAB7217BABDC3F179A8F2D29, $66E19E8C4773086F669A1E06A3F8C23B6C079129, $6748BA531097A930776F90E20B6EFBA3519A23C0, $7070199EF60B5B1AE4EA2EFB4881F9F90B6FA9EF, $764BF8A03868F84C8F323C1A676AA254B80DC3BF, $7A3DD280EA4CD4DD16EF8C67B93D9BDE184D1A81, $7B046CEA50092F9AB2F712D84F52E8FA00D83877, $7E6E9A6FDDB8DC7C92F0CFCC3CBE76C29F061799, $7FA8E7E44F1392A4E40FFC3B69DB3B00091B7FD3, $891F75C9EA906010BE9097D9573F72F462D88A19, $8A63E4CC86E4AFF54E07AF9D2340DFDFB8674312, $8B80169BEF71450FC4069A190853523B7AEA45E1, $90BF7147B422A1BABEFA503656EBD17987424441, $9314BD9503B9014261A65C221D77E57389DBCCC1, $9370F55D4BBF72FBDF13C41D65C8B814B4300CC8, $9C1E7D92115D431385B8CAEA6A7C15FB89CE236B, $9C2232ADECCEC6AE0C457D2ED3AE63425540C596, $9D21F034C3BFF4E7737D08CF775DC1745706801F, $9E2D7C6981269404AA1970B53891701A20424EF8, $9F2856F6D2B89AD4EF6D5723FAB167DB5A53519A, $A0DB820FEC87C0405F7BF05DEE5E4ADED2BB9904, $A2C3CB1520C75BEDB21244FD1DF1C371C26E959E, $A389C523BE3B29EA59C75AC557BF5CFB69586DCB, $A4A393FEF48640961AACE92D041934B55348CEF9, $A868303126987902D51F2B6F06DD90038C45B119, $A8AC5C1BA09DB94123E2BB990984B8A89E2896B0, $A8B887B3DC8B864E2F0B58F92B0C10D493B5AFFE, $A8BEB150F4BAD51BC6309ED93BFD40022A72A618, $A91F3E4D2CCD81F5C865E143525F92A790CDF17E, $AF57275D067ACF1EADE51E32860C8E569190BB24, $B028707969D8ED84E6DEA597A884F78AAD471971, $B0CD9F9B5B60651ADC5919C0F1EAA87DBA1D9249, $B2197C23A4FF5D1C49EE45BA7688BA8BCCD89A0B, $B6320E44A230302C7BF9319E67597A9B87882241, $B7047FBDE9C53C39011CA84E5CB2A8E3543066D0, $B7ECD9C6A910A170B55165742049CBCC777494F2, $C5A53BCC174EF8FD0DCB223E4AA929FA557DEDB2, $C78AFFEEE320EA0F860961763E613FD2FAC855F5, $CB7C0D841FE376EF43F7845FF201B0290C0A239E, $CC14C97F1D23EE97766828FC8ED8582E21E11665, $CC4A3AE960E3617F49BF9887B79186C14CBA6813, $D25210CE07C49F2A4F2BC7A506EB0F5EA7F5E2C2, $D33292FEDE24DD40F2385283E55C87F85C0943B6, $D3A1B7DEF370CBC6055F3FC540A518C8576D7570, $D4E585CE0E37A8B34F0D534CA396B97238B12BFA, $D6FF2697CEA5C0C7DA84797C2E71163814FC2466, $DA5C9A58DBD49E83821BC56FCBC8AD9209680CA3, $DA77AAFE0CC38DB958432B45910F47714D614AA8, $DAC0FD4D1A6D18ABF95AF248935EC56F46E5A921, $DAE94877199B925D5F79C2B00D7B834D16993AF9, $DD9849BD5CF3DC769FA2AEDBF9F15BF5B96A6957, $DE4F7A7B2DF8689B1F8D23ABA9E320D17638EAFD, $DF20497E487A979995D851A5BCEC313DF7E5BC51, $E3A491D490DC1C3832D7F68615EEB4508C857D8B, $E480D577F58E782A5BC4FA6F49A6650E9389302F, $EA7D575C91BB1F53C25510FBF91B87D81AE14646, $EABC2DD0D47B5DB11F2D37EB3C60C2A4D91C10F2, $EC15DB62D9101481F364DE52EB8313C838BDDC29, $EDCDF42475CD60A8EBDFA7CFE64B5006AB9FA783, $F0B60D25C2E83EA707C2886CFA59CDFE026A0D59, $F21DE9C7DE31601D9716781E17E24380887883D1, $F664E5E50B4D216E5940DA7E9CF653F5F9DC561B, $F7447E99EB5CBD4D5EB913EE0E35AC642B5C1EF3, $FAB96E0069596CACADCF1FCBA2407FCE505ED06D, $FDD700C791CC6BB0AC1C2099A82CBC367AD4B764, $FE00A3A835680E67FBBC895A724E2657BB253E97'), family_id=('40188730'::int8), fingerprint=('B0CD9F9B5B60651ADC5919C0F1EAA87DBA1D9249'), is_hibernating=('FALSE'::boolean), is_hidden_service_dir=('TRUE'::boolean), link_protocol_versions=(NULL), month=('2024-05'), nickname=('QuintexColinNSarah'), platform=('Tor 0.4.8.10 on Linux'), protocols=('Conflux (1), Cons (1, 2), Desc (1, 2), DirCache (2), FlowCtrl (1, 2), HSDir (2), HSIntro (4, 5), HSRend (1, 2), Link (1, 2, 3, 4, 5), LinkAuth (1, 3), Microdesc (1, 2), Padding (2), Relay (1, 2, 3, 4)'), tunnelled_dir_server=('TRUE'::boolean), uptime=('2721745'::int8) where id=('39686119'::int8) was aborted: ERROR: deadlock detected
  Detail: Process 10749 waits for ShareLock on transaction 12475738; blocked by process 10750.
Process 10750 waits for ShareLock on transaction 12475739; blocked by process 10749.
  Hint: See server log for query details.
  Where: while updating tuple (88028,73) in relation "relay_details"  Call getNextException to see other errors in the batch.], SQL: update relay_details set address=?, allow_single_hop_exits=?, autonomous_system_name=?, autonomous_system_number=?, bandwidth_burst=?, bandwidth_observed=?, bandwidth_rate=?, caches_extra_info=?, circuit_protocol_versions=?, contact=?, day=?, family_entries=?, family_id=?, fingerprint=?, is_hibernating=?, is_hidden_service_dir=?, link_protocol_versions=?, month=?, nickname=?, platform=?, protocols=?, tunnelled_dir_server=?, uptime=? where id=?
2024-05-20 04:25:39,426 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper [task-62] SQL Error: 0, SQLState: 40P01
2024-05-20 04:25:39,426 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [task-62] Batch entry 1 update relay_details set address=('204.8.96.111'), allow_single_hop_exits=('FALSE'::boolean), autonomous_system_name=('QUINTEX'), autonomous_system_number=('62744'::int4), bandwidth_burst=('1073741824'::int4), bandwidth_observed=('14206093'::int4), bandwidth_rate=('1073741824'::int4), caches_extra_info=('FALSE'::boolean), circuit_protocol_versions=(NULL), contact=('email:john[]quintex.com url:https://www.quintex.com proof:uri-rsa abuse:john[]quintex.com keybase:aquintex twitter:aquintex hoster:Quintex Alliance Consulting uplinkbw:10000 btc:1Ab93nZzqBhhoTetUfzjXxpNjUoEQ84Jrt dnslocation:sameas dnsqname:y dnssec:y ...'), day=('2024-05-19 +00'), family_entries=('$0077BCBA7244DB3E6A5ED2746E86170066684887, $00DCAEAE3E54C32809E7F7CC4BF2A6FC68FC552F, $00EB3A12C77E70F19F66A7BBD094D62969AF350D, $041646640AB306EA74B001966E86169B04CC88D2, $139C86C4C9BC94E89BAF79B15EBFDF9396DD5BB0, $155D6F57425F16C0624D77777641E4EB1B47C6F0, $18E753214CD827244280BBD36AAC4F8E70B3EE8D, $1AE949967F82BBE7534A3D6BA77A7EBE1CED4369, $1BBCAEB763CA46DEB565A7251FAA6AC79A388ADD, $1DB25DF59DAA01B5BE3D3CEB8AFED115940EBE8B, $1E5136DDC52FAE1219208F0A6BADB0BA62587EE6, $21187DDD552605957B2EE5ABCBDA0E19251293ED, $2ED4D25766973713EB8C56A290BF07E06B85BF12, $2F81DA22A649EBB0CCD20A97527207E2E841F5F6, $30C55C496F5C7B9868E0BC6349A1CD5623F0B75F, $320D73AF6CC78987E710789847BFB8D61C31BD4B, $3268B4C7DC8D31C30FF14480409DD6D6B0702C9E, $3687FEC7E73F61AC66F7AE251E7DEE6BBD8C0252, $36D68478366CB8627866757EBCE7FB3C17FC1CB8, $3A1BC65DF03ECD50FDF7CFF9C5A4E049FCB9C1AF, $3CA0D15567024D2E0B557DC0CF3E962B37999A79, $3F91F384ABF6D115080638E52381231D1BFBFC84, $40E7D6CE5085E4CDDA31D51A29D1457EB53F12AD, $4316866D578B06DC79728883C8E5C9609F8311D8, $43209F6D50C657A56FE79AF01CA69F9EF19BD338, $4673A9741D2B7CF367CE3A6CE90C30E1820AAF9E, $4EBF459880000DAB6FDDAF0AE5122D0149097559, $4EE488AC0742BC6B747BB637A5635CE14E877F39, $501F5C4AACABEBFDC6A23543379F19B066396946, $521C30A968743B13E17E22E39BE52DA020FB92E1, $54A4820B46E65509BF3E2B892E66930A41759DE9, $5649CB2158DA94FB747415F26628BEC07FA57616, $5F4CD12099AF20FAF9ADFDCEC65316A376D0201C, $60D3667F56AEC5C69CF7E8F557DB21DDF6C36060, $654D634FC4281B16FAB7217BABDC3F179A8F2D29, $66E19E8C4773086F669A1E06A3F8C23B6C079129, $6748BA531097A930776F90E20B6EFBA3519A23C0, $7070199EF60B5B1AE4EA2EFB4881F9F90B6FA9EF, $764BF8A03868F84C8F323C1A676AA254B80DC3BF, $7A3DD280EA4CD4DD16EF8C67B93D9BDE184D1A81, $7B046CEA50092F9AB2F712D84F52E8FA00D83877, $7E6E9A6FDDB8DC7C92F0CFCC3CBE76C29F061799, $7FA8E7E44F1392A4E40FFC3B69DB3B00091B7FD3, $891F75C9EA906010BE9097D9573F72F462D88A19, $8A63E4CC86E4AFF54E07AF9D2340DFDFB8674312, $8B80169BEF71450FC4069A190853523B7AEA45E1, $90BF7147B422A1BABEFA503656EBD17987424441, $9314BD9503B9014261A65C221D77E57389DBCCC1, $9370F55D4BBF72FBDF13C41D65C8B814B4300CC8, $9C1E7D92115D431385B8CAEA6A7C15FB89CE236B, $9C2232ADECCEC6AE0C457D2ED3AE63425540C596, $9D21F034C3BFF4E7737D08CF775DC1745706801F, $9E2D7C6981269404AA1970B53891701A20424EF8, $9F2856F6D2B89AD4EF6D5723FAB167DB5A53519A, $A0DB820FEC87C0405F7BF05DEE5E4ADED2BB9904, $A2C3CB1520C75BEDB21244FD1DF1C371C26E959E, $A389C523BE3B29EA59C75AC557BF5CFB69586DCB, $A4A393FEF48640961AACE92D041934B55348CEF9, $A868303126987902D51F2B6F06DD90038C45B119, $A8AC5C1BA09DB94123E2BB990984B8A89E2896B0, $A8B887B3DC8B864E2F0B58F92B0C10D493B5AFFE, $A8BEB150F4BAD51BC6309ED93BFD40022A72A618, $A91F3E4D2CCD81F5C865E143525F92A790CDF17E, $AF57275D067ACF1EADE51E32860C8E569190BB24, $B028707969D8ED84E6DEA597A884F78AAD471971, $B0CD9F9B5B60651ADC5919C0F1EAA87DBA1D9249, $B2197C23A4FF5D1C49EE45BA7688BA8BCCD89A0B, $B6320E44A230302C7BF9319E67597A9B87882241, $B7047FBDE9C53C39011CA84E5CB2A8E3543066D0, $B7ECD9C6A910A170B55165742049CBCC777494F2, $C5A53BCC174EF8FD0DCB223E4AA929FA557DEDB2, $C78AFFEEE320EA0F860961763E613FD2FAC855F5, $CB7C0D841FE376EF43F7845FF201B0290C0A239E, $CC14C97F1D23EE97766828FC8ED8582E21E11665, $CC4A3AE960E3617F49BF9887B79186C14CBA6813, $D25210CE07C49F2A4F2BC7A506EB0F5EA7F5E2C2, $D33292FEDE24DD40F2385283E55C87F85C0943B6, $D3A1B7DEF370CBC6055F3FC540A518C8576D7570, $D4E585CE0E37A8B34F0D534CA396B97238B12BFA, $D6FF2697CEA5C0C7DA84797C2E71163814FC2466, $DA5C9A58DBD49E83821BC56FCBC8AD9209680CA3, $DA77AAFE0CC38DB958432B45910F47714D614AA8, $DAC0FD4D1A6D18ABF95AF248935EC56F46E5A921, $DAE94877199B925D5F79C2B00D7B834D16993AF9, $DD9849BD5CF3DC769FA2AEDBF9F15BF5B96A6957, $DE4F7A7B2DF8689B1F8D23ABA9E320D17638EAFD, $DF20497E487A979995D851A5BCEC313DF7E5BC51, $E3A491D490DC1C3832D7F68615EEB4508C857D8B, $E480D577F58E782A5BC4FA6F49A6650E9389302F, $EA7D575C91BB1F53C25510FBF91B87D81AE14646, $EABC2DD0D47B5DB11F2D37EB3C60C2A4D91C10F2, $EC15DB62D9101481F364DE52EB8313C838BDDC29, $EDCDF42475CD60A8EBDFA7CFE64B5006AB9FA783, $F0B60D25C2E83EA707C2886CFA59CDFE026A0D59, $F21DE9C7DE31601D9716781E17E24380887883D1, $F664E5E50B4D216E5940DA7E9CF653F5F9DC561B, $F7447E99EB5CBD4D5EB913EE0E35AC642B5C1EF3, $FAB96E0069596CACADCF1FCBA2407FCE505ED06D, $FDD700C791CC6BB0AC1C2099A82CBC367AD4B764, $FE00A3A835680E67FBBC895A724E2657BB253E97'), family_id=('40188730'::int8), fingerprint=('B0CD9F9B5B60651ADC5919C0F1EAA87DBA1D9249'), is_hibernating=('FALSE'::boolean), is_hidden_service_dir=('TRUE'::boolean), link_protocol_versions=(NULL), month=('2024-05'), nickname=('QuintexColinNSarah'), platform=('Tor 0.4.8.10 on Linux'), protocols=('Conflux (1), Cons (1, 2), Desc (1, 2), DirCache (2), FlowCtrl (1, 2), HSDir (2), HSIntro (4, 5), HSRend (1, 2), Link (1, 2, 3, 4, 5), LinkAuth (1, 3), Microdesc (1, 2), Padding (2), Relay (1, 2, 3, 4)'), tunnelled_dir_server=('TRUE'::boolean), uptime=('2721745'::int8) where id=('39686119'::int8) was aborted: ERROR: deadlock detected
  Detail: Process 10749 waits for ShareLock on transaction 12475738; blocked by process 10750.
Process 10750 waits for ShareLock on transaction 12475739; blocked by process 10749.
  Hint: See server log for query details.
  Where: while updating tuple (88028,73) in relation "relay_details"  Call getNextException to see other errors in the batch.
2024-05-20 04:25:39,426 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [task-62] ERROR: deadlock detected
  Detail: Process 10749 waits for ShareLock on transaction 12475738; blocked by process 10750.
Process 10750 waits for ShareLock on transaction 12475739; blocked by process 10749.
  Hint: See server log for query details.
  Where: while updating tuple (88028,73) in relation "relay_details"
2024-05-20 04:25:39,554 ERROR org.tormap.service.RelayDetailsUpdateService [task-62] Could not update relay families for month 2024-05! could not execute batch; SQL [update relay_details set address=?, allow_single_hop_exits=?, autonomous_system_name=?, autonomous_system_number=?, bandwidth_burst=?, bandwidth_observed=?, bandwidth_rate=?, caches_extra_info=?, circuit_protocol_versions=?, contact=?, day=?, family_entries=?, family_id=?, fingerprint=?, is_hibernating=?, is_hidden_service_dir=?, link_protocol_versions=?, month=?, nickname=?, platform=?, protocols=?, tunnelled_dir_server=?, uptime=? where id=?]; nested exception is org.hibernate.exception.LockAcquisitionException: could not execute batch
2024-05-20 04:25:39,554 INFO org.tormap.service.RelayDetailsUpdateService [task-62] Finished updating relay families
2024-05-20 04:25:39,554 INFO org.tormap.service.RelayDetail

2024-05-20 06:12:42,989 INFO org.tormap.service.RelayDetailsUpdateService [task-68] ... Updating ASs for month: 2024-03
2024-05-20 06:12:43,060 INFO org.tormap.service.RelayDetailsUpdateService [task-68] Determined the AS of 0 / 152 relays for month 2024-03
2024-05-20 06:13:26,630 WARN com.zaxxer.hikari.pool.PoolBase [task-68] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@2d834f53 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-05-20 06:13:26,636 WARN com.zaxxer.hikari.pool.PoolBase [task-68] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@76bf0724 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-05-20 06:13:26,646 WARN com.zaxxer.hikari.pool.PoolBase [task-68] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@7be348f8 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-05-20 06:13:58,770 WARN com.zaxxer.hikari.pool.PoolBase [task-68] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@46419af5 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-05-20 06:13:58,775 WARN com.zaxxer.hikari.pool.PoolBase [task-68] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@63a66859 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-05-20 06:13:58,782 WARN com.zaxxer.hikari.pool.PoolBase [task-68] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@3be43cd4 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-05-20 06:13:58,788 WARN com.zaxxer.hikari.pool.PoolBase [task-68] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@71389f6f (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-05-20 06:14:00,130 INFO org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl [task-62] HHH000010: On release of batch it still contained JDBC statements
2024-05-20 06:14:00,133 ERROR org.hibernate.engine.jdbc.batch.internal.BatchingBatch [task-62] HHH000315: Exception executing batch [java.sql.BatchUpdateException: Batch entry 1 update relay_details set address=('202.59.9.238'), allow_single_hop_exits=('FALSE'::boolean), autonomous_system_name=('Gigabit Hosting Sdn Bhd'), autonomous_system_number=('55720'::int4), bandwidth_burst=('1073741824'::int4), bandwidth_observed=('807677'::int4), bandwidth_rate=('1073741824'::int4), caches_extra_info=('FALSE'::boolean), circuit_protocol_versions=(NULL), contact=(NULL), day=('2024-05-19 +00'), family_entries=('$1AE1CBDAAC8DF7130743710D8A0E598157873ED5, $4080222C8B944201B516AC5B145275E6ECEE4700, $9F3534DDA4865C0A88F3EF84BB5F1E6C35854379'), family_id=('40199178'::int8), fingerprint=('4080222C8B944201B516AC5B145275E6ECEE4700'), is_hibernating=('FALSE'::boolean), is_hidden_service_dir=('TRUE'::boolean), link_protocol_versions=(NULL), month=('2024-05'), nickname=('uknc'), platform=('Tor 0.4.8.11 on Linux'), protocols=('Conflux (1), Cons (1, 2), Desc (1, 2), DirCache (2), FlowCtrl (1, 2), HSDir (2), HSIntro (4, 5), HSRend (1, 2), Link (1, 2, 3, 4, 5), LinkAuth (1, 3), Microdesc (1, 2), Padding (2), Relay (1, 2, 3, 4)'), tunnelled_dir_server=('TRUE'::boolean), uptime=('518467'::int8) where id=('39683341'::int8) was aborted: ERROR: deadlock detected
  Detail: Process 10902 waits for ShareLock on transaction 12487314; blocked by process 10906.
Process 10906 waits for ShareLock on transaction 12487317; blocked by process 10902.
  Hint: See server log for query details.
  Where: while updating tuple (90855,47) in relation "relay_details"  Call getNextException to see other errors in the batch.], SQL: update relay_details set address=?, allow_single_hop_exits=?, autonomous_system_name=?, autonomous_system_number=?, bandwidth_burst=?, bandwidth_observed=?, bandwidth_rate=?, caches_extra_info=?, circuit_protocol_versions=?, contact=?, day=?, family_entries=?, family_id=?, fingerprint=?, is_hibernating=?, is_hidden_service_dir=?, link_protocol_versions=?, month=?, nickname=?, platform=?, protocols=?, tunnelled_dir_server=?, uptime=? where id=?
2024-05-20 06:14:00,133 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper [task-62] SQL Error: 0, SQLState: 40P01
2024-05-20 06:14:00,133 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [task-62] Batch entry 1 update relay_details set address=('202.59.9.238'), allow_single_hop_exits=('FALSE'::boolean), autonomous_system_name=('Gigabit Hosting Sdn Bhd'), autonomous_system_number=('55720'::int4), bandwidth_burst=('1073741824'::int4), bandwidth_observed=('807677'::int4), bandwidth_rate=('1073741824'::int4), caches_extra_info=('FALSE'::boolean), circuit_protocol_versions=(NULL), contact=(NULL), day=('2024-05-19 +00'), family_entries=('$1AE1CBDAAC8DF7130743710D8A0E598157873ED5, $4080222C8B944201B516AC5B145275E6ECEE4700, $9F3534DDA4865C0A88F3EF84BB5F1E6C35854379'), family_id=('40199178'::int8), fingerprint=('4080222C8B944201B516AC5B145275E6ECEE4700'), is_hibernating=('FALSE'::boolean), is_hidden_service_dir=('TRUE'::boolean), link_protocol_versions=(NULL), month=('2024-05'), nickname=('uknc'), platform=('Tor 0.4.8.11 on Linux'), protocols=('Conflux (1), Cons (1, 2), Desc (1, 2), DirCache (2), FlowCtrl (1, 2), HSDir (2), HSIntro (4, 5), HSRend (1, 2), Link (1, 2, 3, 4, 5), LinkAuth (1, 3), Microdesc (1, 2), Padding (2), Relay (1, 2, 3, 4)'), tunnelled_dir_server=('TRUE'::boolean), uptime=('518467'::int8) where id=('39683341'::int8) was aborted: ERROR: deadlock detected
  Detail: Process 10902 waits for ShareLock on transaction 12487314; blocked by process 10906.
Process 10906 waits for ShareLock on transaction 12487317; blocked by process 10902.
  Hint: See server log for query details.
  Where: while updating tuple (90855,47) in relation "relay_details"  Call getNextException to see other errors in the batch.
2024-05-20 06:14:00,133 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [task-62] ERROR: deadlock detected
  Detail: Process 10902 waits for ShareLock on transaction 12487314; blocked by process 10906.
Process 10906 waits for ShareLock on transaction 12487317; blocked by process 10902.
  Hint: See server log for query details.
  Where: while updating tuple (90855,47) in relation "relay_details"
2024-05-20 06:14:00,134 ERROR org.tormap.service.RelayDetailsUpdateService [task-62] Could not update relay families for month 2024-05! could not execute batch; SQL [update relay_details set address=?, allow_single_hop_exits=?, autonomous_system_name=?, autonomous_system_number=?, bandwidth_burst=?, bandwidth_observed=?, bandwidth_rate=?, caches_extra_info=?, circuit_protocol_versions=?, contact=?, day=?, family_entries=?, family_id=?, fingerprint=?, is_hibernating=?, is_hidden_service_dir=?, link_protocol_versions=?, month=?, nickname=?, platform=?, protocols=?, tunnelled_dir_server=?, uptime=? where id=?]; nested exception is org.hibernate.exception.LockAcquisitionException: could not execute batch
2024-05-20 06:14:00,134 INFO org.tormap.service.RelayDetailsUpdateService [task-62] Finished updating relay families

Investigate shorter maxLifetime value for Postgres DB connection

Frequently these warnings appear in the log
2024-01-07 18:23:36,480 WARN com.zaxxer.hikari.pool.PoolBase [http-nio-8080-exec-7] HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@811bf54 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.

Reading descriptors results in out of memory

2023-11-26 06:59:21,192 INFO org.tormap.service.TorDescriptorService [task-2475] Finished collecting descriptors from API path: /recent/relay-descriptors/server-descriptors/
2023-11-26 06:59:21,192 INFO org.tormap.service.TorDescriptorService [task-2475] ... Processing descriptors from API path /recent/relay-descriptors/server-descriptors/
2023-11-26 06:59:22,292 INFO org.tormap.service.TorDescriptorService [task-2474] Finished processing descriptors file 2023-11-26-06-00-00-consensus
2023-11-26 06:59:23,464 INFO org.tormap.service.TorDescriptorService [task-2474] Finished processing descriptors file 2023-11-25-10-00-00-consensus
2023-11-26 06:59:24,585 INFO org.tormap.service.TorDescriptorService [task-2474] Finished processing descriptors from API path: /recent/relay-descriptors/consensuses/
2023-11-26 06:59:25,490 INFO org.torproject.descriptor.index.DescriptorIndexCollector [task-2476] Finished descriptor collection.
2023-11-26 06:59:25,491 INFO org.tormap.service.TorDescriptorService [task-2476] Finished collecting descriptors from API path: /archive/relay-descriptors/consensuses/
2023-11-26 06:59:25,491 INFO org.tormap.service.TorDescriptorService [task-2476] ... Processing descriptors from API path /archive/relay-descriptors/consensuses/
2023-11-26 06:59:38,792 INFO org.tormap.service.TorDescriptorService [task-2475] Finished processing descriptors file 2023-11-26-06-04-24-server-descriptors
2023-11-26 06:59:39,160 INFO org.tormap.service.RelayDetailsUpdateService [task-2475] ... Updating relay families for months: 2023-11
2023-11-26 06:59:48,245 INFO org.tormap.service.RelayDetailsUpdateService [task-2480] Finished updating Autonomous System
2023-11-26 07:01:04,851 INFO org.torproject.descriptor.index.DescriptorIndexCollector [task-2477] Finished descriptor collection.
2023-11-26 07:01:04,851 INFO org.tormap.service.TorDescriptorService [task-2477] Finished collecting descriptors from API path: /archive/relay-descriptors/server-descriptors/
2023-11-26 07:01:04,852 INFO org.tormap.service.TorDescriptorService [task-2477] ... Processing descriptors from API path /archive/relay-descriptors/server-descriptors/
2023-11-26 07:01:07,725 ERROR org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable [Thread-396] Bug: uncaught exception or error while reading descriptors.
java.lang.OutOfMemoryError: Java heap space
2023-11-26 07:01:08,209 INFO org.tormap.service.TorDescriptorService [task-2477] Finished processing descriptors from API path: /archive/relay-descriptors/server-descriptors/
2023-11-26 07:01:27,273 ERROR org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable [Thread-395] Bug: uncaught exception or error while reading descriptors.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at org.torproject.descriptor.impl.DescriptorParserImpl.parseOneDescriptor(DescriptorParserImpl.java:263)
	at org.torproject.descriptor.impl.DescriptorParserImpl.parseOneOrMoreDescriptors(DescriptorParserImpl.java:240)
	at org.torproject.descriptor.impl.DescriptorParserImpl.detectTypeAndParseDescriptors(DescriptorParserImpl.java:61)
	at org.torproject.descriptor.impl.DescriptorParserImpl.parseDescriptors(DescriptorParserImpl.java:33)
	at org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.readTarball(DescriptorReaderImpl.java:304)
	at org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.readTarballs(DescriptorReaderImpl.java:254)
	at org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.run(DescriptorReaderImpl.java:145)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.reflect.InvocationTargetException: null
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.torproject.descriptor.impl.DescriptorParserImpl.parseOneDescriptor(DescriptorParserImpl.java:256)
	... 7 common frames omitted
Caused by: java.lang.OutOfMemoryError: Java heap space
2023-11-26 07:02:56,682 INFO org.tormap.service.TorDescriptorService [task-2476] Finished processing descriptors from API path: /archive/relay-descriptors/consensuses/

Switch infinite scrolling library

The current library https://github.com/danbovey/react-infinite-scroller seems to be unmaintained and does not work smoothly with the latest React versions anymore.

Improve caching behaviour

Improve caching of available days and single days. Possibly add Redis?
https://www.ehcache.org/

Restore default settings button

A button to restore the frontend settings to the default.
The button should only be visible, if the current settings are non-default.

Fix scrolling and scrollbars in dialogs

Add HTTP client request caching with ETags

https://www.javadevjournal.com/spring/etags-for-rest-with-spring/
https://developer.mozilla.org/de/docs/Web/HTTP/Caching

Fix cache update

Open Graph Metadata Tags

Open Graph Metadata Tags for Previews of the website if the link gets shared

Show node nicknames on hover

This could be done similar to how this project solved it: https://www.starlinkmap.org/

Within a certain proximity, the nickname of the nearest node can be shown and the node made clickable.

Black screen, when backend is not available on first reload

Improve Google Chrome Lighthouse score

Currently we get this score with tormap.org. Lighthouse already suggests many improvements, which we could implement.

Obvious areas of improvement are:

• Enable compressed API responses
• Reduce used npm libraries (e.g. moment.js and date-fns are both used)
• Enable Progressive Web App
• Avoid logging to browser console and especially avoid errors

Add more request caching

Cache the request for specific data as long as it surely did not change.

Renaming and restructure backend API

• Rename occurrences of GeoRelay and the likes to RelayLocation
• Rename occurrences of NodeDetails and the likes to RelayDetails
• Two new controllers (RelayLocationController, RelayDetailsController) and delete ArchiveDataController after migration

Migrate from H2 to PostgreSQL

Fix docker image tagging

Add secured restart endpoint

Buy me a coffee in about section

Action to display search results back onto map

See: https://news.ycombinator.com/item?id=40302644

Add backend tests

• Add Kotest with StringSpec
• Run tests in github workflow

Investigate caching providers and effective cache size

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: renovate.json
Error type: Invalid JSON (parsing failed)
Message: Syntax error near ],

Initial page load after longer period of no traffic seems to hit no cache

FireFox certificate issue

The backend server certificate seems to be revoked for FireFox.
Reproduce error:

1. https://api.tormap.org/
2. SEC_ERROR_REVOKED_CERTIFICATE

Add more relay details to dialog

• Confirmed family members (fingerprints/nicknames)
• Action to select family from relay details
• Verified/unverified domain names (reverse DNS lookup)

Inspiration: https://metrics.torproject.org/onionoo.html#details

Add admin access to logs and possibly more

https://codecentric.github.io/spring-boot-admin/2.5.1/#getting-started

Optimize UI for small screens

• All settings are usable
• Stats can be displayed when needed
• Dialogs can be properly viewed
• Update to MUI v5: https://mui.com/blog/mui-core-v5/