Comments (14)
Yeah, that's the 8.x
build (Codecov sometimes misbehaves, could not figure out why so far). The 7.x
builds are running on our internal CI servers and they have been completed 30 minutes now. Dockerhub refreshes itself quite slowly, I recommend pulling the image anyway via docker pull b2ihealthcare/snow-owl-oss:7.x
and checking its image ID. It should be f0a33069ff08
.
I hope this helps.
Cheers,
Mark
from snow-owl.
Hi @mattecasu,
Snow Owl is unaffected of this vulnerability (it uses a different logging library than log4j), but Elasticsearch as storage and search backend is somewhat vulnerable, RCE is not possible, but information leakage in certain cases yes, please refer to this site: https://xeraa.net/blog/2021_mitigate-log4j2-log4shell-elasticsearch/.
Whether you are running your Snow Owl instance in embedded mode or connecting it to an external Elasticsearch cluster, it is recommended to set the -Dlog4j2.formatMsgNoLookups=true
command-line argument for both Snow Owl (just in case) and Elasticsearch and restart the service(s). This eliminates the possibility of information leakage and mitigates the issue while we are in the process of releasing an official patch.
All three dev streams (6.x
, 7.x
and 8.x
) have received their respective patches already back in December before the holidays, but due to other issues (see #958) with the latest 7.16.2 Elasticsearch we couldn't release the 7.x
and 8.x
streams that time, only the 6.x
patch. Also, it is very likely that another Elasticsearch patch (7.16.3 that contains log4j 2.17.1) will be released soon and we have decided to wait for it before we perform an actual release.
The currently planned release date for Snow Owl 7.18.1 and 8.1.0 is the 21st of January.
Cheers,
Mark
from snow-owl.
Hi @mattecasu,
Snow Owl 7.18.1 is out, which depends on Elasticsearch 7.16.3 and resolves any potential security vulnerability around log4j2.
Please find the docker image here.
If you have any further questions or issues feel free to contact us.
Cheers,
Mark
from snow-owl.
Dear Mark,
thanks for the patched tags - we are having an issue when using 7.18.1, I suspect it's not compatible with the latest AWS ElasticSearch clusters version (that is on 7.10). It seems like it cannot contact the cluster or find the indexes ("cluster not available" and also Couldn't check the existence of ES index 'locks-lock'
).
[Are you aware of this? Or maybe we are not considering something]
By the way: I confirmed 7.18.0 instead worked fine (which is on ES 7.10)
from snow-owl.
Hi @mattecasu,
Do you have the full stack trace?
It could be a compatibility issue between the Elasticsearch 7.16.3 client and the AWS Elasticsearch 7.10 service.
Cheers,
Mark
from snow-owl.
Hi @cmark, here's the full stacktrace we're getting:
[2022-01-31 16:28:06.733] [Start Level: Equinox Container: 7e07ab0e-607f-4e6e-9464-35eeabc71548] INFO elastic-snowowl - Connecting to Elasticsearch cluster at 'https://our-es-url.eu-west-1.es.amazonaws.com', connect timeout: 2000 ms, socket timeout: 120000 ms.
[2022-01-31 16:28:07.222] [Start Level: Equinox Container: 7e07ab0e-607f-4e6e-9464-35eeabc71548] INFO index.locks - Preparing 'locks' indexes...
[2022-01-31 16:29:07.275] [Start Level: Equinox Container: 7e07ab0e-607f-4e6e-9464-35eeabc71548] ERROR snowowl - Couldn't check the existence of ES index 'locks-lock'.
com.b2international.index.IndexException: Couldn't check the existence of ES index 'locks-lock'.
at com.b2international.index.es.admin.EsIndexAdmin.exists(EsIndexAdmin.java:150)
at com.b2international.index.es.admin.EsIndexAdmin.create(EsIndexAdmin.java:171)
at com.b2international.snowowl.core.locks.DefaultOperationLockManager.<init>(DefaultOperationLockManager.java:85)
at com.b2international.snowowl.core.internal.locks.LockPlugin.preRun(LockPlugin.java:43)
at com.b2international.snowowl.core.setup.Plugins.preRun(Plugins.java:88)
at com.b2international.snowowl.core.SnowOwl.run(SnowOwl.java:291)
at com.b2international.snowowl.core.SnowOwl.run(SnowOwl.java:272)
at com.b2international.snowowl.core.SnowOwl.run(SnowOwl.java:260)
at com.b2international.snowowl.server.product.SnowOwlServerActivator.start(SnowOwlServerActivator.java:52)
at org.eclipse.osgi.internal.framework.BundleContextImpl$2.run(BundleContextImpl.java:814)
at org.eclipse.osgi.internal.framework.BundleContextImpl$2.run(BundleContextImpl.java:1)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.osgi.internal.framework.BundleContextImpl.startActivator(BundleContextImpl.java:806)
at org.eclipse.osgi.internal.framework.BundleContextImpl.start(BundleContextImpl.java:763)
at org.eclipse.osgi.internal.framework.EquinoxBundle.startWorker0(EquinoxBundle.java:1011)
at org.eclipse.osgi.internal.framework.EquinoxBundle$EquinoxModule.startWorker(EquinoxBundle.java:365)
at org.eclipse.osgi.container.Module.doStart(Module.java:605)
at org.eclipse.osgi.container.Module.start(Module.java:468)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel$2.run(ModuleContainer.java:1845)
at org.eclipse.osgi.internal.framework.EquinoxContainerAdaptor$1$1.execute(EquinoxContainerAdaptor.java:136)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1838)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1781)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.doContainerStartLevel(ModuleContainer.java:1743)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1665)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234)
at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:345)
Caused by: com.b2international.commons.exceptions.BadRequestException: Cluster at 'https://our-es-url.eu-west-1.es.amazonaws.com' is not available.
at com.b2international.index.es.client.EsClientBase.checkAvailable(EsClientBase.java:101)
at com.b2international.index.es.client.http.IndicesHttpClient.exists(IndicesHttpClient.java:60)
at com.b2international.index.es.admin.EsIndexAdmin.exists(EsIndexAdmin.java:148)
... 26 common frames omitted
(I slightly redacted our Elasticsearch URL)
from snow-owl.
Hi @aelred,
Could you please try the latest 7.x
image from Dockerhub (it should be available at any minute now)? That has a change that will print out the cluster diagnostic message next to the Cluster at 'https://our-es-url.eu-west-1.es.amazonaws.com' is not available.
error, so we can see what is going on under the hood and hopefully get a fix for it quickly.
Thank you,
Mark
from snow-owl.
Thanks Mark, we'll give it a try when it's pushed up!
It looks like the build failed though:
-> Pinging Codecov
https://codecov.io/upload/v4?package=github-action-1.0.6&token=<hidden>&package=github-action-1.0.6&token=&branch=8.x&commit=f1bc20cf61320f7162c7bb20853e5915e33fc699&build=1773785924&build_url=https%3A%2F%2Fgithub.com%2Fb2ihealthcare%2Fsnow-owl%2Factions%2Fruns%2F1773785924&name=&tag=&slug=b2ihealthcare%2Fsnow-owl&service=github-actions&flags=&pr=&job=Java%20CI&cmd_args=n,F,Q,Z
{'detail': ErrorDetail(string='Unable to locate build via Github Actions API. Please upload with the Codecov repository upload token to resolve issue.', code='not_found')}
404
==> Uploading to Codecov
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 575k 100 171 100 575k 1195 4025k --:--:-- --:--:-- --:--:-- 4027k
{'detail': ErrorDetail(string='Unable to locate build via Github Actions API. Please upload with the Codecov repository upload token to resolve issue.', code='not_found')}
Error: Codecov failed with the following error: The process '/usr/bin/bash' failed with exit code 1
https://github.com/b2ihealthcare/snow-owl/runs/5009331342?check_suite_focus=true
from snow-owl.
Oh you're right! Sorry, I was looking at the wrong thing.
With the new 7.x
we get: Invalid or missing build flavor [oss]
.
Here's the full stack trace:
[2022-01-31 18:18:52.724] [Start Level: Equinox Container: 70218979-5f75-4858-810c-033d1cf7d27d] INFO elastic-snowowl - Connecting to Elasticsearch cluster at 'https://our-es-url.eu-west-1.es.amazonaws.com', connect timeout: 2000 ms, socket timeout: 120000 ms.
[2022-01-31 18:18:53.030] [Start Level: Equinox Container: 70218979-5f75-4858-810c-033d1cf7d27d] INFO index.locks - Preparing 'locks' indexes...
[2022-01-31 18:19:53.041] [Start Level: Equinox Container: 70218979-5f75-4858-810c-033d1cf7d27d] ERROR snowowl - Couldn't check the existence of ES index 'locks-lock'.
com.b2international.index.IndexException: Couldn't check the existence of ES index 'locks-lock'.
at com.b2international.index.es.admin.EsIndexAdmin.exists(EsIndexAdmin.java:150)
at com.b2international.index.es.admin.EsIndexAdmin.create(EsIndexAdmin.java:171)
at com.b2international.snowowl.core.locks.DefaultOperationLockManager.<init>(DefaultOperationLockManager.java:85)
at com.b2international.snowowl.core.internal.locks.LockPlugin.preRun(LockPlugin.java:43)
at com.b2international.snowowl.core.setup.Plugins.preRun(Plugins.java:88)
at com.b2international.snowowl.core.SnowOwl.run(SnowOwl.java:291)
at com.b2international.snowowl.core.SnowOwl.run(SnowOwl.java:272)
at com.b2international.snowowl.core.SnowOwl.run(SnowOwl.java:260)
at com.b2international.snowowl.server.product.SnowOwlServerActivator.start(SnowOwlServerActivator.java:52)
at org.eclipse.osgi.internal.framework.BundleContextImpl$2.run(BundleContextImpl.java:814)
at org.eclipse.osgi.internal.framework.BundleContextImpl$2.run(BundleContextImpl.java:1)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.osgi.internal.framework.BundleContextImpl.startActivator(BundleContextImpl.java:806)
at org.eclipse.osgi.internal.framework.BundleContextImpl.start(BundleContextImpl.java:763)
at org.eclipse.osgi.internal.framework.EquinoxBundle.startWorker0(EquinoxBundle.java:1011)
at org.eclipse.osgi.internal.framework.EquinoxBundle$EquinoxModule.startWorker(EquinoxBundle.java:365)
at org.eclipse.osgi.container.Module.doStart(Module.java:605)
at org.eclipse.osgi.container.Module.start(Module.java:468)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel$2.run(ModuleContainer.java:1845)
at org.eclipse.osgi.internal.framework.EquinoxContainerAdaptor$1$1.execute(EquinoxContainerAdaptor.java:136)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1838)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1781)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.doContainerStartLevel(ModuleContainer.java:1743)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1665)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234)
at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:345)
Caused by: com.b2international.commons.exceptions.BadRequestException: Cluster at 'https://our-es-url.eu-west-1.es.amazonaws.com' is not available. Diagnosis: 'The cluster at 'https://our-es-url.eu-west-1.es.amazonaws.com' reported an error: 'Invalid or missing build flavor [oss]''
at com.b2international.index.es.client.EsClientBase.checkAvailable(EsClientBase.java:102)
at com.b2international.index.es.client.http.IndicesHttpClient.exists(IndicesHttpClient.java:60)
at com.b2international.index.es.admin.EsIndexAdmin.exists(EsIndexAdmin.java:148)
... 26 common frames omitted
from snow-owl.
Hello Mark,
we think it's probably this? elastic/elasticsearch#76091
The thing is that the AWS-provided ES is probably used by many (including us 😅), so it would be best for Snowowl to make sure the ES client adopted can query it (hopefully AWS will soon bump their version..).
As for log4j, one way is to:
- exclude any old log4j group
- include log4j-1.2-api (translates versione 1 to version 2)
- optionally include a bridge to something else, like
log4j-to-slf4j
from snow-owl.
Hi @mattecasu,
Interesting, according to the official Elasticsearch docs, it should be backward compatible with the older 7.x versions, but apparently that's not true.
Elastic has changed Elasticsearch's licensing from Apache license starting from 7.11.0, none of the other Elasticsarch hosting providers will ever going to upgrade their version above 7.10.2 due to that, because the new license forbids hosting like that. AWS also forked the project and created OpenSearch as an alternative, which is interesting and sad at the same time.
Anyway, we are going to debug this and hopefully get a fix for it sometime soon, in the meantime I recommend a rollback to 7.18.0 version which is able to communicate with your ES cluster without an issue and configure the mentioned -Dlog4j2.formatMsgNoLookups=true
to mitigate the security vulnerability around log4j.
I hope this helps, if you have any questions please let us know.
Cheers,
Mark
from snow-owl.
Thanks Mark, we'll apply the configuration for now!
from snow-owl.
Hello again Mark,
did you find a way to make the patched versions compatible with AWS's ElasticSearch? Just asking to understand if you have it on the roadmap.
from snow-owl.
Hi @mattecasu,
Not yet, we could fix it by altering the Elasticsearch source somehow but that could easily turn into a licensing issue which we would like to avoid. Feel free to open another ticket that tracks the compatibility of Snow Owl with the AWS Elasticsearch service, which only fixes the client-side part of the Log4j2 vulnerability, the AWS Elasticsearch service will still offer Elasticsearch 7.10.2, which is somewhat vulnerable to information leakage attacks (but not to RCE attacks).
I still recommend either switching to Elastic Cloud as Elasticsearch hosting solution or hosting the Elasticsearch service on your own. Please refer to the AWS documentation to see the supported Elasticsearch versions (up to Elasticsearch OSS 7.10).
I hope this helps. Feel free to reach out to us if you have any further questions.
Cheers,
Mark
from snow-owl.
Related Issues (20)
- Missing results in GET `/snomed-ct/v3/{path}/members` HOT 9
- SnowOwl & AWS ElasticSearch
- ArrayIndexOutOfBoundsException on Import HOT 3
- Can we control how the searching works on the search APIs? HOT 5
- Error launching snow owl HOT 7
- SNOMED CT International Edition import not working HOT 2
- hapi-fhir-cli for upload of custom terminology HOT 1
- Opensearch integration HOT 2
- Target Platform is not able to Resolve HOT 3
- snowowl and ES / Opensearch on AWS HOT 2
- Snow Owl doesn't start from source in Eclipse HOT 3
- Publishing PreRelease content (to OntoServer) HOT 1
- Publication of Live and Pre-Release content to OntoServer Terminology Server HOT 4
- Concept Search API semanticTag filter issue HOT 3
- Contribution by code smell removal HOT 1
- UNABLE TO IMPORT SNOMED AFTER INSTALL HOT 7
- Support the "url" search parameter for both CodeSystem and ValueSet resources HOT 4
- SNOMED CT Import not working HOT 9
- Import of FHIR ValueSet HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from snow-owl.