Comments (36)
@sarbull @lpalashevski We need to change the default port for the docker container produced by egeria UI to > 1024, and ideally 8080 to keep it the same as the previous UI
I am happy to make change or you can. This requires a change to Dockerfile & etc/nginx.conf. I don't think anything else. Happy to make this change and test in k8s if you like.
from egeria-ui.
Confirmed on openshift dashboard
- 3.x container is using port 8080, 4.x container is using 80
- the pod definition in both cases (same chart!) is using 8080
from egeria-ui.
@sarbull I'll need a new container image published for egeria-ui with those fixes before it makes sense to merge the chart update.
from egeria-ui.
Docker image for egeria-ui v4.1.2 build running now https://github.com/odpi/egeria-ui/actions/runs/4546319965/jobs/8014803693?pr=572
from egeria-ui.
The only activity seen in the nginx pod is
127.0.0.1 - - [23/Sep/2022:14:26:48 +0000] "GET /nginx_status/ HTTP/1.1" 200 640 "-" "Sysdig Agent/1.0"
127.0.0.1 - - [23/Sep/2022:14:26:49 +0000] "GET /nginx_status/ HTTP/1.1" 200 640 "-" "Sysdig Agent/1.0"
127.0.0.1 - - [23/Sep/2022:14:26:50 +0000] "GET /nginx_status/ HTTP/1.1" 200 640 "-" "Sysdig Agent/1.0"
172.17.57.135 - - [23/Sep/2022:14:26:50 +0000] "GET / HTTP/1.0" 200 640 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15"
172.17.57.135 - - [23/Sep/2022:14:26:50 +0000] "GET /manifest.json HTTP/1.0" 200 519 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15"
172.17.57.135 - - [23/Sep/2022:14:26:50 +0000] "GET /static/css/main.8a4273b5.css HTTP/1.0" 200 190018 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15"
172.17.57.135 - - [23/Sep/2022:14:26:50 +0000] "GET /static/js/main.d3c2741a.js HTTP/1.0" 200 3873780 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15"
172.17.57.135 - - [23/Sep/2022:14:26:50 +0000] "GET /favicon.ico HTTP/1.0" 200 15406 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15"
172.17.57.135 - - [23/Sep/2022:14:26:50 +0000] "GET /egeria-logo.svg HTTP/1.0" 200 6658 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15"
127.0.0.1 - - [23/Sep/2022:14:26:51 +0000] "GET /nginx_status/ HTTP/1.1" 200 640 "-" "Sysdig Agent/1.0"
127.0.0.1 - - [23/Sep/2022:14:26:52 +0000] "GET /nginx_status/ HTTP/1.1" 200 640 "-" "Sysdig Agent/1.0"
Whilst the most recent log from the backend UI container ends:
2022-09-23 14:14:07.518 - INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8443 (https) with context path ''
2022-09-23 14:14:07.579 - INFO 1 --- [ main] o.o.o.u.u.springboot.EgeriaUIPlatform : Started EgeriaUIPlatform in 30.35 seconds (JVM running for 32.378)
➜ master git:(master)
from egeria-ui.
The UI itself may also need a code enhancement so that it does not continually hang if misconfigured - timeout/error ? @sarbull
from egeria-ui.
We will target this to coincide with the 3.13 release (work during Oct, release by ~ 1 Nov)
from egeria-ui.
Plan to revisit this in time for the 3.13 release since the new UI should now be ready.
Likely that there are some minor changes relating to environment
from egeria-ui.
I took a look at this again now 3.13 is released.
Installed with the additional config file (add -f ~/thisfile.yaml
)
image:
uistatic:
tag: "4.1.0"
I get an nginx 'Bad gateway' error on a service pointing to the nginx pod (port 8443)
the deployed configuration on nginx is:
server {
listen 8443 ssl;
#listen 80;
server_name lab-nginx;
ssl_certificate /etc/nginx/ssl/tls.crt;
ssl_certificate_key /etc/nginx/ssl/tls.key;
ssl_password_file /etc/nginx/pass/pass.txt;
#root /var/www/;
#index index.html;
# Force all paths to load either itself (js files) or go through index.html.
location /api {
proxy_pass https://lab-ui:8443;
proxy_set_header Host $http_host;
proxy_ssl_verify off;
proxy_ssl_session_reuse on;
proxy_ssl_server_name on;
}
location / {
proxy_pass http://lab-uistatic:8080;
proxy_set_header Host $http_host;
proxy_ssl_verify off;
proxy_ssl_session_reuse on;
proxy_ssl_server_name on;
}
}
Checking within the nginx container we see we cannot connect to the latter:
1000660000@lab-odpi-egeria-lab-nginx-6fbf4bc8bc-4kgff:/etc/nginx/conf.d$ curl http://lab-uistatic:8080
curl: (7) Failed to connect to lab-uistatic port 8080: Connection refused
Looking at the container definition (in our charts)for egeria-ui it only declares 8080 as the exposed port.
HOWEVER in it's configuration we have:
# SPDX-License-Identifier: Apache-2.0
# Copyright Contributors to the Egeria project
server {
listen 80;
server_name _;
root /var/www/;
index index.html;
# Force all paths to load either itself (js files) or go through index.html.
location / {
try_files $uri /index.html;
}
}
Inspecting the container image, in our staticui layer, only port 80 is exposed, and the docs in the embedded metadata refer to port 80. This is true for the base nginx image used by the UI
As such it seems the //intent// is now for the staticui container to expose port 80 (not 8080)
If we compare this with the older UI (version 3.2.0) used in the helm charts up until 3.2.0 we see the config file is:
$ cat staticui.conf
# SPDX-License-Identifier: Apache-2.0
# Copyright Contributors to the Egeria project
server {
listen 8080;
server_name _;
root /var/www/;
index index.html;
# Force all paths to load either itself (js files) or go through index.html.
location / {
try_files $uri /index.html;
}
}
confirming the 8080 port was correct then, and indeed the UI works correctly.
The fix therefore is to either
- change the container to work with port 8080 by default
- change the k8s deployment to use port 80
(Aside: We could also get rid of the additional nginx component, but this can be a seperate issue for optimization)
Since remapping is simple, the minimal change is the latter. I will do this. Any subsequent change can be done via a new issue/PR cc: @lpalashevski @sarbull
from egeria-ui.
I've added a conditional check . If version >=4 we basically use the port 80 setup. otherwise we use port 8080. Should work for old and new UI. Probably unnecessary, but it makes it easier to test the new UI in the charts. May then remove in a future release.
from egeria-ui.
Having implemented this check, we still cannot run with the container exposing port 80, since we need (want) to run our charts without root priviliges.
ie the pods running on k8s will not be running under root, therefore no ports under 1024 can be bound to.
This is in fact why we ran on 8080 in the first place
The docker container should setup the default to run on a port not requiring root. Optional to provide the ability to configure this differently if required.
If this is changed to port 8080, no change to the charts is then required.
Therefore reassigning this to the egeria-ui component
from egeria-ui.
In both cases our nginx related files in the helm definition create config files
/etc/nginx.conf is set to fixed content from within 'egeria-uistatic.yaml'. this is done by creating a configmap, which is mounted as that file in the container filesystem:
# SPDX-License-Identifier: Apache-2.0
# Copyright Contributors to the Egeria project.
worker_processes auto;
error_log /var/log/nginx/error.log notice;
events {
worker_connections 1024;
}
pid /tmp/nginx.pid;
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
client_body_temp_path /tmp/client_temp;
proxy_temp_path /tmp/proxy_temp_path;
fastcgi_temp_path /tmp/fastcgi_temp;
uwsgi_temp_path /tmp/uwsgi_temp;
scgi_temp_path /tmp/scgi_temp;
include /etc/nginx/conf.d/*.conf;
}
The actual server definition is then built by creating a template in nginx. this is taken from etc/staticui.conf.template. That is placed into a volume which is mounted at /etc/nginx/templates :
# SPDX-License-Identifier: Apache-2.0
# Copyright Contributors to the Egeria project
server {
listen 8080;
server_name _;
root /var/www/;
index index.html;
# Force all paths to load either itself (js files) or go through index.html.
location / {
try_files $uri /index.html;
}
}
This is then copied by the nginx container at startup, with variable replacement, and ends up with a correct port 8080 configuration
However it seems as if whilst the old nginx image does this templating, the new version does not .... and therein lies the reason that nginx starts on the wrong port.
Need to look into why templating isn't working / what's changed.
Alternatively we could build explicit config files from within the helm template.
from egeria-ui.
In the old template, we left the entrypoint as it:
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
When we look at the cmd/entrypoint definitions.
Old:
"Cmd": [
"nginx",
"-g",
"daemon off;"
],
"Entrypoint": [
"/docker-entrypoint.sh"
New:
"Entrypoint": [
"nginx",
"-g",
"daemon off;"
],
In the new image the entrypoint is overwritten by the Dockerfile:
ENTRYPOINT ["nginx", "-g", "daemon off;"]
Since we are reusing a standard image, which already defines a good default entrypoint, we have no need to override
Two other observations
- We do write a default.conf file to the image. this will be overwritten in the k8s environment, but will be applied for standalone containers. This is configured for port 80. I will not modify, but an issue should be raised to document behaviour, - and change default if needed
- This container build is slow for multi-arch, as a full npm build & container build is done each time. Since all we are doing is copying built files, we could do either of these to reduce the additional container build to seconds from ~10 minutes (on m1 mac)
a) Copy files from the build directory. This does make the docker build slightly less portable as the host environment needs npm tools
b) Copy files in at runtime. But this means no pre-built image. good for maintainance, but probably less easy to use.
For now I will not change or raise issues on the above.
Note that any change to the container will need a 'release' to push
from egeria-ui.
👀
from egeria-ui.
Thanks, I'm going to update the charts to a prerelease level to include the current UI
from egeria-ui.
ok, will do, i'll let you know
from egeria-ui.
Agreed on commmunity call 20230201 that we'd like to add into v4
cc: @lpalashevski
from egeria-ui.
Now that an image is available I have merged the change I made to the charts
Current status
- Can connect to ui (static) pod 8443 & get login panel
- admin/secret does not work
from egeria-ui.
We now have additional fixes for the ui-chassis which are being integrated into release 4
This is currently building, will be merged, and then I will rerun the release pipeline. We should then have a complete v4 build later.
There is also a js commons fix made under
We will need a new 'egeria-ui' container image, ideally at a release version (or make it a prerelease, we test, then repeat with final version). Can you arrange this @sarbull @lpalashevski @bogdan-sava
The charts use nginx, and redirect /api to the ui-chassis container. I think we expect this to work ok with the above changes?
from egeria-ui.
@bogdan-sava is everything in pace now?
if so next steps for the UI are:
- release new version for egeria-js-comons
- update reference to egeria-js-commons in egeria-ui
- release egeria-version whcih triggers docker image build
If all commits are in I can take over with the versions and build the image.
from egeria-ui.
All good except for the cors. It works only with nginx or similar solution.
from egeria-ui.
@lpalashevski The release pipeline has been started at https://github.com/odpi/egeria/actions/runs/4545935086
We need an 'egeria-ui' container image (ie your 2 first bullets above )
from egeria-ui.
I am looking for fixing the cors issues also
from egeria-ui.
If we're ok ui is good enough for now, would be good to go with the v4 builds we have as final and fix cors fully for 4.1?
I think, that we think, that it should work with the charts now? Since we use nginx there (and redirect /api to the ui chassis)
But we need to test before we know for sure.
from egeria-ui.
Thanks for getting that underway :-)
We'll need to do the merge to get the containers published - the PR build just verifies the build, it does not push - nor can it (permissions). The merge will actually push them to quay.io & docker.io
Ok if we merge?
from egeria-ui.
I have updated the charts to include this latest image, and to facilitate wider testing.
To install use:
helm repo update && helm search repo egeria --devel --set jupyter.tokenPlain='s3cr3t!' --set imageDefaults.pullPolicy=Always
These additional parms
a) pick the prerelease charts
b) set a specific password for jupyter (easier)
c) ensure the image is always pulled. This is the default for snapshots,but for releases we allow caching, but this image has been built multiple times, so to be sure...
However, the UI is not working for me. I now get a blank screen when going to the static ui service (previously this was a login panel). Let's discuss/debug when free
from egeria-ui.
Update - incognito in chrome/safari shows different behaviour, so the blank screen was probably caching/cookies related.
I can now login - so we know the uichassis is working
This may be config related. Perhaps view service configuration?
No warnings being seen in the ui chassis pod
from egeria-ui.
The following environment is currently set (as from old UI):
OMAS_SERVER_NAME=cocoMDS1
OMAS_SERVER_URL=https://lab-datalake:9443
OPEN_LINEAGE_GRAPH_SOURCE=MAIN
OPEN_LINEAGE_SERVER_NAME=cocoOLS1
OPEN_LINEAGE_SERVER_URL=https://lab-datalake:9443
Similar properties still seem appropriate as per the application.properties for ui chassis
from egeria-ui.
Enabled debug, restarted ui pod, all ok - was able to login as erinoverview & garygeeke & retrieve assets 'Week*'
from egeria-ui.
On first login it looks like a fetch of types is failing.
This is after the pods all report as up - but may be timing related
from egeria-ui.
However when in this state, a logout/login does not fix this problem.
- the fetch of types worked second time around, but the UI remained in the original 'circling' state (with the logo/greyed out)
- once in this state, a reload of the UI returns to the same page. Logout is greyed out, so at this point one is 'stuck'
from egeria-ui.
The initial error is startup timing. Debug logs show:
2023-03-29 07:35:57.397 - INFO 1 --- [ main] o.o.o.u.u.springboot.EgeriaUIPlatform : Started EgeriaUIPlatform in 60.405 seconds (process running
for 61.971)
2023-03-29 07:37:56.824 -ERROR 1 --- [nio-8443-exec-2] o.o.o.c.ffdc.RESTExceptionHandler : Detected Invalid Parameter Exception in REST Response
org.odpi.openmetadata.frameworks.connectors.ffdc.InvalidParameterException: OMAG-MULTI-TENANT-404-001 The OMAG Server cocoMDS1 is not available to service a
request from user erinoverview
at org.odpi.openmetadata.commonservices.ffdc.RESTExceptionHandler.throwInvalidParameterException(RESTExceptionHandler.java:289) ~[ffdc-services-4.0.j
ar!/:na]
at org.odpi.openmetadata.commonservices.ffdc.RESTExceptionHandler.detectAndThrowInvalidParameterException(RESTExceptionHandler.java:206) ~[ffdc-servi
ces-4.0.jar!/:na]
at org.odpi.openmetadata.accessservices.assetcatalog.AssetCatalog.detectExceptions(AssetCatalog.java:307) ~[asset-catalog-client-4.0.jar!/:na]
at org.odpi.openmetadata.accessservices.assetcatalog.AssetCatalog.getSupportedTypes(AssetCatalog.java:276) ~[asset-catalog-client-4.0.jar!/:na]
at org.odpi.openmetadata.userinterface.uichassis.springboot.service.AssetCatalogOMASService.getSupportedTypes(AssetCatalogOMASService.java:236) ~[cla
sses!/:na]
This is due to the 'cocoMDS1' server 'built in' to the UI
NOTE: Having TWO cocoMDS1 servers in the same deployment -- as is the case for coco pharma - is confusing. I think it should be named something else (but not for this release)
The timing issue can probably be put in the release notes for now, though we should consider how the UI should handle that situation more elegantly -- and perhaps hint to the user to retry later. For most people experimenting, they'll likely start the UI after the cocoMDS1 has started. But, The UI should not hang without any exit route.
For proper deployments of egeria (a notebook based approach is quite particular for education, developers) & as we move forward with cloud native work, we need to clearly define an appropriate health check, so that we can coordinate when pods are ready to do real work, and have requests routed to them (a much biggger topic).
from egeria-ui.
Repeated the test, but left about 7 minutes after deployment before running the UI (this was after the configure/data catalog notebooks).
Was able to login to the UI first time.
However also noticed 'About' does not load - again the swirling progress indicator
In summary, I think the UI is now deployed ok, but there are a number of issues we should open, and document in the release notes for 4.0 - but practically, it's probably what we should go with for this release
@bogdan-sava @sarbull @lpalashevski Does this make sense? Can you check what behaviour you see?
I can open up the issues & add to release notes - or please add your text if you wish?
from egeria-ui.
Also
- clicking on glossary icon on the left Hand Side, when reviewing search results, takes one back to the login page
- clicking on profile gives blank page
- the (i) button, similar to the menu item for about, gives spinning circle
These may still be in development
from egeria-ui.
I have not raised an issue on the blank profile as I presume this is not developed yet, and at least does not cause an error, it's just there are no profile contents.
Given we can now deploy the UI, closing this issue
from egeria-ui.
this should work now, there were some changes on the token implementation and cors functionality
from egeria-ui.
Related Issues (20)
- Updates to Docker build HOT 3
- Egeria UI broken HOT 6
- User roles fix
- Appropriate UI release for base egeria release 2.9 HOT 2
- Error in homepage. "roles" request - 404 HOT 2
- Migrate build of container to egeria-ui project HOT 3
- Create stable releases (inc. docker images) HOT 3
- something wrong about egeria-ui HOT 1
- Introduce search flags into Rex HOT 2
- Replace iron-localstorage
- Switch UI Docker base image to UBI HOT 8
- Migrate docker build to this repo & all builds to github actions HOT 7
- 'About' does not work HOT 2
- Error handling needs improving when ui-chassis is not ready HOT 7
- Glossary View results in being thrown back to login page (user not authorized) HOT 1
- Simplifying the load of a glossary
- Initial feedback on building egeria-ui HOT 2
- Is egeria ui the management interface for egeria
- Random "OK" button on the login page
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from egeria-ui.