Go from graph data to a secure and interactive visual graph app in 15 minutes. Batteries-included self-hosting of graph data apps with Streamlit, Graphistry, RAPIDS, and more!
License: BSD 3-Clause "New" or "Revised" License
A typical case is public vs private dashboards:
Document
The default image base uses RAPIDS, except:
We should provide a way to swap in alternative bases:
Public dashboards in Enterprise should not require any authentication. Private dashboards are working as expected, however public gives the following 404 error:
Connection failed with status 404, and response " <!DOCTYPE html> <html class="loading" lang="en" data-textdirection="ltr"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=0, minimal-ui"> <meta name="author" content="Graphistry, Inc."> <title>Graphistry: Page not found </title> <meta name="apple-mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent"> <meta name="apple-mobile-web-app-title" content="@graphistry/pivot"> <link rel="shortcut icon" href="/pivot/favicon/favicon.ico"> <meta name="mobile-web-app-capable" content="yes"> <meta name="theme-color" content="#fff"> <meta name="application-name" content="@graphistry/pivot"> <link rel="stylesheet" type="text/css" href="/static/fontawesome/css/all.min.css" /> <link rel="stylesheet" href="/static/bootstrap/css/bootstrap.min.css"> <link href="/static/legacy/css/graphistry.css" rel="stylesheet"> <link rel="stylesheet" type="text/css" href="/static/app-assets/css/vendors.css"> <link rel="stylesheet" type="text/css" href="/static/app-assets/css/app.css"> <link rel="stylesheet" type="text/css" href="/static/assets/css/style.css"> <link rel="stylesheet" type="text/css" href="/static/app-assets/css/core/menu/menu-types/vertical-menu-modern.css"> <!-- Start of graphistry Zendesk Widget script --> <script id="ze-snippet" src="[https://static.zdassets.com/ekr/snippet.js?key=7b26a76d-6f7e-4a3d-849c-372ae6c556b6">](https://static.zdassets.com/ekr/snippet.js?key=7b26a76d-6f7e-4a3d-849c-372ae6c556b6%22%3e) </script> <!-- End of graphistry Zendesk Widget script --> </head> <body class="vertical-layout vertical-menu-modern 2-columns menu-collapsed fixed-navbar" data-open="click" data-menu="vertical-menu-modern" data-col="2-columns"> <!-- fixed-top--> <nav class="header-navbar navbar-expand-md navbar navbar-with-menu navbar-without-dd-arrow fixed-top navbar-dark navbar-shadow"> <div class="navbar-wrapper"> <div class="navbar-header"> <ul class="nav navbar-nav flex-row"> <li class="nav-item mobile-menu d-md-none mr-auto"><a class="nav-link nav-menu-main menu-toggle hidden-xs" href="#"><i class="ft-menu font-large-1"></i></a></li> <li class="nav-item mr-auto"><a class="navbar-brand" href="/"><img class="brand-logo" alt="modern admin logo" src="/static/assets/images/logo/logo1.png"> <h3 class="brand-text"> </h3></a></li> <li class="nav-item d-none d-md-block float-right"><a class="nav-link modern-nav-toggle pr-0" data-toggle="collapse"><i class="toggle-icon ft-toggle-right font-medium-3 white" data-ticon="ft-toggle-right"></i></a></li> <li class="nav-item d-md-none"><a class="nav-link open-navbar-container" data-toggle="collapse" data-target="#navbar-mobile"><i class="la la-ellipsis-v"></i></a></li> </ul> </div> <div class="navbar-container content"> <div class="collapse navbar-collapse" id="navbar-mobile"> <ul class="nav navbar-nav float-right"> <li class="dropdown dropdown-user nav-item"> <a class="dropdown-toggle nav-link dropdown-user-link" href="#" role="button" id="dropdownMenuButton" data-toggle="dropdown" aria-expanded="false"> <span class="user-email">ricky</span> <span style="display: inline-block; margin-left: 0.5em; margin-right: 0.5em;"> <i class="fa fa-user"></i> </span> <span class="user-name">personal account</span> </a> <ul class="dropdown-menu dropdown-menu-right" aria-labelledby="dropdownMenuButton"> <li><a class="dropdown-item" href="/admin/"><i class="fa fa-users-cog"></i> Admin portal</a></li> <div class="dropdown-divider"></div> <li><a class="dropdown-item" href="/users/update/profile/"><i class="fa fa-user-edit"></i> Edit profile</a></li> <li><a class="dropdown-item" href="/accounts/password/change/"><i class="fa fa-edit"></i> Change password</a></li> <li> <a class="dropdown-item" href="/accounts/password/reset/"> <span class="fa-stack" style="font-size: 0.5rem; margin-right: 7px"> <i class="fa fa-undo fa-stack-2x"></i> <i class="fa fa-lock fa-stack-1x"></i> </span> Reset password</a> </li> <div class="dropdown-divider"></div> <li><a class="dropdown-item" href="/healthcheck/"><i class="fa fa-medkit"></i>Health check</a></li> <div class="dropdown-divider"></div> <li><a class="dropdown-item" href="/users/o/"><i class="far fa-building"></i> Manage organizations </a></li> <li><a class="dropdown-item" href="/users/o/SITE/sso/list/"><i class="fas fa-shield-alt"></i> Manage site-wide SSO </a> </li> <div class="dropdown-divider"></div> <li id="id-dropdown-hover" class="dropdown dropend"> <a class=" align-items-center row" id="dropdownMenuButtonSubMenu" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false"> <span class="col-10"> <span class="dropdown-item no-click label">Active organization:</span> <span class="dropdown-item no-click label text-body">personal account</span> </span> </a> <li><a class="dropdown-item" href="/users/personal/key/"><i class="fa fa-key"></i> Manage API keys</a></li> </li> <div class="dropdown-divider"></div> <li> <a class="dropdown-item" href="/accounts/logout/"><i class="fa fa-sign-out-alt"></i> Logout </a> </li> </ul> </li> </ul> <ul class="nav navbar-nav float-right"> <i class="fas fa-circle-notch fa-spin d-none spinner"></i> <li class="dropdown dropdown-user nav-item "> <span class="dropdown-toggle nav-link dropdown-user-link" style="font-size: 18px"> <span class="badge badge-warning">Enterprise</span> <style> .orange-color { color: #ff9149; } .orange-color:hover { color: #ff6500; } </style> </span> </li> </ul> <span class="ml-auto mr-2"> <span style="color:red; font-size:14px;"></span> </span> </div> </div> </div> </nav> <!-- ////////////////////////////////////////////////////////////////////////////--> <!-- main menu--> <div class="main-menu menu-fixed menu-dark menu-accordion menu-shadow" data-scroll-to-active="true"> <div class="main-menu-content"> <ul class="navigation navigation-main pt-2" id="main-menu-navigation" data-menu="menu-navigation"> <li class="nav-item"> <a href="/" class="nav-link"> <i class="fa fa-home"></i> <span class="menu-title" data-i18n="">Home</span> </a> </li> <li class="nav-item"> <a href="/pivot/home" class="nav-link"> <i class="fa fa-eye"></i> <span class="menu-title" data-i18n="">Visual Playbooks (beta)</span> </a> </li> <li class="nav-item"> <a href="/pivot/connectors" class="nav-link"> <i class="fa fa-plug"></i> <span class="menu-title" data-i18n="">Connectors</span> </a> </li> <li class="nav-item"> <a href="/notebook" class="nav-link"> <i class="fa fa-file-code"></i> <span class="menu-title" data-i18n="">Notebook Directory</span> </a> </li> <li class="nav-item"> <a href="/private/dash/" class="nav-link"> <i class="fa fa-tachometer-alt"></i> <span class="menu-title" data-i18n="">Private Dashboards</span> </a> </li> <li class="nav-item"> <a href="/public/dash/" class="nav-link"> <i class="fa fa-bullhorn"></i> <span class="menu-title" data-i18n="">Public Dashboards</span> </a> </li> <li class="nav-item"> <a href="/datasets/" class="nav-link"> <i class="fa fas fa-images"></i> <span class="menu-title" data-i18n="">Gallery</span> </a> </li> <li class="nav-item"> <a href="/files/" class="nav-link"> <i class="fa fas fa-file-alt"></i> <span class="menu-title" data-i18n="">Files</span> </a> </li> <li class="nav-item"> <a href="/docs" class="nav-link"> <i class="fa fa-question-circle"></i> <span class="menu-title" data-i18n="">Documentation</span> </a> </li> <li class="nav-item"> <a href="[https://www.graphistry.com/support"](https://www.graphistry.com/support%22) class="nav-link"> <i class="fa fa-phone"></i> <span class="menu-title" data-i18n="">Support</span> </a> </li> <li class="nav-item"> <a href="/pivot/upload-file" class="nav-link"> <i class="fa fas fa-plus-circle"></i> <span class="menu-title" data-i18n="">Visualize files (beta)</span> </a> </li> </ul> </div> </div> <div class="app-content content"> <div class="content-wrapper"> <div class="content-body"> <div class="card"> <div class="card-body"> <h5 class="card-title"> <i class="fa fa-question-circle fa-sm"></i> Oh no! Page not found </h5> <p class="card-text">You may want to:</p> <ul> <li>Check the original link for errors: cut off, misformatted, ...</li> </ul> If the problem persists, please contact staff </div> </div> </div> </div> </div> <!-- ////////////////////////////////////////////////////////////////////////////--> <footer class="footer footer-static footer-dark navbar-border navbar-shadow"> <p class="clearfix blue-grey lighten-2 text-sm-center mb-0 px-2"> <span class="float-md-left d-block d-md-inline-block"> Copyright © 2023 <a class="text-bold-800 grey darken-2" href=https://www.graphistry.com/ target="_blank">Graphistry, Inc.</a>, all rights reserved. Graphing the planet with version v2.40.55. </span></p> </footer> <script src="/static/app-assets/vendors/js/vendors.min.js" type="text/javascript"></script> <script src="/static/app-assets/js/core/app-menu.js" type="text/javascript"></script> <script src="/static/app-assets/js/core/app.js" type="text/javascript"></script> <script> $('#id-dropdown-hover').hover(function(){ $(this).addClass("show", 100); }, function(){ $(this).removeClass("show", 100); }); </script> </body> </html> ".
Describe the bug
if a user has not configured a splunk connection but selects the "DEMO: cyber alerts" (demo_avr) view, then the following error is presented to the user in main area:
Error loading dashboard
SplunkConnectException: argument of type 'NoneType' is not iterable
Traceback:
File "/apps/views/demo_avr/__init__.py", line 261, in run_all
splunk_client: Union[SplunkConnection, Any] = cache_splunk_client(splunk_username, splunk_password, splunk_host)
File "/opt/conda/envs/rapids/lib/python3.8/site-packages/streamlit/runtime/caching/cache_utils.py", line 211, in wrapper
return cached_func(*args, **kwargs)
File "/opt/conda/envs/rapids/lib/python3.8/site-packages/streamlit/runtime/caching/cache_utils.py", line 240, in __call__
return self._get_or_create_cached_value(args, kwargs)
File "/opt/conda/envs/rapids/lib/python3.8/site-packages/streamlit/runtime/caching/cache_utils.py", line 266, in _get_or_create_cached_value
return self._handle_cache_miss(cache, value_key, func_args, func_kwargs)
File "/opt/conda/envs/rapids/lib/python3.8/site-packages/streamlit/runtime/caching/cache_utils.py", line 320, in _handle_cache_miss
computed_value = self._info.func(*func_args, **func_kwargs)
File "/apps/views/demo_avr/__init__.py", line 301, in cache_splunk_client
assert splunk_client.connect()
File "/apps/components/Splunk.py", line 71, in connect
raise SplunkConnectException(e)
Expected behavior
present a more user-friendly error, or fall back to using a local CSV for input vs splunk so the demo still runs if splunk is not configured, this might be a bit challenging with the filter logic being passed to splunk, so the first option would be quick+easy, but less desirable.
Observed behavior
see above
To Reproduce
gatherUsageStats should be overridden to be off by default: https://docs.streamlit.io/en/latest/streamlit_configuration.html
Need sudo chown -R ubuntu /home/ubuntu/graphistry/data/notebooks/graph-app-kit/ for ubuntu builds (graphistry.yml's), likely ec2-user for minimal.yml's
I've noticed several things need to be updated in the graph-app-kit README.md, and also the cloudformation stack there is using a pretty old version of graphistry.. so that might also need a separate issue created after that is verified..
The StreamLit Graphistry service account does not need admin, we may have regressed there:
-- Main admin should be admin / i-instanceid
-- StreamLit can be something like streamlit / <random>
The reason to separate is in case someone changes the main pwd, that shouldn't break streamlit
Is your feature request related to a problem? Please describe.
It's not obvious which instance types support valid GPUs, so a picker would be great!
Describe the solution you'd like
A dropdown would be ideal, but unclear if formation supports.
Formation does allow specifying a restricted set of allowed values, so an option is:
Describe alternatives you've considered
A cloudformation parameter type:instance... but does not seem to exist
When volume binding w/ jupyter, unrelated hidden files may appear in views/, which AppPicker::list_modules chokes on ( mod = importlib.import_module(f'views.{view_folder}')). It should continue instead, somehow.
Hey Folks, I really much like what you do! Especially since you try to reduce complexity for a normally hard problem.
Yet I'd super appreciate if you could reduce this a little further. Do you think that is possible?
All that setup and stuff looks rather difficult to me. Would be great!
Thank you.
Is your feature request related to a problem? Please describe.
Right now, adding a new dashboard means adding code via the file system or Jupyter, and then relaunching streamlit. That's hard and non-obvious. It should be fast and easy!
Describe the solution you'd like
Ex: Make the dashboard component updateable, and maybe even switch to reading a tracked file (def list_dashboards(): ...), so streamlit gracefully handles changes
Describe alternatives you've considered
Is your feature request related to a problem? Please describe.
Unclear how to do gpu / multi-gpu well, such as on logs -> graph
Describe the solution you'd like
Using a challenge dataset:
Describe alternatives you've considered
Additional context
Part of GTC 2021 talk & graphistry/pygraphistry#221
package deps to facilitate local dev for docker-less install & use (no 4GB base image)
unclear conda vs pip
Instead of doing the public IP lookup (https://github.com/graphistry/graph-app-kit/blob/master/src/bootstraps/neptune/graphistry.sh#L45), which does not work in some modes, we should support:
I am following the AWS blog post and when going through the CloudFormation Template I have to provide a public subnet. If I provide a private subnet, then the stack creation fails.
I would like to use a private subnet so that I can restrict traffic to my VPC only. I also don't want to attach a public subnet to my Neptune database. Is there a way to use this CloudFormation Template (or another template) using a private subnet?
Describe the bug
I execute docker-compose build. Below are the console logs. It throws an error.
D:\yusuf\graph-app-kit\src\docker>docker-compose build
WARNING: The BASE_PATH variable is not set. Defaulting to a blank string.
autoheal uses an image, skipping
Building streamlit
failed to get console mode for stdout: The handle is invalid.
[+] Building 4.7s (8/15)
[+] Building 4.8s (9/15)
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/graphistry/graphistry-forge-ba 2.0s
=> importing cache manifest from graphistry/graph-app-kit-st:latest 1.8s
=> [ 1/10] FROM docker.io/graphistry/graphistry-forge-base:v2.36.6-11.0@ 0.0s
=> [internal] load build context 0.0s: exit code: 126
=> => transferring context: 229B 0.0s
=> CACHED [ 2/10] WORKDIR /app 0.0s
=> CACHED [ 3/10] COPY python/conda-app.sh ./ 0.0s
=> ERROR [ 4/10] RUN source activate rapids && ./conda-app.sh 0.8s[ 4/10] RUN source activate rapids && ./conda-app.sh:
#9 0.756 /bin/bash: ./conda-app.sh: /bin/bash^M: bad interpreter: No such file or directory
ERROR: Service 'streamlit' failed to build
Expected behavior
Work
Observed behavior
Error
To Reproduce
Clone the repo and then run docker-compose build
Environment
Additional context
Right now, multi-st guidance is git-clone-per-instance. But typical case may be external vol mounts for user dashes & config, and can likely have them reuse the same graph-app-kit repo for code. Maybe there is a way to to [my_proj].env and share the repo across st's?
Is your feature request related to a problem? Please describe.
Upon launching graphistry.yml in AWS, I got this error:
Your requested instance type (p3.2xlarge) is not supported in your requested Availability Zone (us-east-1a).
Please retry your request by not specifying an Availability Zone or choosing us-east-1b, us-east-1c, us-east-1f.
(Service: AmazonEC2; Status Code: 400; Error Code: Unsupported; Request ID: ***; Proxy: null)
Being able to specify AZ would help avoid it
Describe the solution you'd like
By default, AZ gets auto-picked. But if desired, can specify.
Is your feature request related to a problem? Please describe.
Basic operations like starting and stopping are surprising copy-paste, they'd be easier as simple commands, e.g., just run them with no docker knowledge necessary
Describe the solution you'd like
Minimal mode users:
./bin/start, ./bin/stop, ./bin/restart, ./bin/recreate
Full mode users:
./bin/public_start, ..., ./bin/private_start, ... ./bin/graphistry_start, ...
Describe alternatives you've considered
Just documenting
Additional context
Design thoughts:
bin/, bin/full/~/bin/ (~/graph-app-kit/bin/?)To match main Graphistry
Is your feature request related to a problem? Please describe.
When using the experimental native loading support (cd src/python && streamlit ...), the .env is not used. It'd be helpful if it was -- currently a non-obvious workaround like env $(cat .env | xargs) streamlit ... is required .
Describe the solution you'd like
If a .env is locally available, or passed in (ENV_FILE=... streamlit), it should be recognized
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
The pydantic env package has a convenient abstraction that we use elsewhere for this
Next round of Neptune/Gremlin demos, in rough priority order:
Deferred:
Describe the bug
dc up from src/docker/ results in an error on newer versions of Docker.
./dc up+ DOCKER_BUILDKIT=1
+ COMPOSE_DOCKER_CLI_BUILD=1
+ docker-compose up
./dc: line 5: docker-compose: command not found
Why? There are references to the deprecated docker-compose command that should be replaced by the current docker compose - but not when referring to files.
Expected behavior
I expect the system to launch docker compose.
Observed behavior
See above, dc up crashed before it could start. It failed to run docker compose.
To Reproduce
Steps to reproduce the behavior, ideally entirely copy-pastable for running on an out-of-the-box install
cd src/docker && dc upScreenshots
If applicable, add screenshots to help explain your problem.
Environment
gh repo clone graphistry/graph-app-kit; cd src/docker && ./dc upAdditional context
This is like, my first ticket that results in an instant PR behind it? :)
Describe the bug
Neptune guide's quicklaunch link points to the full launcher
Expected behavior
Should point to the minimal launcher
Streamlit now supports multipage apps, so we should switch to that to avoid confusion: https://blog.streamlit.io/introducing-multipage-apps/
This will take a bit of research first:
Is your feature request related to a problem? Please describe.
Graphistry url viz params are handy pass in via url, especially after the fact. Supporting that in the st component would be great!
Describe the solution you'd like
Default-on implicit url param passing. /public/views?pointSize=1.0 => graph.html?pointSize=1.0
Default-on seems unsafe though.. so maybe opt-om" st_g.plot(url, propagate_url_params=True)
Additional context
Happened during a prototyping session..
Graphistry + Neptune demos should show more friendly & guiding messages on initial start
We should setup standard semvar versioning:
As mamba is already used for the base layer for speed reasons, we should do the same for the graph-app-kit layers
Is your feature request related to a problem? Please describe.
Bootstraps are slow to start on aws, with a lot of initial time on:
########################################
+ echo '##'
##
+ echo '## Start: Installing CloudFormation Bootstrap'
## Start: Installing CloudFormation Bootstrap
+ echo '##'
##
+ echo '########################################'
########################################
+ echo
+ echo
+ sudo apt-get update -y
The update is unnecessary because AMIs are updated every 2-4w, and if users are more patch-sensitive, they should be regularly doing their own schedule. ~All other other dependencies should be within the container scripts so not need system ones.
+ cd ../../docker
+ docker-compose build
The BASE_PATH variable is not set. Defaulting to a blank string.
autoheal uses an image, skipping
Building streamlit
Step 1/20 : ARG DOCKER_TAG=latest
Step 2/20 : ARG GRAPHISTRY_FORGE_BASE_VERSION=latest
Step 3/20 : FROM graphistry/graphistry-forge-base:${GRAPHISTRY_FORGE_BASE_VERSION}
v2.35.9-11.0: Pulling from graphistry/graphistry-forge-base
The pull should be unnecessary. A temporary workaround is basing on forge-etl-python. Even better is if Graphistry explicitly tagsforge-etl-python (which is public container).
Describe the solution you'd like
This should just work :)
Describe alternatives you've considered
Graphistry AMIs can also just start shipping w/ graph-app-kit so most of this goes away
Latest version:
Ideally we get this into the next release
New auth modes:
Cloud formation currently only exposes NEPTUNE_READER_HOST, should also:
This can probably be put into simpler PRs:
Clarify config for sharing data + code w/ a Jupyter instance
Support conda envs
-- As an initial version, allow full user formulas, and have sandwiched: base conda+pip -> user conda -> user pip
-- Later, figure out staged builds for base conda -> user conda -> base pip -> user pip
-- Bonus: mamba instead of conda
Ex:
Dockerfile:
COPY python/conda-app.sh ./
...
RUN source activate rapids \
&& ./conda-app.sh \
&& pip install streamlit==0.68.0 \
&& pip install -r requirements-system.txt
python/conda-app.sh:
#!/bin/bash
set -ex
echo "=========================="
echo "="
echo "= Conda app dependencies"
echo "="
echo "=========================="
echo ""
conda install -c conda-forge hdbscan=0.8.26
echo "**** Successfull install conda app deps ***"
Describe the bug
Cloud formation for neptune fails on a p3.2 and p3.16 yet succeeds on a g4dn
Reported by a Neptune user
To Reproduce
Run through Neptune tutorial and use a p3.16
Expected behavior
It launches
Actual behavior
Formation template stalls out and auto-deletes
Working on getting logs. After 10min, GPU services (forge-etl-python + streamgl) failed to start. V100 issue?
Screenshots
Browser environment (please complete the following information):
all
PyGraphistry environment
All
Additional context
Current graph-app-kit
Is your feature request related to a problem? Please describe.
We should provide video tutorial links, starting with https://www.youtube.com/watch?v=F7ssqGqmJNo for manual + TG mode (readme.md, tigergraph.md, and manual.md)
Describe the bug
ST_PUBLIC_PORT is not getting passed to container results in wrong value in printed to logs
Expected behavior
want the correct value of ST_PUBLIC_PORT to appear
Observed behavior
No matter what you set ST_PUBLIC_PORT to, the entrypoint.sh in the container only ever sees 8501 as it's not being passed into the container.. e.g.
streamlit_1 | _ _ _ _
streamlit_1 | __ _ _ __ __ _ _ __ | |__ __ _ _ __ _ __ | | _(_) |_
streamlit_1 | / _\ | '__/ _\ | '_ \| '_ \ _____ / _\ | '_ \| '_ \ _____| |/ / | __|
streamlit_1 | | (_| | | | (_| | |_) | | | |_____| (_| | |_) | |_) |_____| <| | |_
streamlit_1 | \__, |_| \__,_| .__/|_| |_| \__,_| .__/| .__/ |_|\_\_|\__|
streamlit_1 | |___/ |_| |_| |_|
streamlit_1 |
streamlit_1 |
streamlit_1 | Browser path (BASE_URL): http://localhost:8501/dashboard
streamlit_1 | StreamLit internal base path (BASE_PATH): dashboard/
streamlit_1 | StreamLit host port (ST_PUBLIC_PORT): 8501
streamlit_1 | StreamLit views mount host path (GRAPH_VIEW):
streamlit_1 | Log level (LOG_LEVEL): DEBUG
streamlit_1 | Graphistry user (GRAPHISTRY_USERNAME): tcookaustin
To Reproduce
Steps to reproduce the behavior, ideally entirely copy-pastable for running on an out-of-the-box install
Latest gak (graphistry/graph-app-kit-st v2.40.11-11.5 22ac66583738 5 days ago 22.5GB) breaking staging
@DataBoyTX Can you triage as you've been in this code most recently? Not sure if we need to land one of the PRs or what, maybe @tanmoyio can help once we've identified the issue
Logs:
graphistry-graph-app-kit-public-1 exited with code 1
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:03.076242647Z pwd: /app
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:03.077334594Z .
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:03.077335946Z ./requirements-app.txt
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:03.077336137Z ./requirements-system.txt
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:03.077337399Z ./conda-app.sh
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771781161Z Traceback (most recent call last):
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771784688Z File "/opt/conda/envs/rapids/bin/streamlit", line 5, in <module>
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771785138Z from streamlit.web.cli import main
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771785539Z File "/opt/conda/envs/rapids/lib/python3.8/site-packages/streamlit/__init__.py", line 70, in <module>
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771786090Z from streamlit.delta_generator import DeltaGenerator as _DeltaGenerator
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771787353Z File "/opt/conda/envs/rapids/lib/python3.8/site-packages/streamlit/delta_generator.py", line 90, in <module>
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771787774Z from streamlit.elements.arrow_altair import ArrowAltairMixin
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771788244Z File "/opt/conda/envs/rapids/lib/python3.8/site-packages/streamlit/elements/arrow_altair.py", line 35, in <module>
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771789988Z from altair.vegalite.v4.api import Chart
graphistry-graph-app-kit-private-1 | 2023-05-18T05:29:05.771790589Z ModuleNotFoundError: No module named 'altair.vegalite.v4'
graphistry-graph-app-kit-public-1 | 2023-05-18T05:29:05.933702640Z ModuleNotFoundError: No module named 'altair.vegalite.v4'
And:
ubuntu@graphistry-cloud-v24011-run44:/var/graphistry$ docker run --rm -it --entrypoint=/bin/bash 22ac66583738
(rapids) root@822be0976f04:/app# ls
conda-app.sh requirements-app.txt requirements-system.txt
(rapids) root@822be0976f04:/app# cat ^C
(rapids) root@822be0976f04:/app# cat requirements-app.txt
### your custom requirements here
#protobuf==3.13.0(rapids) root@822be0976f04:/app# cat requirements-system.txt
streamlit==1.12.0
protobuf==3.18
graphistry==0.27.1
################
#
# Per-DB
#
################
### Neptune
gremlinpython==3.4.10
sshtunnel==0.1.5
### TigerGraph
pyTigerGraph # Unpinned as TG Cloud is rapidly moving
plotly # for demos(rapids) root@822be0976f04:/app#
Describe the bug
The Quick Launch instructions bring up the v2.35.9. Needs update to v2.36.6
Expected behavior
Clicking on "Launch Stack" launching the latest v2.36.6
Observed behavior
Clicking on "Launch Stack" launches the v2.35.9
To Reproduce
Steps to reproduce the behavior, ideally entirely copy-pastable for running on an out-of-the-box install
Preventing correct default bindings
Describe the bug
heathchecks against /caddy/health/ fail
Expected behavior
200
because
status 200 /caddy/health/
should be in the Caddyfile
Observed behavior
not 200... because no such entry
To Reproduce
Steps to reproduce the behavior, ideally entirely copy-pastable for running on an out-of-the-box install
Screenshots
If applicable, add screenshots to help explain your problem.
Environment
PyGraphistry 0.19.3
Additional context
Add any other context about the problem here.
Describe the bug
As part of #97 I created files other than src/python/views/__init__.py such as src/python/views/marlowe.py. Logs from these modules sometimes appear and sometimes disappear. It seems to vary depending on the type of exception. When I've researched the topic, it seems related to the use of importlib to load src/python/views/__init__.py dynamically. Log configurations often get reloaded when importlib imports code. Streamlit also dynamically re-runs code, but I don't know if that is involved.
I iterated through numerous log configurations before settling on this which does produce logs in ./dc up but not ./dc up -d && ./dc logs -f -t --tail=100.
# Make sure logs get through to STDERR
logger = logging.getLogger(__name__)
logger.setLevel(os.getenv("LOG_LEVEL", "DEBUG"))
stream_handler = logging.StreamHandler(stream=sys.stderr)
logger.addHandler(stream_handler)Expected behavior
I expect my logs to show up when I call util.log.getChild(app_id) from other files. As demos get more complicated, we need to split files into more that __init__.py.
Observed behavior
Logs from marlowe.py don't print, or don't print from the logger given util.log.getChild(app_id). This seems to vary by the type of exception an may be a stochastic process. It behaves strangely.
To Reproduce
I need to work on this...
Screenshots
If applicable, add screenshots to help explain your problem.
Environment
https://github.com/graphistry/graph-app-kit/blob/demo_avr/src/python/views/demo_avr/marlowe.py
./dc upAdditional context
In general, gak behaves... haunted. I think the combination of streamlit reloading things, importlib reloading things and streamlit repeating itself for requests is causing strange side effects. Unsure if this is happening here.
Describe the bug
Quicklaunch link names are like graph_app_kit, which cloud formation rejects due to underscores . See on Neptune docs, maybe others.
Expected behavior
Should be syntactically valid, like graph-app-kit
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.