gchq / bailo Goto Github PK

At the time of writing this ticket we use 'any' 188 times throughout our application. This leads to less stringent TypeScript checking and could cause errors. We should look to replace most of these 'any' statements with actual types.

There are four methods of deletion we should consider:

• Retire a deployment - this is a soft delete of a deployment. The deployment is marked for deletion and becomes unavailable. An automatic process then removes the deployment after a set period of time
• Retire a variant - this is a soft delete of a variant of a model including all deployments. The model variant and any deployments ares marked for deletion and becomes unavailable. An automatic process then removes the model and deployment after a set period of time
• Delete - This is a the full removal of the model including all variants. The model card is kept so that there is a record of the model
• Purge - This is a the full removal of the model, the model cards from live and all backups so that no trace exists beyond the logs. This may occur when copyrighted data is uploaded to the system.. It may be a manual, documented process.

Our continuous integration takes a long time at the moment because it needs to constantly build all images from scratch. We should take advantage of the GitHub caching service to speed up this process. You can use an action such as https://github.com/marketplace/actions/docker-layer-caching to cache Docker images.

Helm chart should have dependencies as subcharts.

Secrets should be generated via helm functions instead of via typescript file.

We intend to support users hosting Bailo on AWS EKS. As part of this, we should support using AWS Cognito for authentication. We already have a Cognito instance setup in our test environment, this ticket only relates to connecting it to our application.

Hello, Im facing an auth issues when installing bailo with helm, everything is running but an auth issue with the backend.

Gives auth error when connecting to mongoDB, have all the secret keys in place, host is also correct.

Any ideas?

The Bailo README links to a product screenshot image that doesn't resolve:

https://github.com/gchq/Bailo/blame/8e0fc4905cce872659cd6a14636aba2e6573499f/README.md#L62

Hi,
Is there an endpoint to upload docker images to the registry instead of pushing it manually through CLI?
I see there's one to retrieve images.

Subject of the issue

When running the docker-compose up command everything builds correctly but the bailio-regiustry service will not start and I get the following error:

Ports are not available: exposing port TCP 0.0.0.0:5000 -> 0.0.0.0:0: listen tcp 0.0.0.0:5000: bind: address already in use

From some research on the internet it appears that when running on a Mac, port 5000 is used for Airplay by default.

Do you have any suggestions or are you able to provide a docker-compose script that uses an alternate port for the registry service?

Your environment

Apple Macbook Pro
M1 Pro

Completely fresh install of Docker, no other containers running.

• version / commit hash of bailo (git rev-parse HEAD)
• runtime environment (docker-compose, openshift)

Steps to reproduce

Follow the instructions on the GitHub page to run in docker

Expected behaviour

Bailio should run as expected

Actual behaviour

Error as above

Our Python client is out of date and doesn't support working with the latest version of Bailo. It should be updated to be compatible with the new API endpoints.

Should be given to let administrators test out their new instance with a model zip / metadata that is proven to work.

This does not affect builds, which still work. It just errors out in the logs with the following error:

$ s2i build /var/folders/cs/fd4s_57x2b7_2h26xdfvgv0m0000gp/T/0069ef1c-2da6-4197-bce8-ffa80ede45cc seldonio/seldon-core-s2i-python37:1.10.0 localhost:5000/internal/test-model-5cheqa:1error: Unable to load docker config: json: cannot unmarshal string into Go value of type docker.dockerConfig

(Docker for Mac, default ~/.docker/config.json)

Subject of the issue

I am following the instructions mentioned in Helm chart instructions. The configuration step mentions creating and hosting Bailo image. The image can be built using docker build -t bailo . from root directory of project.

This step fails to build a docker image.

Your environment

• version / commit hash of bailo (git rev-parse HEAD) : main branch
• runtime environment (docker-compose, openshift) : kubectl / helm

Steps to reproduce

Running docker build -t bailo . from root directory.

Error

unable to convert uid/gid chown string to host mapping: can't find gid for group nextjs: no such group: nextjs

COPY command in docker recommends using --chown=<user>:<group>. Replacing --chown=nextjs:nextjswith --chown=nextjs:nodejs, the image build is a success.

Running the container, I get the following error

 [email protected] start
> cross-env NODE_ENV=production node dist/server/index.js

node:internal/modules/cjs/loader:959
  throw err;
  ^

Error: Cannot find module '../../lib/p-mongo-queue/pMongoQueue'
Require stack:
- /app/dist/server/utils/queues.js
- /app/dist/server/processors/processDeployments.js
- /app/dist/server/index.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:956:15)
    at Function.Module._load (node:internal/modules/cjs/loader:804:27)
    at Module.require (node:internal/modules/cjs/loader:1028:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/app/dist/server/utils/queues.js:10:39)
    at Module._compile (node:internal/modules/cjs/loader:1126:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
    at Module.load (node:internal/modules/cjs/loader:1004:32)
    at Function.Module._load (node:internal/modules/cjs/loader:839:12)
    at Module.require (node:internal/modules/cjs/loader:1028:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/app/dist/server/processors/processDeployments.js:13:18)
    at Module._compile (node:internal/modules/cjs/loader:1126:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
    at Module.load (node:internal/modules/cjs/loader:1004:32)
    at Function.Module._load (node:internal/modules/cjs/loader:839:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/app/dist/server/utils/queues.js',
    '/app/dist/server/processors/processDeployments.js',
    '/app/dist/server/index.js'
  ]
}
npm notice 
npm notice New minor version of npm available! 8.15.0 -> 8.19.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.19.2
npm notice Run npm install -g [email protected] to update!
npm notice 

On comparing Dockerfile with Dockerfile.prod, I find that the following line is missing

COPY --from=builder --chown=nextjs:nextjs /app/lib/p-mongo-queue ./dist/lib/p-mongo-queue

Adding the above line with the right group, the build is a success.

Pushing this image and running the instructions as provided in helm chart instruction, the pod crashes with CrashLoopBackOff error.

Pod Events

Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  102s               default-scheduler  Successfully assigned bailo/bailo-6b995bcf9-g7wqh to bailo-control-plane
  Normal   Pulling    101s               kubelet            Pulling image "dudeperf3ct7/bailo:latest"
  Normal   Pulled     20s                kubelet            Successfully pulled image "dudeperf3ct7/bailo:latest" in 1m20.652425634s
  Warning  Unhealthy  20s                kubelet            Readiness probe failed: Get "http://10.244.0.5:3000/": dial tcp 10.244.0.5:3000: connect: connection refused
  Normal   Created    19s (x2 over 20s)  kubelet            Created container bailo
  Normal   Started    19s (x2 over 20s)  kubelet            Started container bailo
  Normal   Pulled     19s                kubelet            Container image "dudeperf3ct7/bailo:latest" already present on machine
  Warning  BackOff    12s (x4 over 18s)  kubelet            Back-off restarting failed container

Tailing log for the pod provides following information

Failed to open temporary file /etc/ssl/certs/bundleXXXXXX for ca bundle
No schemas setup script set; You will have to add schemas after startup
Running 'npm run start'

Expected behaviour

Everything should work. i.e. just building the image and adding that to helm should give access to Model UI at 3000 port.

Models / deployments should be removed after a set period of time. Each model and deployment should be able to specify a specific retention time.

Our queue is currently called once per model submission and once per deployment submission. This rate of submission makes using Redis unnecessary for a queue service.

We can eliminate Redis and simplify our stack by replacing Redis with a queue built on our existing MongoDB instance.

There is no documentation around how to create a custom schema (for upload and deployment) in Bailo.

From what I gather perusing through the code, there are example upload and deployment json schemas inside example_schemas folder that are used as default.

The repo does mention a way

Some example schemas are installed by default. More schemas can be added by altering and running the addDeploymentSchema.ts and addUploadSchema.ts files.

ts-node --project tsconfig.server.json server/scripts/addDeploymentSchema.ts
ts-node --project tsconfig.server.json server/scripts/addUploadSchema.ts

There are various references pointing to example schema being tightly coupled with front-end (written using typescript).

Is there a way to add new fields or keys to model cards in the json schema? Will the front-end reflect the changes automatically?

Requested Feature

We'd like to open source Bailo so that we can collaborate with the community.

Justification for feature

It'll let us create a uniform template across multiple organisations to handle information about models.

Possible Solution

Migrate BAILO to GitHub.

Subject of the issue

When you select "yes" on a boolean question in the upload/deployment forms, if there's a dependency it will sometimes append an additional question (usually a text field prompting users to explain why they selected "yes") object. If you then change your answer to "no", the formData will keep the additional text object, meaning it causes validation issues. These have been band-aid fixed for now so that the validator ignores dependency errors, but we need to find out if we can remove the additional question objects completely from the form data if the user selects "no".

Steps to reproduce

• Select "Yes" on any boolean question that has a dependency
• Enter a value in the dependency question field that appears (usually a text field)
• Change "Yes" to "No"in the parent question field

Expected behaviour

The dependency question should disappear from the form and the object removed from the form data

Actual behaviour

The dependency question disappears from the form, but the answer entered still exists in the form's data.

For instance, visiting http://localhost:8080/model/yolo-v4-tiny-gzbbqt/deployments should link directly to the 'deployments' tab.

To complete this, listen to the router event and base the active tab from it. On clicking an active tab, switch the URL to the new route.

A user should be able to upload a .tar.gz file and receive a built image, in the same way as uploading the same .zip file. The program should automatically determine whether the file is a .tar.gz or a .zip file and unpack it as specified.

To complete this:

• The UI should accept files other than .zip (src/UploadFlow/UploadFile.tsx & src/common/FormFlow/FormFlow.tsx)
• server/routes/v1/upload.ts should accept .tar.gz files.
• server/utils/build.ts should be able to unzip .tar.gz files.

You can use the default zlib library to uncompress, and tar-stream to parse the tar file.