awslabs / aiops-modules Goto Github PK

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
I would like to be able to use RDS for mlflow metadata storage. mlflow-fargate module should provide configuration options to support RDS instead of default EFS-based persistence. The mlflow-image module should be adapted to allow specify database URI to connect to and store experiments and model metadata.

Describe alternatives you've considered
Alternative is already built-in via EFS volume mounted to mlflow-fargate. This adds another option for RDS-backed persistence.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
Current multi-account basic template has a static CodePipeline that delivers endpoints to dev/test/prod. The pipeline is currently built into the Service Catalog template and does not allow customisation after it's been created.

Describe the solution you'd like

• Move CodePipeline into a separate {project_name}-infra repository (in addition to -build and -deploy repos created by the template).
• Enable self-mutation on the pipeline.

Describe the bug
Following IAM Role lookups fail in multi-account deployment.

Resource handler returned message: "The role with name cdk-hnb659fds-deploy-role-... cannot be found

Is your feature request related to a problem? Please describe.
Integration test deployments leave EFS volumes after module destroy... which is a reasonable default. We might want to add cofigurable removal policy.

Describe the solution you'd like
Add efs_removal_policy input parameter. Default to RETAIN.

Describe the solution you'd like
Basic multi-account template in sagemaker-templates-service-catalog module along with training Abalone with XGBoost has an example CICD pipeline (CodePipeline) to deploy an endpoint to dev, pre-prod, and prod. Separate the concerns and move the CICD into a) separate template OR b) separate module.

TBC

Additional context
Add any other context or screenshots about the feature request here.

Describe the solution you'd like
We need a batch inference example/template in sagemaker-templates-service-catalog.

Describe the solution you'd like
SageMaker Service Catalog Templates module currently does not accept pre-prod and prod account ids/regions as inputs, as well as VPC config. We want to be able to configure it using the manifest i.e. create VPCs in other accounts via seed-farmer:

name: networking
path: git::https://github.com/awslabs/idf-modules.git//modules/network/basic-cdk?ref=release/1.3.0&depth=1
targetAccount: primary
parameters:
  - name: internet-accessible
    value: True
---
name: networking-preprod
path: git::https://github.com/awslabs/idf-modules.git//modules/network/basic-cdk?ref=release/1.3.0&depth=1
targetAccount: pre-prod
parameters:
  - name: internet-accessible
    value: False

And reference them in the templates module via:

name: templates
path: modules/sagemaker/sagemaker-templates-service-catalog
targetAccount: primary
parameters:
...
  - name: preprod_vpc_id
    valueFrom:
      moduleMetadata:
        group: networking
        name: networking-preprod
        key: VpcId
  - name: preprod_subnet_ids
    valueFrom:
      moduleMetadata:
        group: networking
        name: networking-preprod
        key: PrivateSubnetIds

Task

• Add optional vpc id/subnets parameters for templates module for each of the stages in multi-account mlops template.
• Add parameters as pre-populated template inputs.

Is your feature request related to a problem? Please describe.
Existing basic multi-account template during build phase, calls SageMaker pipeline in the same account where the CodeBuild runs. However, this does not cater to multi-account scenario where a separate tooling account hosts the infra, and development/experimentation account is used to run training and provides access to the data.

Describe the solution you'd like

• Add assume role to the build CodeBuild to run training pipeline in the target/dev/experimentation account.
• Add required IAM config/ S3 buckets/ KMS in dev to run the pipeline.
• Keep the template flexible to allow simple (current) scenario that runs the pipeline in the same account.