Comments (19)
@shailesh-vaidya Here is the tested snippet for my suggested workflow
name: Base Docker Image for <component name>
on:
push:
branches: [ dev ]
paths:
- 'scripts/docker/s3server-devel/Dockerfile' # Mentioning dockerfile path is must here
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build and push ${{ github.event.repository.name }} docker image
uses: docker/[email protected]
env:
TOKEN: ${{ secrets.USER_TOKEN }}
TAG: 7.7.1908
with:
# Path to the Dockerfile (Default is '{path}/Dockerfile')
dockerfile: scripts/docker/s3server-devel/Dockerfile # Define `dockerfile` variable only if your dockerfile isn't present in {path}
# Build Arguments for docker image build.
build_args: 'token=${{ env.TOKEN }}' # Define build arguments required in your dockerfile here
username: ${{ github.actor }}
password: ${{ env.TOKEN }}
registry: docker.pkg.github.com
repository: ${{ github.repository }}/${{ github.event.repository.name }}
tags: ${{ env.TAG }}
from cortx-re.
@shailesh-vaidya I just joined as a developer advocate on @johnbent 's team. I'm also new to CORTX and not familiar with the cortx-re module.
I'd love to help more and work with you on this. Could you help me understand more about these Docker images and how we can utilize them?
from cortx-re.
@justinzw Sure. May be we can have call and I will show you the way we are doing product releases currently using Jenkins and Docker.
from cortx-re.
@shailesh-vaidya, What is the difference between DEFAULT_PASSWORD
and GITHUB_TOKEN
?
from cortx-re.
@gauravchaudhari02 As discussed DEFAULT_PASSWORD is used to set password for user in container. Which was required for SSH connections in Jenkins. We will not require it in GitHub actions. I will remove it.
from cortx-re.
@shailesh-vaidya Would it be good to avoid set ENV
step?
Instead use GitHub context variables github.repository
, github.event.repository.name
replacing new environment variables INPUT_REPOSITORY
, IMAGE_NAME
and move TAG
environment variable to env
section of step Build and push Docker images
from cortx-re.
I have updated workflow based on inputs. Please review.
steps:
- uses: actions/checkout@v2
- name: Set ENV
run: |
echo ::set-env name=INPUT_REPOSITORY::$( echo $GITHUB_REPOSITORY | tr '[:upper:]' '[:lower:]')
echo ::set-env name=IMAGE_NAME::cortx_centos
echo ::set-env name=TAG::7.7.1908
- name: Build and push Docker images
uses: docker/[email protected]
env:
DEFAULT_PASSWORD: ${{ secrets.DEFAULT_PASSWORD }}
REPO_NAME: ${{ github.event.repository.name }}
with:
# Path to the Dockerfile (Default is '{path}/Dockerfile')
dockerfile: docker/cortx_components/centos/Dockerfile
# Build Arguments for docker image build.
build_args: 'CENTOS_RELEASE=7.7.1908'
path: docker/cortx_components/centos/
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: docker.pkg.github.com
repository: ${{ env.INPUT_REPOSITORY }}/${{ env.IMAGE_NAME }}
tags: ${{ env.TAG }}
from cortx-re.
@shailesh-vaidya Would it be good to avoid
set ENV
step?
Instead use GitHub context variablesgithub.repository
,github.event.repository.name
replacing new environment variablesINPUT_REPOSITORY
,IMAGE_NAME
and moveTAG
environment variable toenv
section of stepBuild and push Docker images
@gauravchaudhari02 Can you please test and add snippet from workflow here.
from cortx-re.
@gauravchaudhari02 As mentioned in #39 we can use ENTRYPOINT in Dockerfile so we will not need TOKEN: ${{ secrets.USER_TOKEN }}
as build_arg. We can move logic of downloading third party packages in component wise build scripts. will this help?
from cortx-re.
Hey Shailesh,
How can we have docker images if our code has kernel dependencies?
CC @xahmad
from cortx-re.
@johnbent To build artifacts we need required files, modules etc. in place and available for build scripts in build environment. This can be achieved using Docker with installing all required rpm's inside container. This helps in setting up isolated environments to build multiple components at same time on Docker Host (VM in our case). We can install kernel and kernel devel packages inside container required to build Motr and S3Server rpm's. We have created component specific Dockerfile's and using them to build artifacts in CI workflow
from cortx-re.
Thanks @shailesh-vaidya ! So I think I understand. We create dockerfiles with all necessary dependencies inside that allow us to build motr and S3 rpm's.
Can we create dockerfiles with motr and S3 already built inside and use them to run CORTX? Or we can only use them to build but not run?
from cortx-re.
@johnbent AFAIK Kernel is shared between Docker containers so even through we create Dockerfile's with Motr + S3 installed. We will be able to run only one such container on VM at a time. IO operation of multiple Motr+S3 containers might cause issue. However this need to be validated practically.
from cortx-re.
@gauravchaudhari02 As you mentioned removed set ENV
step .
Added path
argument. This is due to directory structure in cortx-re repository. We can use it make workflow independent of repository directory structure. Also we do not need to provide Dockerfile
path explicitly and copy of additional files like .repo files scripts can be handled.
Also need to give cortx_centos
explicitly as repo name is not same as images as in S3Sever cases. I feel that much customization is ok. Please review.
Below is code snippet.
name: Base Docker Image CI
on:
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build and push ${{ github.event.repository.name }} docker image
uses: docker/[email protected]
env:
TOKEN: ${{ secrets.TOKEN }}
TAG: 7.7.1908
with:
path: docker/cortx_components/centos/
build_args: 'CENTOS_RELEASE=7.7.1908' # Define build arguments required in your dockerfile here
username: ${{ github.actor }}
password: ${{ env.TOKEN }}
registry: docker.pkg.github.com
repository: ${{ github.repository }}/cortx_centos
tags: ${{ env.TAG }}
from cortx-re.
Also we do not need to provide Dockerfile path explicitly and copy of additional files like .repo files scripts can be handled.
Do you mean that we will provide path
variable instead of giving dockerfile
variable?
Are we planning to handle .repo
file copy in scripts?
from cortx-re.
@gauravchaudhari02 - We need to provide .repo files and custom scripts for some of the components. like for CSM during Docker Image creation.
from cortx-re.
We can finalize below workflow as Generic one to build Docker Images. Let me know if anyone has any issues. If we are good we can add this as part of documentation.
steps:
- uses: actions/checkout@v2
- name: Build and push ${{ github.event.repository.name }} docker image
uses: docker/[email protected]
env:
TOKEN: ${{ secrets.TOKEN }}
TAG: 7.7.1908
with:
path: docker/cortx_components/centos/
build_args: 'CENTOS_RELEASE=7.7.1908' # Define build arguments required in your dockerfile here
username: ${{ github.actor }}
password: ${{ env.TOKEN }}
registry: docker.pkg.github.com
repository: ${{ github.repository }}/cortx-centos
tags: ${{ env.TAG }}
from cortx-re.
This issue/pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
from cortx-re.
This issue/pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
from cortx-re.
Related Issues (20)
- CVE-2022-36087 (Medium) detected in oauthlib-3.2.1-py3-none-any.whl - autoclosed HOT 7
- Test Board-Genius Sync HOT 2
- CVE-2022-24439 (Critical) detected in GitPython-3.1.29-py3-none-any.whl HOT 1
- CVE-2022-23491 (Medium) detected in certifi-2022.9.24-py3-none-any.whl - autoclosed HOT 1
- CVE-2022-40897 (Medium) detected in setuptools-36.0.1-py2.py3-none-any.whl HOT 1
- CVE-2023-23931 (Medium) detected in cryptography-3.3.2-cp36-abi3-manylinux2010_x86_64.whl HOT 1
- CVE-2023-0286 (High) detected in cryptography-3.3.2-cp36-abi3-manylinux2010_x86_64.whl HOT 1
- CVE-2023-25577 (High) detected in Werkzeug-2.2.2-py3-none-any.whl HOT 1
- CVE-2023-23934 (Low) detected in Werkzeug-2.2.2-py3-none-any.whl HOT 1
- CVE-2023-30861 (High) detected in Flask-1.1.1-py2.py3-none-any.whl HOT 1
- Merge HW deployment code HOT 4
- Combine NODECLI and OPTIMIZED HW Deployment HOT 4
- CVE-2021-33503 (High) detected in urllib3-1.25.11-py2.py3-none-any.whl HOT 16
- cortx-all image generation instructions for community. HOT 3
- BG test HOT 4
- CVE-2021-21330 (Medium) detected in aiohttp-3.6.1-cp37-cp37m-manylinux1_x86_64.whl - autoclosed HOT 3
- CVE-2021-41496 (High) detected in numpy-1.21.5-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl - autoclosed HOT 3
- CVE-2021-34141 (Medium) detected in numpy-1.21.6-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl HOT 10
- CVE-2022-24302 (Medium) detected in paramiko-2.7.1-py2.py3-none-any.whl - autoclosed HOT 5
- Need to change our template asking for new third party software HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cortx-re.