Jenkins stability about cortx-re

[shailesh-vaidya]

After cleaning up -

[root@ssc-vm-0279 jenkins]# du -sh *
32G     jenkins-dev
79G     jenkins-prod
[root@ssc-vm-0279 jenkins]#

[shailesh-vaidya]

Seems disk space utilization is again increasing.

[root@ssc-vm-0279 jenkins]# du -sh *
32G     jenkins-dev
81G     jenkins-prod
[root@ssc-vm-0279 jenkins]# pwd
/mnt/bigstorage/releases/jenkins
[root@ssc-vm-0279 jenkins]#

[shailesh-vaidya]

• There are many unwanted folders and build's stored. We need to analyze are remove them. I checked disk usage for folders inside /mnt/bigstorage/releases/

[root@ssc-vm-1321 releases]# du -sh *
61G backups
30M ci-storage_backups
1.6T cortx
599G cortx_builds
35G dev
0 eos
338M hermi
5.2G iso_mount
109G jenkins
5.2G local_test
15G master
9.0G mfg_builds
76G opensource_builds
289M sage
[root@ssc-vm-1321 releases]#

From above list, http://cortx-storage.colo.seagate.com/releases/local_test/ and http://cortx-storage.colo.seagate.com/releases/iso_mount/ looks unnecessary to me. @gowthaman We can discuss and remove them. Need to recursively check for all folders. Also we should avoid dumping anything on cortx-storage server.

• Currently we are storing release build at two locations. e.g. for release Build#419

[root@ssc-vm-1321 ~]# du -sh  /mnt/bigstorage/releases/cortx/github/cortx-1.0/centos-7.8.2003/419/
509M    /mnt/bigstorage/releases/cortx/github/cortx-1.0/centos-7.8.2003/419/
[root@ssc-vm-1321 ~]# du -sh  /mnt/bigstorage/releases/cortx_builds/centos-7.8.2003/419
1.7G    /mnt/bigstorage/releases/cortx_builds/centos-7.8.2003/419
[root@ssc-vm-1321 ~]#

Hope after EOS-14452 one of the location will be discarded. @swatiseagate We need to prioritizeEOS-14452

• Custom build creation need to be optimized.
Currently custom builds are consuming disk space. I am working on optimizing it.

• We do not have any disk space usage monitoring for cortx-storage server. Need to setup Prometheus or similar tool for disk usage monitoring.

[hessio]

@shailesh-vaidya what is the status of this issue?

[shailesh-vaidya]

@hessio - We are not seeing any issue due to disk space now. However recently we had to restart Jenkins due to slowness. Let's keep this open to track it.

[cortx-admin]

For the convenience of the Seagate development team, this issue has been mirrored in a private Seagate Jira Server: https://jts.seagate.com/browse/EOS-28329. Note that community members will not be able to access that Jira server but that is not a problem since all activity in that Jira mirror will be copied into this GitHub issue.

[cortx-admin]

Anmol Vijwani commented in Jira Server:

Already was taken care

[cortx-admin]

Anmol Vijwani commented in Jira Server:

Already been taken care of

[cortx-admin]

Anmol Vijwani commented in Jira Server:

Already been taken care of.

[cortx-admin]

Anmol Vijwani commented in Jira Server:

Already been taken care of