Coder Social home page Coder Social logo

galaxy-future / costpilot Goto Github PK

View Code? Open in Web Editor NEW
28.0 3.0 20.0 5.2 MB

CostPilot is an all-in-one cloud cost management platform that applies to applications hosted on all cloud platforms.

Home Page: https://www.galaxy-future.com/productCostPilot

License: Apache License 2.0

Dockerfile 0.65% Makefile 0.41% Go 98.38% Shell 0.18% CSS 0.03% HTML 0.35% JavaScript 0.01%
cloudnative finops

costpilot's Introduction

CostPilot

CodeFactor

Language

English | 中文

Introduction

CostPilot is an all-in-one cloud cost management platform that applies to applications hosted on all cloud platforms. Developed by Galaxy-Future based on the FinOps discipline, CostPilot helps enterprises cut cloud spending by up to 50%. This platform provides deep insights into your cloud cost, periodic reports that contain both cloud cost data and professional cost optimization suggestions from multiple perspectives, and even state-of-the-art techniques that you can use to efficiently optimize cloud usage.

Getting Started Guide

1. Configuration Requirements

  • To ensure stable operation of the system, we recommend that you use a server with at least 2 CPU cores and 4 GB RAM. For Linux and MacOS systems, CostPilot has already been tested.
  • The AK/SK used in operation needs to have the permission to read the bill (charge) of the corresponding cloud account.

2. Run in Source Code

To run CostPilot in source code, you need to install Git ( Git - Downloads) and Go (version 1.17+ is required) and set your Go workspace first.

  • (1) Download source code

    git clone https://github.com/galaxy-future/costpilot.git

  • (2) Use environment variables. You can analyze only one cloud account at a time if you use this method.
    • Replace 'abc' with your own AK/SK.
         COSTPILOT_PROVIDER=AlibabaCloud COSTPILOT_AK=abc COSTPILOT_SK=abc COSTPILOT_REGION_ID=cn-beijing go run .
  • (3) Use a configuration file. You can analyze multiple cloud accounts at a time if you use this method.
    • Edit conf/config.yaml as follows. You can add multiple items in cloud_accounts. Only AlibabaCloud is supported as provider for now.
       cloud_accounts:
         - provider:  # required :AlibabaCloud
           ak:  # required
           sk:  # required
           region_id:  # required
           name:  # not required
    • Execute the following make command:
      make build && make run
    • After the analysis is complete, visit website/index.html in a browser to view the analysis result.

3. Run in Docker

To run CostPilot in Docker, you need to install Docker first. For more information, see Docker Engine Install.

  • (1) Use environment variables. You can analyze only one cloud account at a time if you use this method.
    • Replace 'abc' with your own AK/SK.
      docker run --env COSTPILOT_PROVIDER=AlibabaCloud --env COSTPILOT_AK=abc --env COSTPILOT_SK=abc --env COSTPILOT_REGION_ID=cn-beijing -p 8504:8504 --name=costpilot galaxyfuture/costpilot
  • (2) Use a configuration file. You can analyze multiple cloud accounts at a time if you use this method.
    • Create your own config.yaml file, and then execute the following command. Replace /tmp/config.yaml with the absolute path of your config.yaml file.
      docker run --mount type=bind,source=/tmp/config.yaml,target=/home/tiger/app/conf/config.yaml -p 8504:8504 --name=costpilot galaxyfuture/costpilot

4. Sample Result

bill utilization

5. Data Flow Diagram

costpilot-architecture

Code of Conduct

Contributor Convention

Authorization

CostPilot uses Apache License 2.0 licensing agreement for authorization.

Contact Us

Weibo | Zhihu | Bilibili | WeChat Official Account

If you want more information about the service, scan the following QR code to contact us:

image

costpilot's People

Contributors

baiyutang avatar ceciliachen123 avatar lbsu avatar ll-res avatar lshmouse avatar michael-zzz avatar sdaulibin avatar zvanyang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

costpilot's Issues

[Task][HuaweiCloud] Cost Analysis Support Huawei

背景

为 CostPilot 增加支持 华为云 账单分析所需的底层 openapi 对接。

任务

  • 完成 HuaweiCloud 的 QueryAccountBill 接口对接;
    (代码入口:internal/providers/huawei/api.go)
  • 完成相应的单元测试;
    (代码入口:internal/providers/huawei/api_test.go)

技能要求和编程语言

  • 熟悉华为云 openapi 对接;
  • Golang;

预期完成结果

  • 可通过 CostPilot 完成 华为云 账单分析;
  • 有相对完整的单元测试用例;

关于 PR

dev 分支为默认主分支, issue 完成后请向 test 分支提交 PR。

参考

CostPilot 任务对接指南


开发者可扫码添加官方助手,入群沟通 :)

[Task][TencentCloud] Cost Analysis Support Tencent

背景

为 CostPilot 增加支持 腾讯云 账单分析所需的底层 openapi 对接。

任务

  • 完成 Tencent 的 QueryAccountBill 接口对接;
    (代码入口:internal/providers/tencent/api.go)
  • 完成相应的单元测试;
    (代码入口:internal/providers/tencent/api_test.go)

技能要求和编程语言

  • 熟悉腾讯云 openapi 对接;
  • Golang;

预期完成结果

  • 可通过 CostPilot 完成 腾讯云 账单分析;
  • 有相对完整的单元测试用例;

关于 PR

dev 分支为默认主分支, issue 完成后请向 test 分支提交 PR。

参考

CostPilot 任务对接指南


开发者可扫码添加官方助手,入群沟通 :)

[Task][TencentCloud] Utilization Analysis Support Tencent

背景

利用率看板要支持腾讯云所需的底层 openapi 对接。

任务

  • 完成 腾讯云 的相关接口对接;
    (代码入口:internal/providers/tencent/api.go)
  • 完成相应的单元测试;
    (代码入口:internal/providers/tencent/api_test.go)

技能要求和编程语言

  • 熟悉 腾讯云 openapi 对接;
  • Golang;

预期完成结果

  • 可通过 CostPilot 完成 腾讯云 利用率分析;
  • 有相对完整的单元测试用例,注意不要误操作把密钥提交;

关于 PR

[重要] 从 feat-tencent-utilization 分支 fork, 完成后向 feat-tencent-utilization 分支提 PR

参考

CostPilot-20221122 利用率看板任务对接指南


开发者可扫码添加官方助手,入群沟通 :)

[Task][BaiduCloud] Cost Analysis Support Baidu

背景

为 CostPilot 增加支持 百度云 账单分析所需的底层 openapi 对接。

任务

  • 完成 Baidu 的 QueryAccountBill 接口对接;
    (代码入口:internal/providers/baidu/api.go)
  • 完成相应的单元测试;
    (代码入口:internal/providers/baidu/api_test.go)

技能要求和编程语言

  • 熟悉百度云 openapi 对接;
  • Golang;

预期完成结果

  • 可通过 CostPilot 完成 百度云 账单分析;
  • 有相对完整的单元测试用例;

关于 PR

[重要] 从 baidu-costanalysis 分支 fork, 完成后向 baidu-costanalysis 分支提 PR

参考

CostPilot 任务对接指南


开发者可扫码添加官方助手,入群沟通 :)

[Bug][alibaba] Fetch cpu/memory metrics failed for too many items

2022/11/22 14:46:15 E! get cloud-acount[alicloud] utilization error = SDKError:
   StatusCode: 500
   Code: InternalError
   Message: code: 500, The request processing has failed due to some unknown error. request id: 6FE09DA8-CC2D-58A6-91A7-696783B17D5E
   Data: {"Code":"InternalError","HostId":"[metrics.cn-hangzhou.aliyuncs.com](http://metrics.cn-hangzhou.aliyuncs.com/)","Message":"The request processing has failed due to some unknown error.","Recommend":"https://next.api.aliyun.com/troubleshoot?q=InternalError\u0026product=Cms","RequestId":"6FE09DA8-CC2D-58A6-91A7-696783B17D5E","statusCode":500}

In alicloud, Fetching daily instance cpu/memory metrics always failed for too many data, we need reduce the length of DescribeMetricListRequest and support multi pages

[Task][BaiduCloud] Utilization Analysis Support Baidu

背景

利用率看板要支持百度云所需的底层 openapi 对接。

任务

  • 完成 Baidu 的相关接口对接;
    (代码入口:internal/providers/baidu/api.go)
  • 完成相应的单元测试;
    (代码入口:internal/providers/baidu/api_test.go)

技能要求和编程语言

  • 熟悉百度云 openapi 对接;
  • Golang;

预期完成结果

  • 可通过 CostPilot 完成 百度云 利用率分析;
  • 有相对完整的单元测试用例,注意不要误操作把密钥提交;

关于 PR

[重要] 从 feat-baidu-utilization 分支 fork, 完成后向 feat-baidu-utilization 分支提 PR

参考

CostPilot-20221122 利用率看板任务对接指南


开发者可扫码添加官方助手,入群沟通 :)

[Task][AWSCloud] Cost Analysis Support AWS

背景

为 CostPilot 增加支持 AWS 账单分析所需的底层 openapi 对接。

任务

  • 完成 AWSCloud 的 QueryAccountBill 接口对接;
    (代码入口:internal/providers/aws/api.go)
  • 完成相应的单元测试;
    (代码入口:internal/providers/aws/api_test.go)

技能要求和编程语言

  • 熟悉 AWS openapi 对接;
  • Golang;

预期完成结果

  • 可通过 CostPilot 完成 AWS 账单分析;
  • 有相对完整的单元测试用例;

关于 PR

dev 分支为默认主分支, issue 完成后请向 test 分支提交 PR。

参考

CostPilot 任务对接指南


开发者可扫码添加官方助手,入群沟通 :)

🎁[Task][AWS] Utilization Analysis Support AWS

背景

利用率看板要支持 AWS 所需的底层 openapi 对接。

任务

  • 完成 AWS 的相关接口对接;
    (代码入口:internal/providers/aws/api.go)
  • 完成相应的单元测试;
    (代码入口:internal/providers/aws/api_test.go)

技能要求和编程语言

  • 熟悉 AWS openapi 对接;
  • Golang;

预期完成结果

  • 可通过 CostPilot 完成 AWS 利用率分析;
  • 有相对完整的单元测试用例,注意不要误操作把密钥提交;

关于 PR

[重要] 从 feat-aws-utilization 分支 fork, 完成后向 feat-aws-utilization 分支提 PR

参考

CostPilot-20221122 利用率看板任务对接指南


开发者可扫码添加官方助手,入群沟通 :)

[Task][HuaweiCloud] Utilization Analysis Support Huawei

背景

利用率看板要支持华为云所需的底层 openapi 对接。

任务

  • 完成华为云的相关接口对接;
    (代码入口:internal/providers/huawei/api.go)
  • 完成相应的单元测试;
    (代码入口:internal/providers/huawei/api_test.go)

技能要求和编程语言

  • 熟悉华为云 openapi 对接;
  • Golang;

预期完成结果

  • 可通过 CostPilot 完成 华为云利用率分析;
  • 有相对完整的单元测试用例,注意不要误操作把密钥提交;

关于 PR

[重要] 从 feat-huawei-utilization 分支 fork, 完成后向 feat-huawei-utilization 分支提 PR

参考

CostPilot-20221122 利用率看板任务对接指南


开发者可扫码添加官方助手,入群沟通 :)

[Bug][alibaba] alicloud QueryAvailableInstances failed when instance number > 100

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1302c54]

goroutine 1 [running]:
github.com/galaxy-future/costpilot/internal/providers/alibaba.(*AlibabaCloud).QueryAvailableInstances(0xc0003edb80, {0x0?, 0x0?}, {{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0xc000b50000, ...}})
	/home/qcraft/workspace/costpilot/internal/providers/alibaba/api.go:427 +0x354
github.com/galaxy-future/costpilot/internal/services/datareader.(*UtilizationDataReader).GetInstanceList(0xc000890ba0, {0x1dc79c8, 0xc000040160}, {0xc000b50000, 0x933, 0x8?})
	/home/qcraft/workspace/costpilot/internal/services/datareader/utilizationdatareader.go:169 +0x8d
github.com/galaxy-future/costpilot/internal/services/databean.(*UtilizationDataBean).fetchRecentInstanceList(0xc0009d81e0, {0x1dc79c8, 0xc000040160})
	/home/qcraft/workspace/costpilot/internal/services/databean/utilization_databean.go:168 +0x312
github.com/galaxy-future/costpilot/internal/services/databean.(*UtilizationDataBean).RunPipeline(0xc000489160?, {0x1dc79c8, 0xc000040160})
	/home/qcraft/workspace/costpilot/internal/services/databean/utilization_databean.go:209 +0x193
github.com/galaxy-future/costpilot/internal/domain.(*ResourceUtilizationDomain).GetUtilization(0xc000838120?, {0x1dc79c8, 0xc000040160}, {{0xc000489160, 0xc}, {0xc00003e780, 0x18}, {0xc0005809a0, 0x1e}, {0xc0004891a0, ...}, ...})
	/home/qcraft/workspace/costpilot/internal/domain/resource_utilization.go:35 +0x7f
github.com/galaxy-future/costpilot/internal/domain.(*ResourceUtilizationDomain).GetUtilizationData(0xc000e2ff10, {0x1dc79c8, 0xc000040160})
	/home/qcraft/workspace/costpilot/internal/domain/resource_utilization.go:51 +0x218
github.com/galaxy-future/costpilot/internal/domain.(*ResourceUtilizationDomain).RunPipeline(0x1a19428?, {0x1dc79c8, 0xc000040160})
	/home/qcraft/workspace/costpilot/internal/domain/resource_utilization.go:87 +0x9b
main.main()
	/home/qcraft/workspace/costpilot/main.go:27 +0x185
Makefile:23: recipe for target 'run' failed
make: *** [run] Error 2

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.