Coder Social home page Coder Social logo

tencent / embedx Goto Github PK

View Code? Open in Web Editor NEW
298.0 14.0 47.0 2.76 MB

embedx 是基于 c++ 开发的、完全自研的分布式 embedding 训练和推理框架。它目前支持 图模型、深度排序、召回模型和图与排序、图与召回的联合训练模型等

License: Other

Makefile 0.92% C++ 98.96% C 0.12%

embedx's Introduction

logo

简介

embedx 是基于 c++ 开发的大规模 embedding 训练和推理系统,累计支持公司 12 个业务30 多个团队使用上线百余次

我们在以下推荐、搜索、支付 和 风控等产品落地使用了 embedx: 微信看一看微信视频号微信搜一搜微信支付微信安全腾讯新闻应用宝QQ 音乐JOOX 音乐腾讯课堂领航平台腾讯黑产打击 等 ,并取得了性能和效果双丰收。

更多介绍请参考详细介绍

EmbedX系统的论文发表在PVLDB'2023, 引用 cite:

@article{10.14778/3611540.3611546,
author = {Zou, Yuanhang and Ding, Zhihao and Shi, Jieming and Guo, Shuting and Su, Chunchen and Zhang, Yafei},
title = {EmbedX: A Versatile, Efficient and Scalable Platform to Embed Both Graphs and High-Dimensional Sparse Data},
year = {2023},
volume = {16},
number = {12},
url = {https://doi.org/10.14778/3611540.3611546},
journal = {Proc. VLDB Endow.},
pages = {3543–3556}
}

embedx 已经实现的模型和评测

  • 已经实现的模型

    • 十亿级节点、千亿级边的 图模型
    • 百亿级样本、百亿特征的 深度排序、召回模型
    • 十亿级节点、千亿级边与百亿级样本、百亿特征的 图与深度排序、图与深度召回的联合建模模型
  • 模型以及评测

快速上手

Contributing

常见问题

更多问题可以联系开发者

embedx's People

Contributors

ccsquare avatar honglitao avatar jmshi123 avatar longsail avatar succ9420 avatar tinkle1129 avatar yuanqingsunny avatar zhitao-wang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

embedx's Issues

请教如何部署embedx分布式环境

首先感谢微信大佬开源那么牛掰的工具。在我们业务上单机实现deepwalk并取得正向效果,速度快得飞起,赞。受限于单机资源问题。请教大佬是否可以提供详细的embedx分布式部署教程。非常感谢。

请问如何基于 metapath 进行采样?

感谢开源 EmbedX。

我的问题如下:

在异构图模型中,我们经常会基于 元路径 (metapath) 进行采样,比如 author-paper-conference 。

看了相关的示例和代码,没有找到如何基于 metapath 进行采样,请问是否可以写个示例 run_metapth2vec.sh ?

谢谢!

请问分布式部署可否支持

大佬好,我们在业务场景应用embedx,性能非常棒,效果也挺好,但是我们数据规模较大,几百亿边,请问是否可以支持分布式部署,谢谢

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.