wxdwfc / rlib Goto Github PK
View Code? Open in Web Editor NEWRLib is a header-only library for easier usage of RDMA.
RLib is a header-only library for easier usage of RDMA.
你好,感谢开源rlib!
我正在使用rlib库完成多节点部署,但是在部署时,我发现跨节点通信会报标题这个错误,在单节点中是没有问题的。
我使用RdmaCtrl来建立qp连接,建立连接的过程没有任何问题,但是当建立连接后,使用RDMA read就会报错。
我想请问rlib以前测试的时候测试过多节点吗?
感谢解答!
您好,感谢开源rlib!我在使用过程中,遇到了一个问题:
背景:client端(在机器1)使用线程t1和server端(在机器2)建立2个RCQP连接(QP1,QP2)后,client端的t1线程内创建一个新线程t2。接下来,t1使用QP1对server进行one-sided RDMA READ,t2使用QP2对server进行one-sided RDMA WRITE。t1和t2的RDMA READ/WRITE是并行的(读写没有任何冲突)。
到这里本应该没有问题,但是t2的RDMA WRITE却无法写成功(通过查看server端mem region未被修改而得知),导致t2在poll cq时出现 “transport retry counter exceeded” 报错。
查阅RDMA Aware Networks Programming User Manual (Rev 1.7),该错误的解释是:
6.2.13 IBV_WC_RETRY_EXC_ERR
This event is generated when a sender is unable to receive feedback from the receiver. This
means that either the receiver just never ACKs sender messages in a specified time period, or it
has been disconnected or it is in a bad state which prevents it from responding.
奇怪的是,如果t2使用QP1进行RDMA WRITE,则可以写成功,poll也没问题(注意到QP1和QP2都是使用class RRCQP中的connect函数分2次成功连接的)。
但我并不希望t1和t2共用一个RCQP,因为t1和t2会争抢completion queue,比如t1 poll到了t2的ack,导致t1认为自己的RDMA READ成功了,但实际上可能还没读到remote data。
希望您可以解答,谢谢!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.