tjunlp-lab / m3ke Goto Github PK
View Code? Open in Web Editor NEWA Massive Multi-Level Multi-Subject Knowledge Evaluation benchmark
m3ke's People
Contributors
Stargazers
Watchers
m3ke's Issues
What are expected to submit for the leaderboard integration?
测试集答案为空的话如何对回复结果进行评估
在测试集中，answer字段为“”，在不知道正确答案的情况下，如何对模型的回复效果进行评估和测试呢？
Column name typo
Should be "Model" rather than "Models"
Can you provide some details of the evaluation code for the reported results?
This assessment data also appears to be in the form of multiple choice tasks similar to MMLU, but there are many detailed differences in the practice of MMLU, and these detailed differences have a significant impact on the quality outcome value. Among them, the accuracy calculation method provided by MMLU is based on the probability normalization of four options, from which the maximum probability is selected as the prediction result. However, many others have changed it to the generated form and then extracted the ABCD option from the generated answer, and the prompt setting and the extraction method of the answer will affect the final result.
So what are the evaluation code details based on which the results table is reported in your repository?
请问训练集数据能够提供吗？
我看下载下来的数据只有测试集和验证集
完整数据集
请问下readme说有两万多个问题，但是我git clone下来好像并没有那么多。请问如何获取完整的数据？
The complete dataset
Hi, how can we access the complete dataset? Thanks!
您好，请问是否能够提供训练集呢
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.