Comments (3)
Hi @mohummedalee, OpenAI API is still needed for atomic fact generation. We welcome any contributions that replace it with LLAMA-based atomic fact generation.
from factscore.
@martiansideofthemoon Hi, I was wondering if you have any experience in replacing text davinci with open-source models for atomic fact generation? Will llama2-65b be a viable option? Thanks a lot🙏
from factscore.
Hi @Rosenberg37, a collaborator of @shmsw25 has seen it work well enough with the LLAMA-2 models. However adding them to the codebase for atomic fact extraction may need some non-trivial refactoring. We welcome any contributions here.
from factscore.
Related Issues (20)
- Typo here HOT 1
- FileNotFoundError (demons.json) for custom knowledge base HOT 2
- Regarding human evaluation HOT 2
- The chatgpt-based AtomicFactGenerator of FactScore sometimes abstain from answering HOT 4
- Generation details for models under data/labeled HOT 1
- How to modify the code to calculate factscore between any two paragraphs/sentences without retrieval steps? HOT 1
- Issue running get_score() on M1 Mac due to Torch build HOT 1
- Some cases couldn't be found in the default db HOT 2
- problem using the factscorer function with LLaMa HOT 1
- AssertionError: `topic` in your data (Jessie Mae Brown Beavers) is likely to be not a valid title in the DB. HOT 2
- Move to ChatCompletion endpoint for call_GPT3
- InvalidRequestError: The model `text-davinci-003` has been deprecated HOT 2
- sqlite3.DatabaseError: file is not a database error after Installation
- About the enwiki-20230401 HOT 5
- Batching for speed.
- Improved version for speed and openai updates
- Maintaining core factscore in a new repository
- Strange token ID mapping
- IPython Debugger popped up when I ran the demo code
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from factscore.