Comments (2)
Not in the library currently, we are tracking it in #38
from factscore.
@shrusti-ghela A quick workaround would be to replace the davinci model with OpenAI's suggested gpt-3.5-turbo-instruct. (See OpenAI Deprecations)
# Look for this code snippet
def call_ChatGPT(message, model_name="text-davinci-003", max_len=1024, temp=0.7, verbose=False):
# call GPT-3 API until result is provided and then return it
response = None
received = False
num_rate_errors = 0
while not received:
# ...
# And replace it with this workaround:
def call_ChatGPT(message, model_name="gpt-3.5-turbo-instruct", max_len=1024, temp=0.7, verbose=False):
# ...
Assuming you're using a virutal environment you will find this piece of code in venv/lib/python3.9/site-packages/factscore/openai_lm.py
. Note: Normally you shouldn't change the library code in your personal installation, because it will be overwritten when a new version is released, but just to get it working for the moment, this might be fine.
from factscore.
Related Issues (20)
- Having issues on using LLAMA-7B model HOT 2
- Scoring existing generated text HOT 1
- Typo here HOT 1
- FileNotFoundError (demons.json) for custom knowledge base HOT 2
- Regarding human evaluation HOT 2
- The chatgpt-based AtomicFactGenerator of FactScore sometimes abstain from answering HOT 4
- Generation details for models under data/labeled HOT 1
- How to modify the code to calculate factscore between any two paragraphs/sentences without retrieval steps? HOT 1
- Issue running get_score() on M1 Mac due to Torch build HOT 1
- Any way to run without an OpenAI key? HOT 3
- Some cases couldn't be found in the default db HOT 2
- problem using the factscorer function with LLaMa HOT 1
- AssertionError: `topic` in your data (Jessie Mae Brown Beavers) is likely to be not a valid title in the DB. HOT 1
- Move to ChatCompletion endpoint for call_GPT3
- sqlite3.DatabaseError: file is not a database error after Installation
- About the enwiki-20230401 HOT 5
- Batching for speed.
- Improved version for speed and openai updates
- Maintaining core factscore in a new repository
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from factscore.