rotemweiss57 / gpt-newspaper

GPT based autonomous agent designed to create personalized newspapers tailored to user preferences.

GPT Newspaper

Welcome to the GPT Newspaper project, an innovative autonomous agent designed to create personalized newspapers tailored to user preferences. GPT Newspaper revolutionizes the way we consume news by leveraging the power of AI to curate, write, design, and edit content based on individual tastes and interests.

🔍 Overview

GPT Newspaper consists of six specialized sub-agents in LangChain's new LangGraph Library:

1. Search Agent: Scours the web for the latest and most relevant news.
2. Curator Agent: Filters and selects news based on user-defined preferences and interests.
3. Writer Agent: Crafts engaging and reader-friendly articles.
4. Critique Agent Provide feedback to the writer until article is approved.
5. Designer Agent: Layouts and designs the articles for an aesthetically pleasing reading experience.
6. Editor Agent: Constructs the newspaper based on produced articles.
7. Publisher Agent Publishes the newspaper to the frontend or desired service

Each agent plays a critical role in delivering a unique and personalized newspaper experience.

Demo

gpt-newspaper-demo.1.mp4

🌟 Features

• Personalized Content: Get news that aligns with your interests and preferences.
• Diverse Sources: Aggregates content from a wide range of reputable news sources.
• Engaging Design: Enjoy a visually appealing layout and design.
• Quality Assurance: Rigorous editing ensures reliable and accurate news reporting.
• User-Friendly Interface: Easy-to-use platform for setting preferences and receiving your newspaper.

🛠️ How It Works

1. Setting Preferences: Users input their interests, preferred topics, and news sources.
2. Automated Curation: The Search and Curator Agents find and select news stories.
3. Content Creation: The Writer Agent drafts articles, which are then designed by the Designer Agent.
4. Newspaper Design: The Editor Agent reviews and finalizes the content.
5. Delivery: Users receive their personalized newspaper to their mailbox.

🚀 Getting Started

Prerequisites

• Tavily API Key - Sign Up
• OpenAI API Key - Sign Up

Installation

1. Clone the repo

   git clone https://github.com/rotemweiss57/gpt-newspaper.git

2. Export your API Keys

    export TAVILY_API_KEY=<YOUR_TAVILY_API_KEY>
    export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>

3. Install Requirements

   pip install -r requirements.txt

4. Run the app

    python app.py

5. Open the app in your browser

    http://localhost:5000/

6. Enjoy!

🤝 Contributing

Interested in contributing to GPT Newspaper? We welcome contributions of all kinds! Check out our Contributor's Guide to get started.

🛡️ Disclaimer

GPT Newspaper is an experimental project and provided "as-is" without any warranty. It's intended for personal use and not as a replacement for professional news outlets.

📩 Contact Us

For support or inquiries, please reach out to us:

• Email

Join us in redefining the future of news consumption with GPT Newspaper!

gpt-newspaper's Issues

422 Client Error: Unprocessable Entity for url: https://api.tavily.com/search

Been digging through the code. Is this reproducable?

Press CTRL+C to quit 127.0.0.1 - - [27/Jan/2024 08:58:17] "OPTIONS /generate_newspaper HTTP/1.1" 200 - [2024-01-27 08:58:17,850] ERROR in app: Exception on /generate_newspaper [POST] Traceback (most recent call last): File "/home/dirk/.local/lib/python3.10/site-packages/flask/app.py", line 1463, in wsgi_app response = self.full_dispatch_request() File "/home/dirk/.local/lib/python3.10/site-packages/flask/app.py", line 872, in full_dispatch_request rv = self.handle_user_exception(e) File "/home/dirk/.local/lib/python3.10/site-packages/flask_cors/extension.py", line 176, in wrapped_function return cors_after_request(app.make_response(f(*args, **kwargs))) File "/home/dirk/.local/lib/python3.10/site-packages/flask/app.py", line 870, in full_dispatch_request rv = self.dispatch_request() File "/home/dirk/.local/lib/python3.10/site-packages/flask/app.py", line 855, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] File "/home/dirk/dev/gpt_newspaper/gpt-newspaper/backend/server.py", line 14, in generate_newspaper newspaper = master_agent.run(data["topics"], data["layout"]) File "/home/dirk/dev/gpt_newspaper/gpt-newspaper/backend/langgraph_agent.py", line 52, in run parallel_results = list(executor.map(lambda q: chain.invoke({"query": q}), queries)) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator yield _result_or_cancel(fs.pop()) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel return fut.result(timeout) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.__get_result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/home/dirk/dev/gpt_newspaper/gpt-newspaper/backend/langgraph_agent.py", line 52, in <lambda> parallel_results = list(executor.map(lambda q: chain.invoke({"query": q}), queries)) File "/home/dirk/.local/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 569, in invoke for chunk in self.stream( File "/home/dirk/.local/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 605, in transform for chunk in self._transform_stream_with_config( File "/home/dirk/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1497, in _transform_stream_with_config chunk: Output = context.run(next, iterator) # type: ignore File "/home/dirk/.local/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 347, in _transform _interrupt_or_proceed(done, inflight, step) File "/home/dirk/.local/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 688, in _interrupt_or_proceed raise exc File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/home/dirk/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3887, in invoke return self.bound.invoke( File "/home/dirk/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2053, in invoke input = step.invoke( File "/home/dirk/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3353, in invoke return self._call_with_config( File "/home/dirk/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1246, in _call_with_config context.run( File "/home/dirk/.local/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 326, in call_func_with_variable_args return func(input, **kwargs) # type: ignore[call-arg] File "/home/dirk/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3229, in _invoke output = call_func_with_variable_args( File "/home/dirk/.local/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 326, in call_func_with_variable_args return func(input, **kwargs) # type: ignore[call-arg] File "/home/dirk/dev/gpt_newspaper/gpt-newspaper/backend/agents/search.py", line 21, in run res = self.search_tavily(article["query"]) File "/home/dirk/dev/gpt_newspaper/gpt-newspaper/backend/agents/search.py", line 12, in search_tavily results = tavily_client.search(query=query, topic="news", max_results=10, include_images=True) File "/home/dirk/.local/lib/python3.10/site-packages/tavily/tavily.py", line 48, in search return self._search(query, search_depth=search_depth, **kwargs) File "/home/dirk/.local/lib/python3.10/site-packages/tavily/tavily.py", line 42, in _search response.raise_for_status() # Raises a HTTPError if the HTTP request returned an unsuccessful status code File "/home/dirk/.local/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://api.tavily.com/search 127.0.0.1 - - [27/Jan/2024 08:58:17] "POST /generate_newspaper HTTP/1.1" 500 -

work badly with chinese content

write to file need to add encode utf-8 like :

    with open(path, 'w', encoding='utf-8') as file:
        file.write(article['html'])

tokens limit

{'error': {'message': "This model's maximum context length is 16385 tokens. However, your messages resulted in 20189 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}

The Writer Agent cant write long text article and also cant receive long text prompt, I think we need to disassemble write task to slice task, for example tell gpt there several source articles need to read and help to summary it, after that , give all the summarys to gpt and ask it to write a article outline with json response, then we send the part of json prompt to write part of article.

The very important work is need to keep the gpt konw the prompt context all time ,we need to contact before task when send prompt。
while, I am trying to do it although i dont know the result.

There is no result on the test

Greetings. We performed all the actions in the Windows environment in the terminal. We launched python app.py and http://localhost:5000/.
We specified the topic in the window that opened, but there was no result.

Article prompts need filtering for special characters that cannot be part of a valid file name

"FileNotFoundError: [Errno 2] No such file or directory: 'outputs/run_1706269702/The_US_GDP_Numbers,_opinion_of_Top_Fund_Managers_and/or_Bank_CEO.html'"

It looks like if the article prompt has characters that can't be part of a valid file name the program breaks before completing with an error message similar to the above.

Allow GPT 4 Turbo?

Is it possible to use GPT 4 Turbo instead of GPT 4? If not it would be a nice feature.

422 Client Error: Unprocessable Entity for url: https://api.tavily.com/search

Hey, I just cloned the repo and got it up and running on Mac OS(Ventura 13.4) with Python version 3.9.6

I tried making a request with the words: 'Martin Luther King' and 'super bowl' and got this error:

HTTPError: 422 Client Error: Unprocessable Entity for url: https://api.tavily.com/search

And In the console of the browser, I got this error message:
POST http://localhost:8000/generate_newspaper 500 (INTERNAL SERVER ERROR) produceNewspaper @ scripts.js:36

400 Client Error: Bad Request for url: https://api.tavily.com/search

Just cloned and started the creation process, get that error and it hangs.
MacOS: 14.2.1 (23C71)
Python3.9

requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.tavily.com/search
127.0.0.1 - - [25/Jan/2024 11:03:57] "POST /generate_newspaper HTTP/1.1" 500 -

Type error: unexpected keyword argument 'start_key'

Hi
I installed the software in a virtual environment, python 3.11, win10, 64bit

When I click on the "produce newspaper " button nothing happens but a short "hick" of the page.
The console shows the following error:

File "C:\Users\Arnd\Desktop 3\gpt-newspaper\gpt-newspaper\backend\langgraph_agent.py", line 39, in run workflow.add_conditional_edges(start_key='critique', TypeError: Graph.add_conditional_edges() got an unexpected keyword argument 'start_key'

Deleting this keyword and the next two ('condition' and 'conditional_edge_mapping') solved the problem.

You should mention to rename .env.dist to .env after the github cloning.

Thanks for this great project.

HTTPError('400 Client Error: Bad Request for url: https://api.tavily.com/search')

Tried to execute the sample tavily search code on tavily website (https://docs.tavily.com/docs/tavily-api/langchain) and got an error. Below is the code and error I am getting

import os
from langchain.utilities.tavily_search import TavilySearchAPIWrapper
from langchain.agents import initialize_agent, AgentType
from langchain_community.chat_models import ChatOpenAI
from langchain.tools.tavily_search import TavilySearchResults

# set up API key
os.environ["TAVILY_API_KEY"] = "..."

# set up the agent
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
search = TavilySearchAPIWrapper()
tavily_tool = TavilySearchResults(api_wrapper=search)

# initialize the agent
agent_chain = initialize_agent(
    [tavily_tool],
    llm,
    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
)

# run the agent
agent_chain.run(
    "What happened in the latest burning man floods?",
)

Output:

> Entering new AgentExecutor chain...
Thought: I should use the search engine tool to find information on the latest Burning Man floods.
Action:
{
  "action": "tavily_search_results_json",
  "action_input": "latest Burning Man floods news"
}
Observation: HTTPError('400 Client Error: Bad Request for url: https://api.tavily.com/search')

Text creator

Rate limit exceeded error - Any way to set the rate limit ?

raise self._make_status_error_from_response(err.response) from None

openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4 in organization org-********** on tokens per min (TPM): Limit 10000, Used 9348, Requested 5086. Please try again in 26.604s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}