This repository contains the code and resources to generate two essay datasets:
- Paul Graham Essay Collection
- Sam Altman Essay Collection
These datasets are designed for various natural language processing tasks such as question-answering, summarization, text generation, and text-to-text generation. The essays in these datasets provide valuable insights and perspectives on a range of topics related to startups, technology, artificial intelligence, leadership, and personal growth.
The Paul Graham Essay Collection dataset contains a complete collection of essays written by Paul Graham, a renowned programmer, venture capitalist, and essayist. The essays cover a wide range of topics including startups, programming, technology, entrepreneurship, and personal growth. Each essay has been cleaned and processed to extract the title, date of publication, and the full text content.
HuggingFace Dataset: https://huggingface.co/datasets/sgoel9/paul_graham_essays
The Sam Altman Essay Collection dataset contains a complete collection of essays written by Sam Altman, an entrepreneur, investor, and former president of Y Combinator. The essays cover a wide range of topics including startups, technology, artificial intelligence, leadership, and personal growth. Each essay has been cleaned and processed to extract the title, date of publication, and the full text content.
HuggingFace Dataset: https://huggingface.co/datasets/sgoel9/sam_altman_essays
The repository is organized as follows:
-
paul_graham_essays/
: All data and scripts to preprocess and generate the Paul Graham Essay Collection dataset./dataset
: The generated Paul Graham Essay Collection dataset./text_data
: Raw essay text from Paul Graham's blog.paul_graham_essays_card.md
: Dataset card for the Paul Graham Essay Collection.
-
sam_altman_essays/
: All data and scripts to preprocess and generate the Sam Altman Essay Collection dataset./dataset
: The generated Sam Altman Essay Collection dataset./sam_altman_essays_card.md
: Dataset card for the Sam Altman Essay Collection.
-
requirements.txt
: The required Python dependencies for running the scripts.
-
Clone the repository:
git clone https://github.com/your-username/essay-datasets.git
-
Install the required Python dependencies:
pip install -r requirements.txt
-
Run the preprocessing scripts to generate the essay datasets in jupyter
The generated datasets will be saved in the
datasets/
directories. -
Use the generated datasets for your natural language processing tasks.
The datasets generated by this code are released under the MIT License. When using these datasets, please attribute them to Paul Graham and Sam Altman, and provide links to their respective websites:
- Paul Graham: http://www.paulgraham.com/
- Sam Altman: https://blog.samaltman.com/
For any questions or inquiries about this repository or the generated datasets, please contact [email protected].