Comments (5)
Thanks for trying aider and reporting this issue and sharing the chat history. That's very helpful information.
It looks to me like this may be a problem with the way aider handles subdirectories on Windows. I will continue to investigate and let you know when I've implemented a solution.
In the meantime, you might want to try and avoid using subdirectories and see if that helps. When starting a new project with aider you could run aider like aider quorum_scraper.py
. Aider will make a new empty file without suggesting a subdirectory. Then when you ask for code, it will go into that file.
For example, here is the transcript of me doing that and giving aider your initial prompt:
aider quorum_scraper.py
Model: gpt-4
Creating empty file quorum_scraper.py
Files not tracked in .git:
quorum_scraper.py
Add them? y
Added quorum_scraper.py to the git repo
Commit 07b56f3 Added new files to the git repo: quorum_scraper.py
Git repo: .git
Repo-map: universal-ctags using 1024 tokens
Added quorum_scraper.py to the chat.
Use /help to see in-chat commands.
I want to make a Quorum web scraper that scrapes text from pages and subpages from http://quorumlanguage.com. The script will create text files so that a large language model can understand them.
To create a web scraper for the Quorum language website, we will need to use a library like BeautifulSoup to parse the HTML of the website and extract the text. We will also need the requests library to send HTTP requests to the website.
Here are the steps we will follow:
- Send a GET request to the main page of the website.
- Parse the HTML of the main page to find links to subpages.
- For each subpage, send a GET request to retrieve the HTML.
- Parse the HTML of each subpage to extract the text.
- Write the text to a file.
Here's how we can modify your quorum_scraper.py
file to implement this:
quorum_scraper.py
<<<<<<< ORIGINAL
=======
import requests
from bs4 import BeautifulSoup
import os
def get_page_text(url):
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
return soup.get_text()
def get_subpage_urls(url):
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
return [a['href'] for a in soup.find_all('a', href=True) if a.text]
def write_text_to_file(text, filename):
with open(filename, 'w') as f:
f.write(text)
def main():
url = 'http://quorumlanguage.com'
subpage_urls = get_subpage_urls(url)
for i, subpage_url in enumerate(subpage_urls):
text = get_page_text(subpage_url)
write_text_to_file(text, f'subpage_{i}.txt')
if __name__ == '__main__':
main()
>>>>>>> UPDATED
This script will create a text file for each subpage on the Quorum language website. The text files will be named subpage_0.txt
, subpage_1.txt
, etc.
Applied edit to quorum_scraper.py
Commit 3ea30de aider: Refactorquorum_scraper.py
to include functions for getting page text, subpage URLs, and writing text to a file, and add a main function that retrieves the subpage URLs, gets the text for each subpage, and writes it to a file.
from aider.
Correct me if i'm wrong -- until this issue is fixed, aider won't be able to access existing repos on Windows, only new ones being created, yes?
from aider.
Thanks for your patience on this. I had to set up a cloud windows server to debug this issue.
I just pushed a fix, so aider should stop asking over and over again to add files that are in subdirectories.
You'll need to install the latest from github to pick up this fix. Let me know if if resolves your issue.
from aider.
I also added a CI step to run all the tests on windows on every push now.
https://github.com/paul-gauthier/aider/actions/runs/5469581577
from aider.
I'm going to close this issue for now. Please feel free to re-open or file a new issue if you continue to see problems.
from aider.
Related Issues (20)
- getting stuck on mac osx and on WSL HOT 2
- Filenames beginning with underscore strip the preceding directory separator (backslash; Windows PowerShell)
- Error loading aider litellm model metadata: name 'litellm' is not defined HOT 6
- Improve user input formatting in `--chat-history-file` HOT 2
- Model openrouter/openai/gpt-4o-mini: Unknown [..] HOT 4
- Aider could better handle malformed filenames surrounded by asterisks
- `aider --model codellama` show duplicate candidates on startup HOT 3
- Consider not pinning requirements in for published package HOT 5
- when aider need to add code,it will replace the code whitch shoud not be replaced
- Feature Request: Novita AI HOT 2
- Deepseek API UPDATE (Added 128k context, 8k output, assistant prefix) HOT 3
- [FEATURE] add loop capability to enhance cheap model performance HOT 1
- Are we getting llama 3.1 405b HOT 3
- UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 106: character maps to <undefined>
- aider opens /web for github pages and crashes HOT 3
- Aider is creating high CPU load when dealing with large LLM responses and long patches. HOT 3
- Surprising and annoying first-use experience HOT 13
- Question: git commit messages - does not follow the conventions.md style HOT 6
- aider GUI fails consistently loading a file with StreamlitAPIException about "the default value is part of the options" HOT 6
- I suspect the just-released "Tool support" in ollama 0.3.0, will be really valuable to Aider.? - can this be integrated.? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aider.