Comments (8)
@KalleV It's tempting to use longer and longer prompts to try and fix cases that didn't work 100% well.
Usually, there are better ways to go.
You can try your modified prompt on the bad cases you saw. Did it improve ?
To try improve the prompting for this specific issue, I would focus on a different aspect -further explain to the model what are the '+' lines. Maybe be a specific example, simulating a hunk in the prompt, would help
(similar to
(If you prefer, share a public example that gave bad results, and I will try an improved prompt)
from pr-agent.
ok.
notice that the '/improve' tool works differently in terms of patch and prompt design, and I think it will happen less there, so you can you it instead
from pr-agent.
I am closing this issue. if you encounter this again in the '/improve' command, feel free to open another
from pr-agent.
Do you have a public PR you can share with this behavior?
from pr-agent.
I can try to set up a reproduction of this issue but I can see the prompts are clearly trying to avoid this already:
Do you think tweaking that instruction or its placement would make a difference? Something like:
- It is essential to avoid suggesting changes that mirror modifications already introduced in the new PR code (the '+' lines). Ensure that your feedback brings new insights and does not reiterate the changes already incorporated in the PR.
from pr-agent.
Cc @mrT23
from pr-agent.
I wasn't able to create a minimum public reproduction yet but I came across a PR consistently requesting a try/catch. Not sure if the PR size makes a big difference but according to https://llmtokencounter.com/, the system prompt + user prompt is about 4801 tokens 19205 characters 5004 words
.
I couldn't consistently prove small tweaks such as:
+ - It is essential to avoid suggesting changes that mirror modifications already introduced in the new PR code (the '+' lines).
- - Make sure not to provide suggestions repeating modifications already implemented in the new PR code (the '+' lines).
Or rearranging the system instructions led to a consistently better response but it was fun to try!
from pr-agent.
Sounds good, thank you!
from pr-agent.
Related Issues (20)
- [feature request] new command `/todo` that opens an issue with details HOT 1
- adding a /help command HOT 1
- Not working with GitHub enterprise? HOT 2
- Add azure_devops_webhook tag to images published on Docker Hub HOT 1
- Fails to process Azure DevOps PR URLs that include spaces HOT 1
- Feature Request - Support Claude3 model via Amazon Bedrock HOT 6
- [GitHub Release] Could you create a new release from latest develop branch. HOT 2
- bug in running pr agent Improve HOT 4
- Please capture and return usage HOT 1
- Issues in Bitbucket pipeline HOT 3
- Layout of improve response for Azure DevOps pull requests is malformed HOT 2
- Feature request: More examples HOT 3
- Codium Agent Deployment Issue via GitHub App HOT 2
- pr-agent module publishing to pypi HOT 3
- Environment variables override HOT 4
- Can I change the response language to something other than English? HOT 2
- how to configure anthropic on github ? HOT 2
- AWS Lambda unhandled exception HOT 6
- Trigger condition improvement in github actions
- Add persistent_comment option for `/describe` HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
š Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ššš
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ā¤ļø Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pr-agent.