Comments (6)
And one more thing, may I know if @qwe661234 is working on this issue? If not, I could help to write the CI script if the above concern is OK.
Please move forward. Before submitting pull request(s), do write down your plans and communicate here.
from rv32emu.
If this CI run is enabled for PRs, the benchmark charts will update with each pull request. Since we must first confirm a performance improvement before updating the benchmark charts, this might not be what we desire. In other words, we need to discuss about how the PR is doing before updating the benchmark chart. Or, is it acceptable to update the benchmark chart with each PR?
If the results generated by the CI are deterministic, we can certainly treat them as references for incremental development. However, due to our current lack of confidence, it is premature to define a clear strategy at this stage.
from rv32emu.
- Run the benchmark workflow and update the GitHub pages whenever "pushing on master branch" and "opening of a pull request and synchronizing the pull request(if any) on master branch" are performed when editing the "emulate.c" and "rv32_template.c" files because these 2 files have the potential to directly impact the benchmark. Under this constraint, we can prohibit excessive benchmark workflow with unrelated files, such as those in docs/ . Do you have any suggestions for other files that might impact the benchmark?
We shall track decode.c
as well.
- Main benchmark will be “Dhrystone” and “CoreMark” in tests/ for now because I can easily modify the script to generate the JSON format file which required by the benchmark action.
Agree.
from rv32emu.
Support pull requests. Instead of updating GitHub pages, add a comment to the pull request to explain benchmark results.
Quote from github-action-benchmark's future work.
If this CI run is enabled for PRs, the benchmark charts will update with each pull request. Since we must first confirm a performance improvement before updating the benchmark charts, this might not be what we desire. In other words, we need to discuss about how the PR is doing before updating the benchmark chart. Or, is it acceptable to update the benchmark chart with each PR?
And one more thing, may I know if @qwe661234 is working on this issue? If not, I could help to write the CI script if the above concern is OK.
from rv32emu.
My plan is as follows:
-
Run the benchmark workflow and update the GitHub pages whenever "pushing on master branch" and "opening of a pull request and synchronizing the pull request(if any) on master branch" are performed when editing the "emulate.c" and "rv32_template.c" files because these 2 files have the potential to directly impact the benchmark. Under this constraint, we can prohibit excessive benchmark workflow with unrelated files, such as those in docs/ . Do you have any suggestions for other files that might impact the benchmark?
-
The benchmark comparison between the most recent commit and the previous commit will be noted, allowing the maintainer and committer to debate the benchmark comparison. An example of a benchmark comparison can be seen in the benchmark action README: link
-
Main benchmark will be “Dhrystone” and “CoreMark” in tests/ for now because I can easily modify the script to generate the JSON format file which required by the benchmark action.
-
The x-axis and y-axis unit of benchmark graph will be commit SHA and unit of benchmark respectively. Let’s take “Dhrystone” be an example: the y-axis will be average MIPS over 10 runs. Example benchmark graph taken from the benchmark action README: link
from rv32emu.
- The benchmark comparison between the most recent commit and the previous commit will be noted, allowing the maintainer and committer to debate the benchmark comparison. An example of a benchmark comparison can be seen in the benchmark action README: link
We can set the threshold to trigger the generation of benchmark comparison, for example 0.5x worse than before, default is 2x worse than before.
from rv32emu.
Related Issues (20)
- Potential uninitialized value read without initialization HOT 2
- Specialize JALR Instruction HOT 6
- Use the canonical order of RISC-V extension names in the repository description
- jit: Flush the code cache when it is full
- Build using emcc failed HOT 6
- Failed to build using emcc when ENABLE_SDL=1 is set outside macOS HOT 2
- Help with the Emulator HOT 13
- Filter out message-less demo programs from WebAssembly demo page HOT 1
- jit: Fail to run recursive programs
- smolnes.elf cannot be executed and no "inferior exit code 0" output to indicate that a process has completed in the rv32emu-demo
- CI: WebAssembly action failed when PR merged HOT 1
- Trigger WebAssembly CI when embedded ELF executables has been rebuilt
- Calling close() system call with fd < 3
- Crash when the memory on host is insufficient HOT 4
- Keep prebuilt ELF files up to date HOT 5
- CI: Setup self-hosted GitHub Actions runner HOT 6
- Don't check out softfloat if no F extension is configured HOT 4
- A segmentation fault occurs when running a simple ELF file HOT 9
- jit: bug if the registers of the source and linkage registers of `JALR`, `CJALR` instructions are same
- Improve the build system, making it more flexible
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rv32emu.