Comments (14)
I took a look into it.
I have few suggestions (more of questions):
- Why do we need to clone all repo? We do get almost all information from search repo.
- Is this project meant to be a single place where user could find any project that has FastAPI in it? Or we can add a logic, say it has to have minimum of say 3 starred?
- Can we use a SQLlite local db to check for duplicates efficiently instead of having bunch of text files?
- Can we add more info to the table? Like number of stars, forks?
Also a very extravagant wish, can we aim of github pages interactive UI with search and other cool stuff?
from awesome-fastapi-projects.
-
To be more precise. We don't need to clone the git history tho. I mean, can you fetch the information of packages used without cloning?
-
We could add a custom logic.
-
Yes.
-
Yes.
And yes. The UI would be cool.
from awesome-fastapi-projects.
Here is the flow I have come up with
Few concerns:
- Do we need to show dependencies? If we skip this, we can avoid cloning :)
- Is deta.sh a good place to run this? Or do you recommend an alternative?
I also would plan for an API interface to query the data from our DBs. Later, we can create a UI for the same!
from awesome-fastapi-projects.
I believe that what brings value to this project is knowing which dependencies they use.
We can run this on GitHub if we create a static page, and use SQLite. Maybe something like https://gohugo.io/? I'm not too familiar with it...
from awesome-fastapi-projects.
Okay, thinking of value it adds, it does make sense.
I think we should complete the core before we start thinking about UI.
If we are going all local, then SQLlite is better 😄
Also what do you think of having two lists, curated (not sure about naming) and general? The curated one looks for READMEs and general looks for code.
from awesome-fastapi-projects.
The number of GitHub queries that can be done is quite limited considering the amount of results. And you'll soon notice that it's important to shrink the numbers.
Like, on a single job, we are not going to be able to query all projects. We query part of them, and store information and with that we query in a paginated way the next time.
The query can probably be refined to something like "from fastapi import FastAPI".
from awesome-fastapi-projects.
Yes, adding a filter stars>1 reduced results from some 300K to 273. 😅
I'll try to get into coding today on this and see what limitations we hit.
from awesome-fastapi-projects.
Ah... If that's the case... Do as many queries as you want hahaha
from awesome-fastapi-projects.
Hi @Kludex, do you still have plans to add maintainers to this repo? I can help with that.
from awesome-fastapi-projects.
I do. The idea is the one in #15 (comment), if anyone is willing to follow that I can give rights here. 🤷
from awesome-fastapi-projects.
I can follow that @Kludex
from awesome-fastapi-projects.
Hi @Kludex
Having looked at some of the discussions and PRs, I would like to contribute as well 🙂
Let me share how I can imagine a solution to the problem and I would appreciate it if you could give me some feedback regarding my suggestions and whether I got the idea of the project right 😄
First, as discussed above, it would be great to implement it as a web app with an SQLite DB (for simplicity and cost efficiency). FastApi with SQLAlchemy could be used. Alternatively, Streamlit could be a good idea because it could ease creating visualizations, though limiting the general UI capabilities. With FastAPI, HTMX can be used on the presentation layer. When it comes to visualizations, packages like Plotly, Bokeh, or ipyvizzu offer support for rendering graphs to HTML so they can get embedded in the templates.
The question is - what kind of visualization do you envision on the page?
Things get trickier when it comes to actually creating the index of the projects. It turns out it is not so obvious to get the dependencies employed by the Python projects. There are a lot of ways one could provide dependencies - either via a requirements file, pyproject.toml, setup.cfg, and so on ... (https://discuss.python.org/t/list-dependencies-of-a-package/12341) The tools exist to parse various formats of dependencies (https://github.com/nexB/dparse2/tree/main) but they are quite limited and do not support most of the formats. Do you have any ideas on how dependencies could be discovered efficiently? Alternatively, we could use a tool like https://pypi.org/project/third-party-imports/ to find the third-party imports - less efficient but might work well
The GitHub Actions could be used to run the periodic scraping of the repositories, as it has been suggested earlier.
I believe the GitHub Search should really be enough. Take a look at the following query: https://github.com/search?q=stars%3A%3E3+path%3A*.py+fastapi+language%3APython&type=&ref=advsearch&l=Python. Does https://sourcegraph.com/search allow to show the dependencies of the projects? Do you think Github Search would end up throttling the app? Which search engine would you prefer?
Please, let me know what you think and I would be happy to create a PR 🙂
from awesome-fastapi-projects.
The question is - what kind of visualization do you envision on the page?
Simple table with filtering on packages used.
Do you have any ideas on how dependencies could be discovered efficiently?
No. On the first implementation, I just queried all that contained "import fastapi" or "from fastapi import".
Does sourcegraph.com/search allow to show the dependencies of the projects?
I don't know.
Do you think Github Search would end up throttling the app?
The limit is short, but I guess not...
Which search engine would you prefer?
If we can avoid GitHub, it would be great, since I don't want to use my personal token for it.
from awesome-fastapi-projects.
Hey @Kludex , I want to contribute to this project.
please add me to the project.
from awesome-fastapi-projects.
Related Issues (17)
- Links Updation HOT 1
- Link to 2020_slowdelivery_take is not working HOT 2
- Serious need for an update HOT 4
- error while running your project HOT 1
- Add CONTRIBUTING.md HOT 1
- Maintenance
- Make the repositories sorted by stars (descending) by default
- Improve pagination
- Improve the mobile UI
- Bug with a hard reload on navigation HOT 2
- Statistics
- Dark mode
- Standardize changelog and releasing
- Improve dependencies parsing performance
- Create curator bot
- Deadlink: the project ExpenseAPI HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from awesome-fastapi-projects.