protontypes / open-sustainable-technology Goto Github PK

Similar to twitter our account.

Milestones:

• Create one
• Continuous integration of PRs
• Move to a foss instance

Hi, I found some broken links:

• https://github.com/ecmwf-projects/copernicus-training (file: education.md, status code: 404)
• https://carbonmapperdata.org/ (file: README.md, error: ECONNRESET)
• https://github.com/fineprint-global/forbio (file: README.md, status code: 404)
• https://forge.ipsl.jussieu.fr/nemo/wiki/Users (file: README.md, status code: 404)
• https://git.outils-is.ird.fr/grelet/TSG-QC (file: README.md, error: ENOTFOUND)
• https://juliaclimate.github.io/GlobalOceanNotebooks/ (file: README.md, status code: 404)
• https://ts-gitlab.iup.uni-heidelberg.de/dorie/dorie (file: README.md, status code: 404)
• https://bitbucket.org/anyelacamargo/aquacropr/ (file: README.md, status code: 404)
• https://github.com/FZJ-IEK2-VSA/HIM (file: docs/blog/gathering_open_sustainable_technology.md, status code: 404)
• https://protontypes.eu/about_free_innovation/ (file: docs/blog/gathering_open_sustainable_technology.md, status code: 404)
• https://https://www.weforum.org/agenda/2021/12/natural-climate-solutions-carbon-markets-climate-justice/ (file: docs/blog/impact_and_potential_of_open_source_on_climate_technology.md, error: ENOTFOUND)
• https://https://chaoss.community/metrics/ (file: docs/blog/impact_and_potential_of_open_source_on_climate_technology.md, error: ENOTFOUND)

@andrew created an fantastic way to get the metadata of all listed projects via his platform: https://ecosyste.ms/
Here a simple visualization of the whole dataset:
https://summary.ecosyste.ms/collections/1

For the next report we need a simple python tool to extract the data automatically into a table (CSV). Additionally, the tool allows us to identify which metadata is missing. This includes, among other things, organizational data. Afterwards we have to decide whether it is in the interest of ecosyste.ms to provide this data via the platform or whether we should write our own tool that obtains the data via the GitHub API.

A simple way to discover new projects of the last year is by automatically scanning the relevant namespaces of this ecosystem for projects that have been active in the last year. This list can than be compared with the projects already on OpenSustain.tech.

Here the last update ( 4 months old ) of the organisations:
https://github.com/protontypes/AwesomeCure/blob/main/csv/github_organizations.csv

I will try to do an update of the list at the weekend.

It looks like there is some overlap with this project: https://github.com/oss-in-energy/oss-in-energy and it has some entries that I do not see in this list, e.g. https://github.com/alliander-opensource/openstf

https://www.eenewseurope.com/en/study-shows-388-open-source-energy-projects/

Please consider the @meironz work on the multiscale Ulva model - it is a central part of the sustinability concept of off-shore macroalgae (Ulva, etc.) growth and biorefinery, as well as sutiable for the waste water and afluent absorptions.

https://github.com/alexliberzonlab/Multiscale_Ulva

This bot would allow us to create a LinkedIn bot like we already using for Twitter:
https://github.com/tomquirk/linkedin-api

Plant trees for contributions:

• New contributor: 5 trees
• Pull request: 1 tree
• Pull request merged: 10 trees
  or something.

For the next report we need a list of people we want to interviews. Criteria for selection should be:

1. High impact in the ecosystem. Long time experience.
2. Developers or users of tools and associated data sets.
3. Variety of organisation perspectives: Academia, For-Profit, Non-Profit Background, Startup, Community,...
4. Variety of topics: Biosphere, Water Suppy, Battery, ....
5. Variety of personal backgrounds: Origin, age, opinions,...

One way to find such interview partners is the list created by @andrew. You will find the contributors commited to the most projects here on the right side under committers:
https://summary.ecosyste.ms/collections/1

We are in the unique position of being able to quantify the application of machine learning, deep learning and artificial intelligence within sustainability. One way to do this is based on the dependencies of the open source software projects and their dependencies. Another way is NLP or simple keyword matching based on the project READMEs.

The results of this analysis could be of interest to a very large group of people.

It would be interesting to see which of this Software is in the conda-forgerepository so we could get it easily installed by the mamba installer.

Are there plans to map this information?

For the next report we need a announcement blog post. This helps us to find more contributors to the database but also to the report. In this way, we can also agree on a common alignment for the report in advance. The announcement blog post on the first report can be found here:
https://opensustain.tech/blog/impact_and_potential_of_open_source_on_climate_technology/

We need an "About" Section:

• Legal Disclosure
• Some more information on the background.
• Some information about the history of this list
• ...

We have already started initial experiments with the Natural Language Processing (NLP) tool BERTopic. First attempts can be found here: https://colab.research.google.com/drive/1Y5exVWXYFvbYp0yzbJcpya6FL8HejYRu?authuser=0&pli=1#scrollTo=BG8S-Qz_Lmy8
Please contact me if you want permission to edit the notebook.

Such a development could have a variety of benefits for our project:

1. Better and consistent labels over all projects. —> Much better plots to show the distribution of projects based on topics.

2. Graph showing connections between topics —> Showing how the ecosystem is connected based on topics and which projects provide important notes.

3. Creation of a dictionary with the relevant words / topics within the open source sustainability ecosystem —> Very important for further investigations and for gap analytics. What topics are missing so far in our analytics.

4. Automatic discovery of new projects based on the dictionary

5. Investigate the role of AI within the ecosystem based on AI related topics within the READMEs. Related to #155

This task is certainly not easy, but the developer of BERTopic has offered us his support. Please contact me if you want me to make a contact here.

Another open source project which could be potentially interesting here is: PandasAI

The OpenSustain.tech datasets represent the largest catalog of knowledge at the intersection of environmental sustainability and OpenSource. Developments such as ClimateBERT have shown that AI can be trained on the IPCC report to answer questions about these knowledge sets:
https://www.chatclimate.ai/

Imagine an AI that could answer the following questions:

1. what open source tools and data can I use for use case X in environmental sustainability?
2. I would like to contribute to an open source project in the area of environmental sustainability. What are the best projects to contribute to based on my skills and interests?

What are the risk, challenges, values, requirements and costs that come with such an development?

In the last report, we relied heavily on the Development Distribution Score (DDS) when considering the health of a project. Andrew has already made an extension here calculating the DDS only for the last years. It would be interesting to analyze the DDS in more detail and compare correlations of this value with other variables. First approaches to this can already be found in the last report. It would be interesting to compare this value with other largest values like the number of issues closed in the last year and the download numbers in the last year. This would be essential in assessing whether a project needs support.

Perhaps we can also identify other indicators that could be useful in evaluating the health of the project.

The website of the latest report divides the different plots into different pages. This makes the report difficult to read and does not give a good overview of the ecosystem. For the next report a dashboard would be optimal that gives a good overview over all the plots. To create this dashboard, several points need to be analyzed:

1. How can we combine the existing plots on a single page?
2. What are the best tools for that and how can we integrate the dashboard into the existing website? So far we used Jupyter Book with Plotly. Quarto could be another option for the dashboard.
3. How can we improve the existing plots?
4. What other plots can be created from the existing data?
5. What other data would be interesting to plot?

The preprocessing for the plots and the creation of the plots is done in this repository:
https://github.com/protontypes/open-source-in-environmental-sustainability

Please be aware that many important preproccesing steps are done in the Overview notebook: https://github.com/protontypes/open-source-in-environmental-sustainability/blob/main/open-source-in-environmental-sustainability/chapters/overview.ipynb

This notebook creates another CSV that is used by all the other notebook. This software architecture is really not elegant but it was enough for the last report.

For the further development on the next report we need a workflow with Jupyter Notebook. The platform should allow to work collaboratively on the data analysis, machine learning and plotting. The workflow should:

1. Allow anyone to open a Jupyter notebook in the cloud without deep Python or cloud skills
2. Be version controlled via GitHub.
3. Remove output from Jupyter Notebook to reduce repository size and enable easy merging of notebook
4. Easy to adapt for multiple data science notebooks
5. Support the tools we are using at the moment: Plotly, BERTopic, pandas
6. GPU for using NLP
7. Support for Jupyter Widgets if possible.

One way to test the workflow is the notebook we used so far for the experimenting with NLP:
https://colab.research.google.com/drive/1Y5exVWXYFvbYp0yzbJcpya6FL8HejYRu?authuser=0&pli=1#scrollTo=BG8S-Qz_Lmy8

After reading this we might consider renaming the carbon footprint section:
https://en.wikipedia.org/wiki/Carbon_footprint

The term was popularised by a $250 million ad campaign by the oil and gas company BP in an attempt to move public attention away from restricting the activities of fossil fuel companies and onto individual responsibility for solving climate change

For the preparation of the report it is of great importance to maintain a common library of scientific publications, blog posts, citations and books on our topic. From this we can draw citations but also derive possible interview partners. A first step would be to collect the links and publications from the first report.

The high value of the projects lies in their completeness, which shows the vast majority of the open source ecosystem in this area. Before this project the open source community had problems to find good and active projects in the field. To achieve this quality, I put more than 1500 hours into curating and researching for the project.

On the one hand, this leads to a massive distortion of my view, and on the other hand, I will no longer be able to devote time maintaining to this extent in the future. Another problem is the "fun" of the research. In the beginning, Tjark and I were able to identify many projects very quickly. Now it often takes hours to identify new projects and I am mainly busy working my way through projects that have already been listed. This is no longer really motivating for external and internal contributors.

To ensure the quality and updates of the website in the long run we need incentives to the community to involve them in the maintenance of the project. What we have tried so far:

1. Sending contributors money automatically via LibreSelery. You will get a lot of pull requests in this way but most of the were garbage and created a lot of management overheat. ( I invested about 300€ in this way)
2. Planting Trees with Continuous Reforestation for every merged pull request. That led to a few more pull requests, but most people didn't really feel motivated by it. I think most of the pull requests would have been created even without the tree planting. Tree planting is also very controversial. Many studies have shown that a lot of the very cheap tree planting does not lead to sustainable ecosystems and forests. ( Invested about 700€ into this)
3. Consulting, marketing and blog post for new projects. This worked much better, but was very time consuming. With a team of people / community this could work much better.

If you look at the pull requests, you'll notice that most people add their own projects to the list. Preferably also always on top of the list ;). I think this is a good place to start.

I had some interesting discussions on this topic with Eirini and Josh in the past. What could be very helpful for many new projects that are listed on OpenSustain.tech is a kind of "Open Source Starter Kit". We could provide a combination of multiple support packages: Open source projects could determine for themselves what kind of support they want in which combination:

1. Consulting on the maintenance and how to sustain the project.
2. Marketing support ( Podcast, Logo, Blog Post, ....)
3. Testing and feedback on Code Quality / Documentation and other technical aspects.
4. A small injection of grants (for example 100€) to cover costs such as website domain, travel or cloud time at the start of the project. Especially the latter is hard to raise for small open source communities what make the development of open source AI application much more difficult.
5. Credit for courses / training of the Linux Foundation
6. Networking with possible partners

What do you think about this proposal. What do you think are possible incentives?

Hi,

I think it would be a good idea to periodically check all the links in this repository and fix the ones that are redirecting.

For example, I noticed that the link to PowerGenome (https://github.com/gschivley/PowerGenome) redirects to https://github.com/PowerGenome/PowerGenome. If, in the future, a new repository or fork is created that points to the old URL, the link will no longer be correct.

To prevent this from happening, you could set up a scheduled GitHub Action with a link validator (such as awesome_bot) to check links every month or so. In addition to redirecting links, this will allow you to identify links that no longer exist, or name changes to projects.

Thanks and I apologise if you have already discussed this. I didn't find anything related in the issues so I thought I'll post my ideas here!

Let us create a high-quality hoodie for OpenSustain.tech Contributors.

Milestones:

• Motivate People that have already contributed by sending them a hoodie
• Give new contributors a hoodie
• Give contributors of the listed projects a hoodie, like Hacktoberfest

Task:

• Research for Hoodies and Manufactures:
  • https://www.neutral.com/
  • Ask FSFE for possible ways to do the sending of hoodies. (Tobi)
• Ask Designer for OpenSustain.tech Hoodie
  • Miriam
  • https://tabletopwhale.com/
  • https://discourse.opensourcedesign.net/
  • ....
• Ask OST contributors for feedback on different designs

With the website in mind and and goals extending beyond the list itself, contribution becomes a broader term.
Contribution (within our cause) can be basically anything that is aiming at sustainability and open source.

I would suggest extending the contribution guide and making a new category, sth. like "Open Sustainable Business".
To have an area to gather events like hackathons and similar events. To make contributing in any sense more accessible.

To have a badge for projects to add to their websites/READMEs to show that they are on the list.

Contribute/Propose by adding/PRing to the logos Repo.
Here are the ideas so far.

It will be important to work with one or more copy editors who have a keen eye for detail and are familiar with sustainability topics. This effort may come from the greater community, but more likely will require funding for external resourcing.

Hey @tjarkdoering @elaristi,

here a simple suggestion how we can create a solid spreadsheet or database out of OpenSustain.tech
https://excalidraw.com/#room=95d13deb482ee2e5039c,kzSqbuh5eHSuJ-29QeuFMw
What is your opinion?

We have several conference presentations, press articles and references for this project. It would be important to show our reach and the public impact in one place.

A key aspect of interest to our readers and users are new projects and organizations identified through our work. To give us an overview of the new projects, a CSV file with the new organization and projects is essential. Hundreds of new projects have been added to the list since the last report. This list could also be essential for identifying possible interview partners.

However, this would first be a first screening. I hope that by publishing the report as a PDF and by publicly announcing the next study, we will be able to identify significantly more projects.

Many section titles are not completely self-explained. In order to improve the ordering of the list every section should be discriped with at least one sentence.
Here an example:

Life-cycle assessment

a methodology for assessing environmental impacts associated with all the stages of the life-cycle of a commercial product, process, or service

The current structure of the fields / topics has resulted in some subject areas becoming very large. This makes the list difficult to navigate. In addition, it would be interesting to better divide the subject area in the analysis to make focal points and gaps more recognizable. This structure has an significant impact on the data analysis we are doing. My good old collage @kikass13 developed the parser we are using at the moment for the list. If we would add another level to the list like field / topic / subtopic we have to change this script. A new structure also has significant on the website framework. Therefore, it would be good to find a structure that allows us to keep with a 2 level structure.
Here are some fields that urgently need splitting and a suggestion how to split them:

• Biosphere

  • Biodiversity
  • Forest and Vegetation
  • Wildfire
  • Biomass
  • Habitats Conservation and restoration
  • Marine life and Fishery

• Hydrosphere

  • Freshwater and Groundwater
  • Oceans
  • Coastal and Reefs

• Atmosphere

  • Atmospheric Dynamics
  • Atmospheric Chemistry
  • Clouds and Water Vapor

• Cryosphere

  • Glacier
  • Arctic and Antarctic
  • Snow and Permafrost

A grant would be helpful for further reports and further work on the website. This could also help to finance travel expenses and workshops.

@rockita has proposed to apply for one grant with the next deadline at the end of the month:
https://investinopen.org/

Based on the experience we had with the last report the method for qualitative interviews and analysis can be extended. In order to conduct a large number of interviews, it is necessary to find a standardized workflow so that the interviews remain comparable with each other. The questionnaire could consist of a standardized questions and a free part tailored to the project.

The workflow includes:

1. Creation of a email template to contact people. In many cases, it is easier to make initial contact via LinkedIn. Initial contact via email should be through a known and trusted email domain (Linux Foundation would be ideal).
2. Interview partner should be offered a common blog post (CC 4.0) that we create based on the interviews. This blog post will be released on the OpenSustain.tech website.
3. Generation of a framework / method for the evaluation of the interviews.
4. Creation of methods to compare qualiative and quantiative questions.

Have an educational resource to help people to start contributing and give them an overview over best practices.

Do you also accept Creative Commons JSON Data?

#73
@danielhuppmann

Beyond the scope of this PR, but there seems to be a bit of confusion between the "Modeling" section and the "Analysis" section - many of the entries in "Analysis" are really modelling frameworks (e.g., PyPSA, oemof, matpower, POMATO), whereas pyam and the Open Energy Platform (and possibly others in that section) are tools for analysis-and-visualization packages/tools (but not for creating the scenario results). A bit of a review and structuring/tagging might be helpful...

Tools like https://github.com/osdg-ai/osdg-tool would allow us to label every project based on a SDG. As input the README, documentation or website could be used.
@SimonasTr @lukas-pkl: Are you interested in joining this effort?

The current design of the website makes it difficult to search and find projects according to criteria such as "programming languages" or specific topics. Therefore, the project always receives poor feedback on its usability. Therefore, we are looking for a concept to better integrate the previously created database into the existing website or to transfer it into a new website.

The requirements are:

• Easy maintainability
• Low energy consumption of the website
• Hosting on GitHub pages if possible
• As far as the functionality is not restricted, based on open source tools.

Database:
https://airtable.com/shr9we419r2TkpLkc

The aim of the website is to give beginners as well as experts in the field of sustainable technologies a possibility to discover software tools, projects, datasets, communities, tutorials and other resources easily related to their own problems and questions.

First, a proof-of-concept is needed that shows how the website in its current state can be transferred to the new state.

OpenSustain.tech needs more partners to grow and sustain the project in the long term. LF Energy can also help build a neutral govermance structure. Additionally, there are a lot of synergies with LF Energy and a lot of interesting people at LF who want to help to develop OpenSustain.tech further.

The spatial resolution / global coverage georeferencing of the projects plays an important role in searching the database. Automated labeling of the datasets by NLP is from my point of view not target-oriented. To manage such labels in the long term a real database is necessary instead of the CSV file.

A tool that compiles a list of new projects in this way of the last X years can significantly help the discovery of new projects and the usage of projects.

Most of the open source project in this ecosystem are developed and used within science. Manually searching for projects with the keyword "git" in combination with the keyword dictionary of the ecosystem is a good way to identify new projects. At the same time it is another way to identify the usage of open source projects.

A short search showed that there are a lot of open source tools that might be suitable for this work:

• https://github.com/scholarly-python-package/scholarly
• https://github.com/nealhaddaway/GSscraper
• https://github.com/matthiasprobst/zenodo_search
• https://github.com/dennissergeev/atmosscibot

Website: https://prototypefund.de/en/
Deadline: The application phase runs until 30 September 2023.

Application: The NLP for Open Source Discovery and Analytics tooling could be an candidate to for this fund. #145

This fund looks like it was made for us:
https://fordfoundation.forms.fm/2023-digital-infrastructure-insights-fund-rfp/forms/9724

"We aim to back the development of pertinent work that examines the convergence of open-source software and digital infrastructure with social movements focused on democracy, rights, justice, the environment and scientific research."

Deadline:
Oct 1 at 12:00am
In your local timezone (GMT +2)

Grant Amount: We seek to support proposals addressing a range of issues and different scopes that are in a ballpark between $50k – $125k for 6 - 12 months projects, managed via Open Collective (exceptions possible for 18 month-proposals)

The current front-end and back-end of the website needs evolution. This includes:

1. Comprehensive search function that includes the metadata collected from the projects.
2. News feed about new listed projects.
3. A way to visualize the projects and organisations as a landscape ( see https://landscape.lfenergy.org/ ). How can we integrate the LF Energy Landscape view?
4. Dashboard of the report
5. Simple way to contribute new projects
6. Traffic Analytics ?

Since the current website with the list on the landing page gives us very good Search engine optimization (SEO) we should be very careful how to change the website. Many developers prefer this list style ( me including )

Changes to the website have a major impact on the entire project. Careful planning with good advice is urgently needed here.

To not double use the protontypes brand.

• Integrate the inital blog post: https://protontypes.eu/blog/2020/10/05/gathering-open-sustainable-technology/
• Integrate the https://hackmd.io/wDUdnuCnRIKZhL9_v1yR6A?edit

Papers related to open source projects are an important part of the documentation and are easy to use with NPL.

Hey, I just checked https://opensustain.tech/contributing/ and found that bullet points in the Contribute to the list section are not being correctly rendered. When I wanted to check what the reason might be I also found that the document under docs/contribution.md has slightly different content, so something seems to be amiss. Unfortunately I couldn't find out much more, but hopefully it helps to identify the issue.