hevia / earthkb Goto Github PK

TODO:

• Get OpenNRE installed
• See if Spacy provides the exact positions of the named entities in the text so you can pass those along to OpenNRE's .infer method

TODO:

• Create a list of relevant wikipedia articles. Maybe write the list down here?

Scrape this whole page: https://en.wikipedia.org/wiki/Index_of_meteorology_articles
Topics at the bottom of this page: https://en.wikipedia.org/wiki/Climate_change

Topics:

• Atmospheric Physics

Pages (Enter only page name so we can easily scrape for it later):

• Polar Vortex
• Rossby wave
• Sudden stratospheric warming
• Teleconnection
• Polar regions of Earth
• Stratosphere
• Mesosphere
• Mesopause
• Radiative cooling
• Carbon dioxide
• Greenhouse gas
• Jet stream
• Thermopause
• Stratopause
• Maximum parcel level
• Atmospheric convection
• Temperature
• Atmosphere
• Exosphere
• Ionosphere
• Thermosphere
• Turbopause
• Ozone layer
• Chlorofluorocarbon
• Dobson unit
• Ozone depletion and climate change
• Air pollution
• Atmosphere of Earth
• Halocarbon
• Tropopause
• Polar night
• Arctic Circle
• Antarctic Circle
• Atmospheric refraction
• Midnight sun
• Climate change
• Ozone depletion
• Antarctic
• Southern Hemisphere
• Volcano
• Extratropical cyclone
• Tropics
• Arctic oscillation
• Climate variability and change
• Troposphere
• Block (meteorology)
• Orography
• Atmospheric waveguide
• Atmospheric wave
• Polar amplification
• Anticyclonic storm
• Storm
• High-pressure area
• Atmospheric pressure
• Low-pressure area
• Rapid intensification
• Central dense overcast
• Annular tropical cyclone
• Bar (tropical cyclone)
• Superstorm
• Hypercane
• Polar High
• Siberian High
• Azores High
• North American High
• North Pacific High
• North Pacific Current
• California Current
• Pacific Ocean
• Southern Ocean
• Arctic Ocean
• El Niño–Southern Oscillation
• El Niño
• Walker circulation
• Ridiculously Resilient Ridge
• Horse latitudes
• South Atlantic High
• South Pacific High
• Kalahari High
• Hadley cell
• Atmospheric circulation
• Polar low
• Kona storm
• Australian east coast low
• 1996 Lake Huron cyclone
• Mediterranean tropical-like cyclone
• Thermal low
• Atlantic hurricane
• Cape Verde hurricane
• Pacific hurricane
• Typhoon
• North Indian Ocean tropical cyclone
• Mediterranean tropical-like cyclone
• South-West Indian Ocean tropical cyclone
• Australian region tropical cyclone
• South Pacific tropical cyclone
• South Atlantic tropical cyclone
• Atlantic Ocean
• Cold-core low
• Cut-off low
• Upper tropospheric cyclonic vortex
• Catalina eddy
• Haida Eddies
• Wake low
• Mesohigh
• Mesovortex
• Mesocyclone
• Funnel cloud
• Tornado
• Supercell
• Wall cloud
• Multiple-vortex tornado
• Satellite tornado
• Anticyclonic tornado
• Landspout
• Waterspout
• Gustnado
• Dust devil
• Steam devil
• Fire whirl
• Season
• Precipitation
• Atmospheric chemistry
• Atmospheric physics
• Atmospheric river
• Climate
• Cloud
• Cloud physics
• Fog
• Fog season
• Cold wave
• Heat wave
• Meteorology
• Severe weather
• Weather forecasting
• Weather modification
• Drizzle
• Freezing drizzle
• Graupel
• Hail
• Megacryometeor
• Ice pellets
• Diamond dust
• Rain
• Freezing rain
• Cloudburst
• Snow
• Rain and snow mixed
• Snow grains
• Snow roller
• Slush
• Cumulonimbus cloud
• Arcus cloud
• Downburst
• Heat burst
• Derecho
• Thunderstorm
• Volcanic lightning
• Lightning
• Air-mass thunderstorm
• Thundersnow
• Dry thunderstorm
• Cyclone
• Anticyclone
• European windstorm
• Nor'easter
• Subtropical cyclone
• Tropical cyclone
• Storm surge
• Dust storm
• Simoom
• Haboob
• Sirocco
• Gale
• Amihan
• Monsoon
• Firestorm
• Winter storm
• Ice storm
• Blizzard
• Snowsquall
• Ground blizzard
• Winter
• Spring (season)
• Summer
• Autumn
• Atmospheric science
• Atmospheric thermodynamics
• Infrared window
• Atmospheric icing
• Atmospheric electricity
• Atmospheric dispersion modeling
• Atmospheric Radiation Measurement Climate Research Facility
• Atmospheric Model Intercomparison Project
• Coupled Model Intercomparison Project
• Climatology
• World Climate Research Programme
• Planetary boundary layer
• Atmometer
• Atlantic hurricane season
• Atlantic Oceanographic and Meteorological Laboratory
• Apparent wind
• Annular tropical cyclone
• Anemometer
• Anabatic wind
• Airshed
• Air quality index
• Fluid parcel
• Aerography (meteorology)
• Aerobiology
• Aeroplankton
• Aeroacoustics
• Advection
• Ball lightning
• Baroclinity
• Barometer
• Berg wind
• Biometeorology
• Buoyancy

We're already using BERTopic, but we can finetune the implementation further based off of this: https://www.pinecone.io/learn/bertopic/

TODO:

• Read through: https://github.com/dwadden/dygiepp to get the data into the proper format

We currently have a working keyphrase and topic index, get index ranking working for these two approaches to thoroughly recommend relevant files

Related #3

Let's get data versioning out of the way early so it doesnt bite back in future