Foundation Models are a type of generative AI that pre-trains large neural networks on expansive amounts of data. These are powerful AI tools for generating creative output, finding connections, and discovering patterns across the body of human knowledge. This type of generative AI has the ability to suggest potential solutions to the prompts it is given. Can these models also be leveraged to help us solve great challenges such as climate change? Should they?
A special concern for sustainability work is that while generative AI and other machine learning models are becoming go-to tools for everyone from data scientists to artists, these deep and complicated calculations are also consuming energy at an increasing rate1. Training a model like GPT-3 is estimated to be the CO2 equivalent2 of operating several households for a year. Models are getting bigger too, with growth in model size and computational complexity outpacing Moore’s law. Although these numbers sound alarming and bear watching3, other studies point out that total machine learning energy consumption is still a tiny fraction of a percent of global energy consumption and that mitigating factors such as efficiency improvements in computer chips, algorithms, and cooling infrastructure appear to be having a stabilizing influence on ML energy consumption. Carbon emissions from a small suburb, retail shipping, or even attendance at Machine Learning conferences still exceed that generated by machine learning models.
While taking note of the associated emissions, Foundation Models have demonstrated power to persuade. If the impacts can be moderated, shouldn’t we try to enlist it to help us innovate as well? We are on the hunt for applications that will embrace the promise of these models by harnessing their power to drive creativity and identify new solutions.
Foundational models sometimes recommend things humans might not think of. Unbound by the restrictions of common sense, readily accessible models today build on semantic relationships derived from large bodies of text, or relationships between large corpuses of text and images. The next generation of models already being built will include video and 3D modeling. Soon, we may hope to design houses by starting with phrases like “single-story mid-century modern, south facing on a forested hillside, three bedrooms”. Could these models be trained to recognize more than building archetypes, but also qualities such as “net-zero” or “recycled materials”? If models can be fine tuned to these features, what will they recommend when we add “Net-Zero” and “sustainable building” to our prompts?
The potential for future AI models is incredibly motivating. New proposals for foundation models include those that support earth monitoring4 and fusing physics-based and deep learning5 models. Could generative AI be used to help us design new kinds of materials? The people at MaterialsAtlas.org think so6. The entire field of generative design7, leveraging generative AI to speed and enhance the engineering process, is just beginning to take shape.
What needs to be built so that these models can recommend alternative building materials? Will they suggest mushrooms as insulation8? Could these tools predict flood events, rain forest regrowth patterns, or help design strategies for coral deacidification9? Today’s models are limited by the data they are trained on, and more importantly lack integrations and constraints necessary to ensure correctness and accuracy. Can foundation models be connected to external models for existing materials, and scientific, biological and engineering databases? What additional guidance is necessary to have foundation models suggest viable sustainable solutions based on the physical and regenerative properties of a material? Humans have a lot of heavy lifting to do to make use of today’s models and still more to create the models of the future, but the opportunities are seemingly infinite.
Although the current generation of foundational models are limited by a lack of physical insight and difficulties with factual accuracy, we nevertheless see exciting potential for these models to stimulate creative thought and to help speed up the search for new solutions. These models are excellent at taking open-ended inputs and connecting them to interesting outputs. We tried posing some questions to foundational models as a way of exploring new opportunities.
Could there be a service that helps people to recycle?
Q: I have 500 pounds of used corks. How can I turn these into environmentally friendly building materials?
A: You can turn your used corks into environmentally friendly building materials by grinding them down into a powder and then mixing the powder with a binding agent such as cement or gypsum to create a cork-based insulation or flooring material. This material is both lightweight and energy efficient. Additionally, the cork powder can be used as an additive in cement or plaster to help improve insulation properties.
Could there be a service that helps people to envision new designs?
Q: environmental apartment complex, solar panels, carbon absorbing walls, green engineering
Can these tools help people find and create new jobs?
Q: What are some options for environmentally friendly jobs?
Environmental jobs include renewable energy engineers, urban hydroponic farmers, green builders, sustainable urban planners, and carbon emission reduction consultants. Additionally, there are many jobs in the field of regenerative agriculture and regenerative design which focus on restoring and regenerating ecosystems and promoting the efficient use of resources.
Come build with us...
Help us dream up new applications for these powerful tools. Apply for Launchable (applications due by Jan 9). We are looking for applicants with varying experiences and backgrounds (agriculture, material engineering, built environment, climate adaptation, city planning, etc.). The possibilities for using these models to discover and build new solutions to our climate challenges seem limited only by our imagination.
1 “Numenta: AI is harming our planet”, https://www.numenta.com/blog/2022/05/24/ai-is-harming-our-planet/
2 “Carbon Emissions and Large Neural Network Training”, https://arxiv.org/pdf/2104.10350.pdf
3 “Here's how AI can help fight climate change”, https://www.weforum.org/agenda/2021/08/how-ai-can-fight-climate-change/
4 “Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark”, https://arxiv.org/pdf/2112.00570.pdf
5 “Fusing physics-based and deep learning models for prognostics” https://www.sciencedirect.com/science/article/pii/S0951832021004725
6 “MaterialsAtlas.org: a materials informatics web app platform for materials discovery and survey of state-of-the-art”, https://www.nature.com/articles/s41524-022-00750-6
7 “What Is Generative Design? A Full Guide”, https://www.vectornator.io/blog/generative-design/
8 “Mycelium”, https://www.biohm.co.uk/mycelium
9 “Dell’s deep learning model to support coral reef conservation”, https://www.computerweekly.com/news/252524201/Dells-deep-learning-model-to-support-coral-reef-conservation
We are with our founders from day one, for the long run.