2.1.1.  Text-Based Knowledge

The static nature of GenAI’s text-based training data constrains its ability to interpret geospatial context dynamically. For instance, while GenAI might “know” that Athens is the capital of Greece, this knowledge is abstract, disconnected from physical coordinates, and often devoid of real-time or topographical details. The lack of temporal and spatial contextuality means that while GenAI can generate plausible text about a location, it cannot independently verify spatial relationships, boundaries, or proximity without additional data.

2.1.2.  Ambiguity in Place Names

GenAI models also struggle with the inherent ambiguity in place names. Many locations share identical names (e.g., Springfield in the United States), and disambiguating them requires more than linguistic understanding—it demands access to hierarchical geospatial data or contextual user input. Without this, responses may be inconsistent or outright incorrect, particularly in areas with lesser-documented geographic features.

2.1.3.  Fragmented Geospatial Data

Another challenge lies in the fragmented nature of geospatial data sources. While authoritative platforms such as Copernicus or OpenStreetMap exist, their integration with GenAI systems is not seamless, often requiring manual effort to reconcile differing data standards and formats. This fragmentation exacerbates inconsistencies in GenAI’s understanding of spatial relationships, reducing its utility for complex geospatial tasks.

2.1.4.  Knowledge Gaps in Undocumented Areas

GenAI models exhibit clear limitations in understanding remote or less-documented regions. For example, a well-documented city like London may yield detailed and accurate responses, while a lesser-known location such as Eresos, Greece, may only elicit generic or partial information. This disparity can be attributed to uneven representation in training datasets, further complicating the equitable application of GenAI across diverse geographies.

2.1.5.  Temporal Changes and Updates

Geospatial knowledge is not static; changes in urban development, climate impacts, or administrative boundaries can render training data obsolete. GenAI models cannot account for such temporal variations unless they are continually retrained with updated datasets or connected to live geospatial information systems. This presents a challenge for applications requiring real-time situational awareness, such as disaster response.

2.2.1.  Causes of Hallucinations

2.2.2.  Implications of Hallucinations

Hallucinated geospatial information can have serious consequences, particularly in critical areas like disaster management, urban planning, and climate resilience. Misleading data can result in ineffective or even harmful strategies, undermining the effectiveness of decision-making. Persistent inaccuracies further erode trust in AI tools, especially in high-stakes applications where precision is essential. Moreover, such errors can propagate across systems if flawed GenAI outputs are used to train other models or update databases, compounding the problem and spreading inaccuracies further.

2.2.3.  Understanding the Dynamics of Climate Systems

Climate systems are complex and challenging for Generative AI (GenAI) to understand. They involve feedback loops, non-linear changes, and variations across different locations and time periods. These factors require specialized data and modeling to make accurate predictions, which general AI systems often cannot handle without significant adjustments.

Translating general AI capabilities to work effectively in climate resilience is not straightforward and comes with several challenges. AI models need significant customization to handle complex climate datasets like satellite imagery and weather models, often requiring specialized preprocessing. Collaboration with climate experts is essential to incorporate critical insights about unique variables, such as atmospheric or oceanic dynamics. Additionally, integrating AI with established climate models, like General Circulation Models (GCMs), is necessary but challenging due to their specialized assumptions. Lastly, the vast scale and complexity of climate data demand high computational resources, making effective adaptation both technically and resource-intensive.

7.1.1.  Use Cases and Functionalities

The D100 virtual assistant enhances the accessibility and usability of various geospatial and climate-related data sources. It serves as an interactive tool that enables users to efficiently search, retrieve, and explore relevant datasets. The assistant’s core functionalities include:

7.1.2.  Data Sources and FAIR Evaluation

The virtual assistant is developed using multiple data sources and evaluated based on the FAIR principles—ensuring that data is Findable, Accessible, Interoperable, and Reproducible.

7.1.3.  OGC Compliance and Interoperability

The assistant builds upon previous efforts in OGC initiatives, including:

These efforts explored integrating machine learning (ML) inferencing as OGC API Processes, particularly for landslide detection. The workflows were developed using the OGC Training Data Markup Language (TrainingDML) and OGC API Processes (Parts 1 and 2). The virtual assistant acts as a user-interaction layer for these implementations, providing an end-to-end Generative AI stack for scientific applications while ensuring seamless interoperability.

7.1.4.  Findings and Recommendations

The D100 virtual assistant improves data search and provides an interactive way to explore the datasets mentioned above.

The figure below illustrates the high-level workflow of the assistant:

 

Figure 2 — D100 (GeoLabs) - High Level Workflow

Users interact with the assistant through a chatbot interface, where queries are processed using one of the following approaches:

Approach 1: Retrieval-Augmented Generation (RAG) using a Local Database Fetch data from various sources Index the data in a Chroma vector database Match stored documents with the user query Retrieve the top 5 matching documents ** Generate a response using an LLM based on the retrieved data

Approach 2: Web Search-Based Responses Conduct a web search for the user’s query Fetch relevant data and index it in a temporary vector database Match the query with indexed documents Retrieve the top 5 documents ** Generate a response using an LLM based on the retrieved data

Users can select from the following LLM models for response generation:

Key Learnings:

7.1.5.  Future Directions

To further enhance the assistant, we plan to:

7.2.1.  Use Cases and Functionalities

The generative AI chatbot use case on GEO.ca is designed to enhance the search and discovery of FAIR and Open geospatial data through a conversational interface. It supports various user scenarios, including researchers seeking climate change impact data for analysis, policymakers needing land use data to inform urban planning, educators accessing open data for classroom demonstrations, and citizens viewing historical temperature trends to understand and advocate for climate action.

The chatbot’s primary functionalities include natural language query processing, contextual, geospatial data-driven search, personalized responses based on user needs, and integration with existing GEO.ca APIs for seamless access to data.

Additionally, outside of the provided use case, we explored data visualization capabilities in a subsequent ongoing proof-of-concept development. This involved investigating how the LLM could be augmented with tools to support visualizing data resources (e.g., GeoPackage) associated with the generated results.

7.2.2.  Data Sources and FAIR Evaluation

The AI chatbot leverages data repositories from GEO.ca, adhering to FAIR principles. The data is made findable through clear metadata and search functionalities that enhance discoverability. Accessibility is ensured by providing open access to data via CCMEO’s APIs and services. The data is interoperable, formatted to integrate with various GIS tools following OGC standards. Additionally, datasets are reusable as they comply with licensing and metadata standards such as ISO-19115.

The chatbot intends to further promote these principles by offering guided assistance on accessing and interpreting data. When search results are returned within the chatbot, more information about each dataset is displayed, including links to associated data and metadata resources. Contextual record searching ensures that when users request more details about a specific result, the LLM focuses solely on the metadata for that record. This focused approach helps to streamline user interactions by presenting only relevant information for a specific record, reducing confusion caused by unrelated search results and reinforcing the authoritativeness of the information and data returned.

7.2.3.  OGC Compliance and Interoperability

The planned integration of the chatbot aims to align with OGC (Open Geospatial Consortium) standards to support key objectives. First, interoperability is prioritized by ensuring that the chatbot interfaces with GEO.ca’s systems using OGC-compliant APIs, thereby adhering to the Canadian Geospatial Data Infrastructure (CGDI) and Canada’s Standard on Geospatial. Additionally, the chatbot is designed to leverage OGC standards, including OGC API — Features and OGC API — Records, to handle geospatial queries effectively. As part of the development, we have also begun experimenting with the GeoPackage standard to improve data handling and visualization capabilities. Although the solution remains in the prototyping stage, ongoing efforts are focused on refining these functionalities. Finally, accessibility and inclusivity are central considerations, with efforts to train the chatbot’s NLP on multilingual and region-specific geospatial terms, respecting Canada’s official languages, commitments to reconciliation with Indigenous Peoples, and the goals outlined in the Accessibility Action Plan.

7.2.4.  Findings and Recommendations

Findings: The chatbot was not fully implemented, so there are no direct findings from its deployment on GEO.ca. However, experimentation with a generative AI chatbot currently under development revealed several key challenges. These include hallucinations, where responses contain information not based on authoritative sources, and low confidence in results, particularly when data is not retrieved from trusted APIs like those provided by CCMEO. Additionally, infrastructure costs for training, fine-tuning, and operating such models pose a significant barrier, particularly during initial deployment and whenever new data is added to the catalogue or data repositories.

Recommendations: To address these challenges, responses generated by the chatbot prototype should clearly indicate when information is not sourced from authoritative APIs, particularly those not provided by CCMEO. Ongoing research and experimentation will focus on mitigation of hallucinations and improvements to model reliability. Optimizing the infrastructure for scalability and cost-efficiency is necessary to ensure the sustainability of the system as usage grows, especially with periodic updates to data catalogs and repositories. Additionally, pilot testing will help validate the prototype’s performance and provide opportunities to gather user feedback before moving toward full implementation.

7.3.1.  Use Cases and Functionalities

The demonstrator showcases its capabilities through real-world geospatial analysis scenarios. Key functionalities include:

7.3.2.  Data Sources and FAIR Evaluation

The demonstrator integrates various data sources, including:

7.3.3.  OGC Compliance and Interoperability

The Danti demonstrator currently indexes multiple OGC and NSG compliant data formats including GeoTIFF and NTF. Future plans include investigating the implementation of OGC API-Records to streamline and standardize metadata management, as well as OGC-API Processes to enable cross-platform analytical capabilities.

7.3.4.  Findings and Recommendations

The demonstrator evaluated different methods for enhancing AI-driven geospatial awareness in LLMs, including:

Key findings indicate that integrating AI-driven search with knowledge graphs significantly improves geospatial data interpretation and usability.

7.4.1.  Use Cases and Functionalities

The demonstrator explores scenarios where AI assists users in interpreting flood risk maps and other geospatial overlays. Users can interact with the system by entering prompts such as specific addresses to determine flood risks. Functionalities include:

Challenges include improving model reliability and minimizing hallucinations, such as misinterpreting map features or geographic contexts.

7.4.2.  Data collection tool

The Canadian flooding imagery data collector allows extraction of data by coordinates and zoom level. These data are extracted from the Flood Susceptibility Index 2015 map layer hosted on geo.ca then overlayed on the street map layer from ArcGISOnline with a specified opacity level.

 

Figure 4 — D120 (CRIM) - Street Map

 

Figure 5 — D120 (CRIM) - Flooding Map

 

Figure 6 — D120 (CRIM) - Combined Map

Central to this tool is the Processing Pipeline. This pipeline consists of a series of operations that transform user input into actionable data. Within this pipeline, four key tasks are performed:

These steps are illustrated in the following architecture diagram.

 

Figure 7 — D120 (CRIM) - Architecture Diagram

The Location Processing begins by finding the location within the user message using a first LLM call. Once identified, the address is fed into the Geocoding Service using the Nominatim Geocoder, which converts the address into precise geographic coordinates. Following geocoding, the Flood Image Retrieval step leverages a Geo Flooding Data Processor. This component accesses flood susceptibility imagery based on the coordinates obtained, providing visual data that is crucial for the analysis. The retrieved flood image is then added as context to the original user prompt. Grounding the response generation on this image helps limit hallucinations and allows the user to verify the claims made. This analysis yields insights into flood risks, considering the specific conditions of the user’s location. The result is communicated back to the user, enhancing their understanding of potential flood threats in their area.

An example of the use of this agent is shown in the figure below.

 

Figure 8 — D120 (CRIM) - Agent Screenshot

Canadian GEO flooding susceptibility agent

Using the previous tool, we can combine it with an AI model to build an agent which will be able, from a user natural language format input, extract location coordinates information to collect the flooding image which will be sent to the AI model for flooding analysis. In our case, we built this system using gpt4o and gpt4o-mini. The User Input Module is where the user provides a message indicating their location, such as “I live in Montréal, am I at risk of flooding”. This message serves as the input for the subsequent processing steps. The System Interface, represented by “ogc-demo.crim.ca” acts as an intermediary, taking the user message and preparing it for analysis. This step involves creating a prompt that combines the user message with the chosen processing pipeline, initiating the data flow through the system.

7.4.3.  Experiments and Findings

As part of the project, we conducted several experiments to assess the reliability of responses provided by such an assistant. We decided to evaluate the performance of the system in a Visual Question Answering (VQA) setting. This task involves asking a model to answer questions based on an image. While this task is relatively classic and often associated with vision tasks with a limited set of answers, the recent arrival of Large Language Models (LLMs) and, more importantly, Large Vision Models (LVMs)—which integrate a visual component—has renewed its approach.

Existing publicly available datasets VQA datasets generally do not include maps, even less ones relevant to flooding. For reference, MAPQA (https://arxiv.org/abs/2211.08545) only contained very general maps, and Floodnet (https://arxiv.org/abs/2012.02951) focused on satellite images showing the presence of floods but without precise localization. We decided to build a dataset that would more closely align with the interests of OGC and the sponsors for this pilot. We think that map-understanding or explanation through VLM is an interesting research avenue that has a lot of potential of impact in the geospatial domain due to the universality of images as a format. More specifically, allowing location-specific queries grounded in trusted sources of data on flooding and other related events seems to be of great interest for both governments and the insurance industry.

Dataset Creation

The dataset was built in a semi-automatic way using GPT-4, followed by human validation. It includes two types of questions:

The process involved using flood risk maps with a color scale ranging from light yellow to dark blue, where dark blue indicates high flood risk areas. The Flood Susceptibility Index 2015 (https://geo.ca/flood-mapping/flood-map-gallery/flood-susceptibility-index/) was used as a data source for this project. The index exposed as a map layer in geo.ca showcases the risk of flooding across Canada. We took screenshots of different areas of Québec (9 different images) as the following:

 

Figure 9 — D120 (CRIM) - Lonqueil

The goal was to generate precise questions about each image using targeted prompts, forcing the model to identify specific street names, intersections, or neighborhoods with unambiguous zone selections (to avoid referencing areas that include both flood-prone and non-flood-prone sections).

We then asked the model to generate answers based on the questions and corresponding images. These answers were manually validated by two annotators.

Dataset Overview

In this modest-sized dataset (104 occurrences), we can distinguish two types of question-answer pairs:

Expected answers: “yes”, “no”, or “partially”, accompanied by an explanation.

Question : Est-ce que les zones près de la rivière des Outaouais sont plus sujettes aux inondations que celles éloignées de la rivière ?

Answer: Oui.

Explanation: les zones proches de la rivière des Outaouais présentent des teintes bleues plus sombres, ce qui indique un risque plus élevé d’inondation par rapport aux zones éloignées

Question : Est-ce que les zones proches du lac Leamy sont indiquées comme étant à risque élevé d’inondation sur cette carte ?

Answer**: Oui.

Explanation: Lower Town montre un niveau de risque plus élevé (couleur bleue plus intense) par rapport à Rockcliffe Park, qui est situé en hauteur et semble moins vulnérable.

Expected answers: precise locations on the map.

Question : Quelles sont les zones principales de Montréal les plus à risque d’inondation (en bleu foncé) ?

Answer: Collège Reine-Marie entre la125 et Boulevard Saint-Michel, entre PArc Extension et Outremont, pied du Mont Royal au nord-ouest

Challenges: Model Hallucination

One of the primary challenges in creating the dataset was model hallucination, both in question generation and answers. In this context, hallucination is defined as:

For categorical questions, we identified:

Additionally, inconsistencies were noted between the explanations and short answers (yes, no, partially). In these cases, the explanation should have led to a different response than the one provided, with this phenomenon occurring 5 times in closed questions.

Thanks to manual validation performed after this first production, questions containing exclusively geographic hallucinations (places not present on the map) were removed from the experiments, leaving 82 valid occurrences. The 5 cases of hallucinations in the answers were not an issue since we used manual validation, considered the ground truth, to evaluate the results.

Experiments Conducted

We conducted three series of experiments to assess the assistant’s answer quality. We used exclusively OpenAI’s GPT-4-o model for this purpose.

1. Evaluation of Generation Techniques for Categorical Questions

We applied different generation techniques to answer the dataset’s questions and evaluated these responses against the manually validated ground truth. The image associated with the question was always provided to the assistant.

The generation techniques used are as follows:

The results are as follows:

7.4.4.  Prediction report for categorical questions

Table 1

Prediction Mode	Precision (Yes)	Precision (No)	Precision (In Part)	Recall (Yes)	Recall (No)	Recall (In Part)	F1 (Yes)	F1 (No)	F1 (In Part)
Direct Generation	NA	NA	NA	NA	NA	NA	NA	NA	NA
Chain-of-Thought	NA	NA	NA	NA	NA	NA	NA	NA	NA
Structured Generation	63	57	29	73	16	54	67	25	38
Structured Generation with CoT	56	71	17	73	20	23	63	31	19
Structured Generation Fallback	64	75	33	80	36	38	71	49	36
Structured Generation Fallback + CoT	69	73	44	91	44	31	78	55	36

Note : NA signifie “Non Applicable”.

We observed that the combination of the three techniques yields the best results. Although the results are encouraging, they vary significantly depending on the type of response: the “yes” response category scores significantly higher than the others.

We also identified cases where the model refuses to generate a response, negatively affecting the scores. Further analysis would be needed to understand the conditions leading to these refusals.

2. LLM as a Judge for Open-Ended Questions

We tested the use of a language model (LLM) as a judge to evaluate open-ended questions. Although our dataset is limited and not statistically significant, this approach was considered for future experiments with an expanded dataset. The LLM was used to verify:  — The factual accuracy of a statement based only on the facts present in the provided map.  — Whether the response contains knowledge explicitly present in the provided map or relies on the model’s implicit knowledge.

This method compares the LLM’s evaluation as a judge for automatically generated and manually generated responses. If the method is effective, the LLM should judge manually generated responses as 100% correct.

However, in our experiment, the LLM judged its own generated responses as factually correct 74% of the time, compared to 62% for manually generated responses. For explicit/implicit knowledge classification, it evaluated its own responses as explicit 80% of the time, compared to 77% for manual responses.

These results are not yet conclusive, given the 38% gap for manual responses (compared to the expected 100%), indicating that further experiments are needed, and probably an optimization of the LLM as a judge query.

3. Exploration of Implicit Knowledge

In the same spirit, we conducted a small exploratory experiment to assess the actual use of the flood layer in the image by the model. Specifically, this involved randomly adding an artificial blue layer to one part of the map to observe whether the model produced different responses based on this addition.

The model should theoretically adjust its responses based on the blue layer. If, however, the responses remain identical, this would indicate that the model relies more on its implicit knowledge of the location rather than the actual flood layer present in the image.

Similarly, one could ask whether the presence of a river on the map automatically leads the model to identify flood-prone areas near the watercourse, even in high-altitude areas where flooding would be unlikely.

This experiment was conducted exploratorily, without definitive conclusions. However, we observed variations in responses depending on the location of the overlay, which seems promising but requires further large-scale investigation.

Final Insights on the Integration of Generative AI for Flood Risk Map Analysis

At the end of the project, the creation of the assistant, the dataset, and the execution of various experiments allowed us to highlight key points regarding the integration of generative AI to support flood risk map analysis.

When the assistant is applied to a narrow domain (e.g., flood map analysis in specific areas), using structured prompting techniques (Chain of Thought, JSON) results in more reliable and easier-to-validate responses. Additionally, fallback mechanisms prove essential: when the model’s output does not conform to the expected format, it is possible to re-query the model or post-process its response to enforce a predefined schema (e.g., a JSON format). This avoids many inconsistencies and forces the model to clarify its information.

Conversely, for more general applications, the model’s performance decreases if its reasoning is not carefully structured, as it becomes more likely to drift towards approximate associations or irrelevant generalizations.

Integrating specialized APIs (geocoding services, on-the-fly map retrieval, detailed cartographic databases) significantly improves the quality and accuracy of responses. By having access to updated and localized information, the model is less likely to fabricate non-existent locations or overlook geographic specifics. These tools provide a factual grounding that can be cross-referenced with the model’s internal knowledge, reducing hallucination risks.

Each map has its own characteristics (legend, scale, symbolization style, colors, etc.). In this context, relying solely on general knowledge (e.g., that a river implies a flood risk) can lead to inaccurate generalizations. Furthermore, a model trained on specific maps may struggle to extrapolate to a different type of map with a very different style (e.g., a completely different color palette or the use of non-standard icons). This highlights the need for additional research and adaptation efforts, so that the model truly takes into account the legend and visual specifics of each cartographic source.

To reduce hallucinations, several strategies can be implemented:

These guardrails would be essential complementary mechanisms, as they help mitigate the model’s automatic biases and misinterpretations.

7.4.5.  Future Work and Conclusion

Overall, this experience highlights the value of integrating generative AI in flood risk map analysis. The results obtained with our prototype are promising, although this integration requires particular attention in prompt design, response validation, and hallucination management. The experiments conducted open the door to further research aimed at improving the reliability and generalizability of such a system.

We therefore propose the following research directions to continue this work:

Expanding the Dataset

Impact of Transparency and Layer Styles

Automatic Legend Retrieval and Analysis

Detection of “No Answer” Cases

Fine-Tuning and Specialized Training

Disclaimers and Contextualization

Improving LLM “as a Judge”

The research avenues outlined above provide concrete levers to better counter hallucinations, improve reasoning transparency, and make the tool more robust and adaptable to various cartographic environments.

7.5.1.  Use Cases and Functionalities

The demonstrator showcases a virtual assistant enabled by the large language models (LLMs), capable of answering users’ queries in plain text about the risks, hazards, and impacts of coastal flooding in Canada. Functionalities include:

7.5.2.  Data Sources and FAIR Evaluation

The project integrates datasets including the cause of flooding, financial impacts, insurance solutions, risk impacts, risk to building/ housing/real estate, economy, residential lending, etc. The data collection methodology of this project emphasizes identifying content with clear topics and focuses on organizing key points and relevant information within each topic.

The data sources are designed to align with the FAIR principles and are stored on platforms that are ate compliant with HTTPS standards, ensuring secure access. All data is open, free, and accessible, provided in standardized formats in English with detailed source information to ensure reusability.

7.5.3.  Findings and Recommendations

7.6.1.  Use Cases and Functionalities

The demonstrator showcases its functionality through real-world scenarios. The pilot used an initial geo-ontology to explore methods for bringing geospatial awareness into LLMs and identified school zones impacted by flooding due to levee breaches. The use cases emphasize:

 

Figure 11 — D120 (TerraFrame) - Insights in a human-readable format

7.6.2.  Data Sources and FAIR Evaluation

The project integrates diverse datasets, including spatial, infrastructure, and administrative data sourced from organizations like the US Army Corps of Engineers. The next phase will also include civil infrastructure data from the Internet of Water Coalition and the U.S. Census.

7.6.3.  OGC Compliance and Interoperability

One of the effort’s objectives is to evaluate OGC standards for making independently developed SKGs interoperable using machine-to-machine readable interfaces, maintaining referential integrity, and propagating changes over time. To that end, the following required metadata properties were identified: defining geo-ontology (i.e., schema for node types and edges), source, temporal validity, and version. Since SKGs are semantic structures that are curated in a decentralized manner, a fixed approach to a modal problem is needed. There does not appear to be an OGC standard to facilitate this.

7.6.4.  Findings and Recommendations

Different methods for integrating geospatial awareness modeled in SKGs into LLMs were explored, including natural language vector-embedded Retrieval-Augmented Generation (RAG) built from RDF triples, creating a RAG from structured Cypher queries with a Planner and ReAct agent, and using LLMs to generate Cypher queries. The latter was the most straightforward and dependable method for identifying cascading effects with multiple degrees of separation modeled in an SKG. It is recommended that the OGC continue to explore an SKG interoperability standard to enable the sustainable creation and maintenance of cross-domain SKGs.

7.7.1.  Sectoral Use Case — Coastal Resilience

The integration of GenAI extends the tool’s usage enabling advanced query interpretation and contextual responses for any coastal area in the world. The demonstrator assists in identifying vulnerable coastal zones, providing actionable insights for engaging communities through intuitive, understandable outputs. Applications in test areas such as Eresos Village (eastern Greece) and Kayafa Lake (western Greece) showcase the tool’s ability to dynamically adapt to different geographic and environmental conditions. The system computes and visualizes coastal vulnerability based on key geospatial indicators—such as land cover, coastal elevation, slope, erosion rates, and tidal influence—offering interactive visualizations and clear, actionable insights. This approach supports both expert users (e.g., urban planners, disaster response teams) and non-experts (e.g., local communities, decision-makers) in understanding and addressing coastal risks.

 

Figure 14 — D101 (Hartis) - AI-Assisted Coastal Vulnerability Analysis Results

7.7.2.  Data Sources and FAIR Evaluation

The demonstrator leverages multiple data sources, including Copernicus services (e.g., C3S, CAMS, and EUMETSAT), along with external geospatial platforms compliant with OGC standards. The integration of GenAI enhances the data retrieval and processing pipeline, ensuring seamless interaction with users.

To align with FAIR principles, the system prioritizes:

7.7.3.  Evaluation and Lessons Learned

The pilot process highlighted several key takeaways:

7.8.1.  Sectoral Use Cases (Health)

The primary use case for the D101 demonstrator is in public health, focusing on understanding and mitigating the impacts of heatwaves and droughts. Specific applications include:

Future extensions will explore integrated heat and drought indices to provide more robust insights into climate-driven health challenges.

During 2022, the UK endured record high temperatures of 40 degC and there were around 3 000 more deaths in the over-65s than usual in England and Wales. So, the indices have been tested over the UK (location in East Anglia, which is in the South East) for 2022.

7.8.2.  Data Sources and FAIR Evaluation

Pixalytics previously developed an OGC-compliant API service, combining meteorology, hydrology, and remote sensing open data/datasets to produce ARD data based on a composite of different indicators. The original API access was been set up following the Building Blocks for Climate Services approach, and within this pilot the workflow is accessed through a Jupyter Notebook that will be deployed on the WEkEO platform.

The Standardized Precipitation Index (SPI) and Universal Thermal Climate Index (UTCI) are being calculated using ERA5/ERA5-Land reanalysis data from the Copernicus Climate Change Service (C3S) Climate Data Store. It is also possible to download both parameters, pre-calculated ERA5-HEAT dataset, from C3S but there were issues with the retrieval failing.

SPI is used to characterize meteorological drought, quantifying the observed precipitation standardized departure from a selected probability distribution function that models the input precipitation data. UTCI is the air temperature of a reference outdoor environment that would elicit the same physiological response as the actual environment (Di Napoli et al. 2020). Its inputs include mean radiant temperature (MRT), which representing how human beings experience radiation, and the temperatures are classified into categories as grades of thermal stress.

Heat stress follows a diurnal pattern with UTCI values at 06:00 or 18:00 generally lower than UTCI values at 12:00 or 15:00, and it’s also more dominant in southern Europe (Di Napoli et al. 2017). To follow the approach of the pre-calculated ERA-HEAT dataset, hourly ERA-5 data was downloaded. Initially, the UTCI was calculated for just summer months (June to August) as these months have the greater likelihood of showing occurrences of heat stress. The calculated health index can also be used to model the impact of cold weather on health so the full year was calculated as well. Values of UTCI were calculated hourly using the xclim Python module, then the maximum value from the day was chosen and the average was calculated across each calendar month.

The SPI needed to be calculated over a long period of time of time and so daily data from 1985 to the present data was used, which took a long time to download as it needed to be downloaded a month at a time to comply with C3S’s policy. So, the data is cached for future analyses run at the same location and monthly data was also tested that would allow for a quicker download. Negative values indicate the situation is drier than normal while positive values indicate it is wetter.

The figure shows an example plot of SPI, UTCI and the Health Index for the UK in 2022 with, for example, low SPI and high UTCI values in the summer. The Health Index is calculated from the combination of SPI & UTCI, so has the high/positive values in the summer indicating heat/drought impact, with low/negative values indicating the health impact of cold/wet weather.

 

Figure 15 — D101 (Pixalytics) - Example plot of SPI, UTCI and Health Index for the South Eastern UK from 2022 to 2024:

7.8.3.  Evaluation and Lessons Learned

As we used daily maximum values for this analysis, so the highest value during the day was captured. Also, different temporal compositing was investigated (monthly versus daily) to speed up the CDS download time, and it was decided monthly values could be used for SPI while hourly values were needed for the UTCI calculation.

There is a delay while the data is accessed from the CDS and downloaded. The delay is reasonable for the SPI calculation, but more significant for UTCI and so users need to patient. A month at a time is downloaded, so if the connection fails it can be restarted and previously downloaded data used, providing a level of resiliency.

The developed code has been deployed and demonstrated using the WEkEO platform, which allows users to run applications within a pre-existing cloud infrastructure with harmonized data access. The developed code has been made available within a public GitHub repository, with archiving using Zenodo providing a DOI for released versions. The aim will be to build on this code base further within future pilot activities, supporting the testing of standards and interoperability.

10.1.1.  Data Quality and Availability

High-quality, climate-specific geospatial data is essential, as GenAI models depend on accurate datasets, including historical weather records, floodplain maps, hazard zones, and critical infrastructure data. Real-time data from Earth Observation (EO) systems, IoT climate sensors, and social media can enhance early warning systems and disaster impact assessments. Investing in data cleaning, validation, and quality control processes is paramount.

10.1.2.  Bias and Data Gaps

Addressing data gaps and biases is vital, as geospatial data is often unevenly distributed, leading to biased models. Filling data gaps in climate-vulnerable regions, such as small island states, coastal zones, and low-income urban areas, is essential. Techniques like data augmentation and synthetic data generation can simulate disaster scenarios to improve model robustness. Promoting data sharing and interoperability is equally important. Siloed data hinders progress, so encouraging open data initiatives, establishing standards like OGC ones, and developing tools for data transformation and interoperability are critical. The effort should include documented APIs and open data formats for easier access and integration.

10.1.3.  Interdisciplinary Collaboration

Collaboration between climate scientists, urban planners, disaster management agencies, and GenAI developers is paramount. This could include developing flood prediction models, drought resilience strategies, and wildfire monitoring frameworks.

10.1.4.  Metadata and Standards

Metadata specific to disaster resilience should include attributes such as evacuation routes, response times, and climate vulnerability indices. Standards like OGC should extend to disaster-relevant features, covering aspects such as provenance, accuracy, spatial and temporal resolution, and known biases.

Fostering interdisciplinary collaboration is essential to bridge expertise gaps. Effective integration of GenAI and geospatial data requires collaboration across fields such as geospatial science, AI/ML, software engineering, and domain-specific disciplines. Establishing collaborative platforms, including joint research projects, hackathons, workshops, and online communities, can facilitate knowledge sharing and accelerate innovation.

10.1.5.  Scalability and Robustness

The development of robust, scalable, and explainable AI models tailored to geospatial data is another key area of focus. Generic GenAI models may not be optimal for geospatial applications, necessitating research into architectures and training techniques designed to handle unique geospatial characteristics such as spatial autocorrelation, scale dependency, or complex spatial relationships. Scalability is also important to process massive geospatial datasets in real-time during disasters, leveraging cloud and distributed computing frameworks, requiring models and infrastructure that can handle large volumes of data efficiently.

10.1.6.  Explainability and Ethical AI

Explainability is equally important in life-critical applications like disaster risk prediction. For example, communities must trust flood forecasts to act promptly. Ethical AI must prioritize inclusion of indigenous knowledge and equity in resilience planning, ensuring outputs do not marginalize vulnerable populations. Addressing bias, ethical considerations, and trustworthiness is paramount. Methods for detecting and mitigating bias in geospatial data and GenAI models should be based on inclusive and fairness-aware datasets and careful evaluation of outputs.

Additionally, establishing trust and accountability is essential, requiring benchmarks and inspection rules to evaluate the accuracy and reliability of autonomous GIS solutions, including reviewing workflows, code, and outputs for potential biases. Promoting open-source development and standardization is another important step. Encouraging the development and sharing of open-source tools, libraries, and platforms can foster community involvement, accelerate innovation, and promote wider adoption.

10.1.7.  Functionality Enhancements

Focusing on enhanced functionality will unlock new possibilities. Autonomous systems should enable real-time disaster detection (e.g., floods, wildfires) by filtering and analyzing live geospatial data catalogs and web services, enabling LLMs to understand metadata, quality, and relevance for specific tasks. LLMs trained on disaster-specific geospatial datasets should assist in identifying safe evacuation routes, prioritizing infrastructure repairs, and simulating disaster scenarios.

Efforts should enable GenAI to answer and provide reasoning, requiring deeper geospatial knowledge and hypothesis generation. Investment in training Large Spatial Models (LSMs) on extensive geospatial data, akin to LLM training on text corpora, can significantly enhance spatial awareness and reasoning.

10.2.1.  Directions for Stakeholders

Governments and funding agencies should prioritize GenAI applications for climate resilience, such as adaptive planning, early warning systems, and post-disaster recovery. Investments should target data acquisition for high-risk regions and tools for climate impact attribution. To effectively guide the integration of GenAI with geospatial data, distinct yet interconnected steps are necessary for both stakeholders and developers. For stakeholders, including governments, research institutions, businesses, and funding agencies, a primary focus should be on building a robust data ecosystem. This involves:

Stakeholders should also foster interdisciplinary collaboration by:

Furthermore, stakeholders must ensure that AI solutions are ethical and equitable, focusing on inclusivity in disaster resilience planning by:

Finally, stakeholders should invest in education and training programs to develop a skilled workforce capable of working with GenAI and geospatial data, ensuring a smooth transition and maximizing the benefits of this technology.

10.2.2.  Directions for Developers

Developers should focus on building robust, scalable, and user-friendly GenAI tools and platforms. This involves:

Developers should also actively engage with stakeholders to understand their needs and priorities, ensuring that GenAI solutions are relevant, practical, and address real-world challenges.

10.3.1.  Research Directions

Several key areas require further research and development to fully realize the potential of GenAI in geospatial applications. Firstly, to develop GenAI models for climate resilience that predict cascading impacts, such as floods following hurricanes or power outages due to extreme heat. Focus on training Large Spatial Models (LSMs) for disaster scenarios, akin to LLMs for text, to enhance geospatial reasoning and response efficiency.

Research should also focus on developing methods for handling uncertainty and imprecision inherent in real-world geospatial datasets, including techniques for data imputation, error propagation, and uncertainty quantification. Secondly, enhancing explainability and interpretability of GenAI models in the geospatial domain for building trust and ensuring responsible use, given that geospatial data is often used in life-critical applications. This involves developing techniques that provide insights into the internal workings and potential biases of these models, allowing users to understand why a particular prediction or classification was made.

Developing multimodal GenAI models that combine EO, LiDAR, sensor networks, and crowd-sourced disaster data for comprehensive analysis is a major research direction. This involves developing techniques for data fusion, multimodal representation learning, and cross-modal reasoning. Research is also needed on developing efficient and scalable infrastructure for deploying and managing GenAI models in real-world geospatial applications. This includes methods for model compression, optimization, and deployment on resource-constrained edge devices.

Finally, investing in the development of LSMs trained on vast amounts of geospatial data, similar to how LLMs are trained on text corpora, is a transformative research direction. This involves developing new training paradigms, data augmentation techniques, and model architectures specifically designed for geospatial data.

10.3.2.  Education and Training

Equally important is to focus investing in education and training for developing geospatial AI expertise. Training programs should emphasize climate resilience applications, teaching professionals how to leverage GenAI for flood prediction, evacuation planning, and adaptive infrastructure design.

Moreover, encouraging public-private partnerships can leverage synergies between sectors. Public organizations can provide access to data and expertise, while private companies can contribute technological resources and market expertise, driving innovation and accelerating adoption of GenAI in geospatial applications. Together, they can interact with academia and educational programs to promote GenAI integration into geospatial science.