Conference Agenda

The project addresses, mainly with the use of Earth Observation data and techniques, two distinct themes : on the one hand climate change as this relates to the thermal resilience of cities and on the other urbanization and environment. In the former theme, the overall aim is to support climate friendly cities through the drafting of climate change adaptation plans as far as urban heat is concerned; in the latter case, the overall aim is to detect and assess urban geological hazards. Areas of application are in principle the overall Beijing and Athens urban areas, although results have strong replication potential for other cities as well.

In terms of climate change, the scientific objectives are: (a) to assess the impact of climate change to the urban thermal environment and work out long term series analysis (also with the use of ERA5 data) to define the response times of extreme air temperatures; (b) to examine the contribution of earth observation for cascade modelling (from RCMs to microscale) at the urban scale; (c) to study the relationship between urban form and the state of urban thermal environment both with respect to air and land surface temperatures; (d) to work out a methdology for delineating cities into climate zones; and (e) to define and map urban heat risk and assess climate resilience.

In terms of urbanization and environment (smart cities), the scientific objectives are: (a) to monitor and model urban geological hazards; (b) to combine remote sensing, geophysical prospecting, and hydrogeological theories methods (by using InSAR, ground penetrating radar, and multi-field numerical analysis) to establish three-dimensional monitoring network of land subsidence in urban area for hydrogeological process and (c) to identify land subsidence mode, establish dynamic models, quantify multi-field contributions, and reveal the mechanisms of land subsidence.

The research objectives of the project as well as the methodologies and a first set of results (estimates for the return period (frequency) of extreme air temperature values for both cities, extreme value analysis in terms of the number of days with daily maximum temperature above the 90th percentile of daily maximum temperature, etc.), will be presented along with a discussion on the potential of the project for climate friendly and smart cities as well as for its replication potential.

Today, 55 per cent of the world’s population live in cities and another 2.5 billion people is expected to move to urban areas by 2050 (UN, 2018). Rapid urbanization poses significant social and environmental challenges, including sprawling informal settlements, increased pollution, urban heat island, loss of biodiversity and ecosystem services, and making cities more vulnerable to disasters. Therefore, timely and accurate information on urban changing patterns on both 2D and 3D is of crucial importance to support sustainable and resilient urban planning and monitoring of the UN 2030 Urban Sustainable Development Goal (SDG). The overall objective of this project is to develop innovative, robust and globally applicable methods, based on Earth observation big data and AI, for urban land cover mapping and urbanization monitoring. The innovative aspects of this research include development of novel methodology through interdisciplinary research and supporting planning smart, sustainable and resilient cities. The proposed methodology includes the development of semantic segmentation with better generalization with weakly supervised and self-supervised training for urban land cover mapping, deep Siamese convolutional neural network for change detection, and unsupervised temporal anomaly detection for time series analysis. In addition, two SARbased methods, i.e, SAR interferometry and radargrammetry, will be explored for 3D change detection as urban areas not only expend in 2D but also in the 3rd dimension. Open and free Earth observation big data will be used to demonstrate the new deep learning-based methods in Jing-Jin-Ji, Yangtze River Delta, Yellow River Delta and Pear River Delta in China plus ten cities around the world including Stockholm, Lagos, Mumbai. It is anticipated that detailed urban land cover information and their changes will be mapped detected in a timely and accurate manner. The urban change in 3D will be estimated to better understand urban density and environmental impact. This research is expected to contribute to 1) advance EO science, technology and applications beyond the state of the art, 2). timely and reliable updating of urban databases to support sustainable planning at municipal and regional levels, 3) the monitoring objectives of the national authorities and the UN SDG 11: make cities and human settlements inclusive, safe, resilient and sustainable. The project is partially funded by the projects that the team partners have been secured. Specifically, the EOAI4ChangeDetection project funded KTH Digital Futures, Sentinel4Urban project is funded by SNSA, ESA CCI HR Landcover. The Chinese partners also have existing projects will apply for the funding from Natural Science Foundation of China and related provinces to support this project.

Surface waves are one of the most common phenomena in the oceans. The accurate monitoring and forecasting of waves are critical for guaranteeing the safety of all kinds of marine activities, such as sailing and fishing, and are also of great importance to understanding air-sea interactions, which significantly impact weather and climate projections. Remotely sensed ocean waves from European and Chinese space missions have significantly supplemented the insufficient coverage of traditional wave observations such as buoys. The objectives of this program are improving the wave remote sensing and enhancing the positive effect of assimilation. The progresses are listed below:

1). A deep learning technique is novelly applied for the calibration of Chinese HY2B SWH and wind speed. Deep neural network (DNN) is built and trained to correct SWH and wind speed by using input from parameters provided by the altimeter such as sigma0, sigma0 standard deviation (STD). The results based on DNN show a significant reduction of the bias, root mean square error (RMSE), and scatter index (SI) for both SWH and wind speed. Several DNN schemes based on different combination of input parameters have been examined in order to obtain the best model for the calibration. The analysis reveals that sigma0 STD is a key parameter for the calibration of HY2B SWH and wind speed.

2). In addition to the nadir significant wave height (SWH), the Surface Waves Investigation and Monitoring (SWIM) onboard Chinese-French Oceanic SATellite (CFOSAT) provides two additional columns of wave spectra observations within wavelengths from 70 m to 500 m. A model based on a DNN is developed to retrieve the total SWH from the partially wave spectra observed by SWIM. The DNN model uses the parameters from both the SWIM spectra and the nearest nadir as the inputs, and the DNN is trained on the SWH from cross-matched altimeter observations. The DNN-based acquisition of the SWH is verified to achieve a high accuracy. A set of assimilation experiments are performed based on MFWAM and show promising results. Compared to the assimilation of SWIM nadir SWHs only, the addition of the newly obtained SWIM SWH notably enhances the positive impacts of assimilation, not only proving the effectiveness and accuracy of the DNN model but also demonstrating the unique potential of SWIM in wave assimilation.

3). The accuracy of a wave model can be improved by assimilating an adequate number of remotely sensed wave heights. The SWIM and Scatterometer (SCAT) instruments onboard CFOSAT provide simultaneous observations of waves and wide swath wind fields. Based on these synchronous observations, a method for retrieving the SWH over an extended swath is developed using the DNN approach. With the combination of observations from both SWIM and SCAT, the SWH estimates achieve significantly increased spatial coverage and promising accuracy. As evidenced by the assessments of assimilation experiments, the assimilation of this ‘wide swath SWH’ achieves an equivalent or better accuracy than the assimilation of the traditional nadir SWH alone and enhances the positive impact when assimilated with the nadir SWH.

Overall, the deep learning, which is based on artificial neural networks, has proved its efficiency and effectiveness in improving the European and Chinese wave remote sensing missions, and obtaining better assimilation effects in wave numerical model simulations.

We present an ocean visualization framework, which focuses on analyzing multidimensional and spatiotemporal ocean data. GPU-based visualization methods are explored to effectively visualize ocean data. An improved ray casting algorithm for heterogeneous multisection ocean volume data is presented. A two-layer spherical shell is taken as the ocean data proxy geometry, which enables. oceanographers to obtain a real geographic background based on global terrain. An efficient ray sampling technique including an adaptive sampling technique and a preintegrated transfer function is proposed to achieve high-effectiveness and high-efficiency rendering. Moreover, an interactive transfer function is also designed to analyze the 3D structure of ocean temperature and salinity anomaly phenomena. Based on the framework, an integrated visualization system called i4Ocean is created. The visualization of ocean temperature and salinity anomalies extracted interactively by the transfer function is demonstrated.

The Lagrangian eddies in the western Pacific Ocean are identified and analysed based on Maps of Sea Level Anomaly (MSLA) data from 1998 to 2018. By calculating the Lagrangian eddy advected by the AVISO velocity field, we analyse the variations in Lagrangian eddies and the average transport effects on different time scales. By introducing the Niño coefficient, the lag response of the Lagrangian eddy to El Niño is found. These data are helpful to further explore the role of mesoscale eddies in ocean energy transfer. Through normalized chlorophyll data, we observed chlorophyll aggregation and hole effects caused by Lagrangian eddies. These findings demonstrate the important role of Lagrangian eddies in material transport. The transportation volume of the Lagrangian eddy is calculated quantitatively, and several major transport routes have been identified, which helps us to more accurately and objectively estimate the transport capacity of Lagrangian eddies in the western Pacific Ocean.

The overall goal of this project is to provide an effective solution for large-scale dense Satellite Image Time Series analysis, being capable of automatic discovery of regularities, relationships, and dynamic evolution patterns that leads to comprehensive understanding of the underlying processes of specific scenes and targets. Young researchers, postgraduate and PhD students from China and Romania joined the research workplan targeting to access various optical and SAR data from Sentinel, ESA, ESA TMP and Chinese Earth observation data, benefiting from EO data complementarity.

In the frame of the first objective of the project this paper addresses the supervised learning techniques for object extraction and semantic classification of EO-SAR urban scenes. The evaluation and validation process considers one of the two envisaged uses cases: monitoring the urban evolution of Shanghai, China, in support of smart and sustainable urban information services.

The need to exploit spatial and temporal information content of EO data increases with a wide range of applications, including urban development. OpenSARUrban is Sentinel-1 dataset dedicated to the content- related interpretation of urban SAR scene, covering 21 major cities of China. This set includes patches of “Denselow”, “General Residential”, “High buildings” and “Single Building”, all composed of strong scattering points reflected from the building surface, that are hard to be classified even by trained experts. The majority of the methods addressing image classification focus on the algorithm design, neglecting the fact that, the dataset itself is an important factor affecting classification performance, particularly for SAR images.

Open Set Recognition (OSR) describes a scenario where new classes, unseen in training, appear in testing challenging the classifiers to not only accurately classify the know classes- labeled positive training samples, but also effectively deal with completely unknow classes.

The SAR Distinguishability Analysor (SAR-DA) we propose, evaluates the distinguishability of the OpenSARUrban dataset. By modeling the latent multivariate Gaussian distribution of each class, SAR-DA can not only classify the classes seen in the training phase, but also can recognize unknown sample if a test sample is out of any known distribution. Each class in OpenSARUrban is set unknown alternatively, then we apply the SAR-DA on the split dataset in OSR and supervised setting. The distinguishability can be reflected by the unknown classification precision. Most importantly, though the classes are semantically different from each other, some classes are similar and of low distinguishability. In addition, SAR Dataset-wise Separability Index (DSI) and SAR Class-wise Separability Index (CSI) are proposed to quantify the separability in open set condition from the dataset level and class level respectively. Extensive experiments have been performed and the results demonstrated that in open set condition, the data set level separability is nearly half of that in supervised setting, leading to more difficult classification than under supervised conditions. In class level, even though the SAR image classes are semantically different from each other, there exits more or less overlap between the latent distributions of supervised known classes and unknown class, classes with low CSI are harder to be recognized as unknown correctly when it is unknown.

This may be the first work that adopts the OSR method to evaluate to evaluate the distinguishability of SAR classification dataset.

The innovation of the research approach could be highlighted as follows:

(1) By modeling each known class as a multivariate Gaussian distribution, SAR Separability Analysor (SAR-SA) is proposed to for known class classification and unknown class recognition.

(2) Implementing the idea of class scatter matrix, Dataset-wise Separability Index (DSI) is defined to quantify the separability of a dataset from dataset level in open set setting.

(3) Combing precision and recall results, Class-wise Separability Index (CSI) is defined by using 𝐹2 score to quantify the separability of each class from class level in open set setting.

(4) Two SAR image datasets were prepared for relevant experiments. These sets also enabled the detailed analysis of results, highlighting the difficulty of classification tasks in open set condition.

The results mentioned above were accepted for dissemination at IGARSS 2021. A journal paper “Analyzing the Separability of SAR Classification Dataset in Open Set Condition” was submitted to IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. Moreover, Ning Liao won the fourth place at “2020 Gaofen Challenge on Automated

High-Resolution Earth Observation Image Interpretation”.

This research work will continue to focus on the discovery of unknown classes in EO scenes. We are also preparing an abstract for ESA Phi-Week while UPB team focused on a dense satellite image time series preparation for the landcover monitoring of Danube Delta, a UNESCO protected site in Dobrogea- Romania.