Chapter 5 Standard Operational Procedure (SOP)

This chapter is arranged to different sections according to the most frequent type of disaster in previous Sentinel Asia Activities. On each section, the explanation will be based on available data.

5.1 Data Download

5.1.1 Sentinel Asia Website

https://youtu.be/yZJi4FRw4Ps

5.1.2 IDC Data Download via FTP

Data can be downloaded from Google Cloud Compute Engine via FTP and put in HTTP folder.

5.2 Data processing

###Flood

5.2.0.1 SAR

5.2.0.1.1 ArcGIS

Before processing the data, it is better to check the processing level from the filename.

The data (pre and post flood) was processed through 5 main steps: (1) Data processing including: Radiometric calibration, Speckle filtering, and Coordinate transformation; (2) Image differencing using image pre and post-flood; (3) Thresholding of backscatter changes; (4) Image filtering using majority filter; and (5) Mapping of flooded area. The methodology is shown in the figure below.

Image for Workflow

Figure 5.1: Flood Detection General Workflow

(Step 1: ArcGIS Pro) Radiometric calibration, Speckle filtering, and Coordinate transformation

Raster Function in ArcGIS Pro is used to process the images (pre and post-flood), with flowchart as follow:

Figure 5.2: ALOS-2 radiometric and geometric calibration Workflow

To convert raw image to sigma nought with the formula 20 * Log10(DN) – 83

Figure 5.3: ALOS-2 radiometric calibration formula

To do image filtering with Lee filter (5x5 window size).

Figure 5.4: ALOS-2 speckle reduction

To convert coordinate to GCS_WGS_1984

Figure 5.5: Reprojection to GCS WGS84

(Step 2: ArcGIS Pro) Image differencing Using the Calculator raster function to create a difference image, with the formula Change = (Post-flood) - (Pre-flood)

Figure 5.6: Image differencing

(Step 3: ArcGIS Pro) Thresholding of backscatter changes Using the Calculator raster function to extract flood inundation areas from the result in step 2 with threshold < -3.

Figure 5.7: Image thresholding

(Step 4: ArcGIS Pro) Image filtering Majority filter is applied to reduce noise (salt and pepper) from result in step 3. (ToolboxesAnalyst ToolsFilter)

Figure 5.8: Image filtering

Then, convert raster to polygon and export to shapefile (.shp) format for a further step in ArcMap.

5.2.0.1.2 QGIS

Flood detection can be carried out in QGIS by using Graphical Modeling as shown in the following video

https://youtu.be/HR_7kENFGT4

5.2.0.2 Optical data

5.2.0.3 Google Earth Engine (GEE)

// Load Sentinel-1 images to map the flooding area, // This script was originally written by Simon Ilyushchenko (GEE team) // Default location var pt = ee.Geometry.Point(96.4633,17.79); //, , Grand Morin near Coulommiers

// Load Sentinel-1 C-band SAR Ground Range collection (log scaling, VV co-polar, Descending) var collection = ee.ImageCollection(‘COPERNICUS/S1_GRD’).filterBounds(pt). filter(ee.Filter.listContains(‘transmitterReceiverPolarisation’, ‘VV’)). filter(ee.Filter.eq(‘orbitProperties_pass’, ‘DESCENDING’)). select(‘VV’);

// Filter by date, ensure that the result will cover only 1 day

var before = collection.filterDate(‘2016-05-5’, ‘2016-05-6’).mosaic(); var after = collection.filterDate(‘2018-07-24’, ‘2018-07-25’).mosaic();

// Threshold smoothed radar intensities to identify “flooded” areas. var SMOOTHING_RADIUS = 100; var DIFF_UPPER_THRESHOLD = -3; var diff_smoothed = after.focal_median(SMOOTHING_RADIUS, ‘circle’, ‘meters’) .subtract(before.focal_median(SMOOTHING_RADIUS, ‘circle’, ‘meters’)); var diff_thresholded = diff_smoothed.lt(DIFF_UPPER_THRESHOLD);

//Define the extent for exporting the result var geometry = ee.Geometry.Rectangle([105.321, 20.3574, 106.971, 21.313473]);

// Display map Map.centerObject(pt, 10); Map.addLayer(before, {min:-30,max:0}, ‘Before flood’); Map.addLayer(after, {min:-30,max:0}, ‘After flood’); Map.addLayer(after.subtract(before), {min:-10,max:10}, ‘After - before’, 0); Map.addLayer(diff_smoothed, {min:-10,max:10}, ‘diff smoothed’, 0); Map.addLayer(diff_thresholded.updateMask(diff_thresholded), {palette:“0000FF”},‘flooded areas - blue’,1);

// Export the image, specifying scale and region.

// Pre-flood Export.image.toDrive({ image: before, description: ‘MMM_image_SentinelBefore’, scale: 30, region: geometry });

// Post-flood

Export.image.toDrive({ image: after, description: ‘MMM_image_SentinelAfter’, scale: 30, region: geometry });

//Flood Export.image.toDrive({ image: diff_thresholded, description: ‘MMM_image_SentinelFlood’, scale: 30, region: geometry });

5.2.0.4 Permanent Water

5.2.1 Earthquake

5.2.1.1 GMT5SAR

Client software

If you use Windows, you have to install BitVise SSH Client.

– The software can be downloaded from https://www.bitvise.com/ssh-client-download. BitVise SSH client provides terminal application to connect via SSH and

– Create New Profile.

Figure 5.9: Create New Profile in BitVise SSH Client

– Save the profile as macpro in the folder.

Figure 5.10: Saving the new profile

– Fill in the New Profile and click Save Profile.

Figure 5.11: Completing the profile

– Click Login

Figure 5.12: BitVise xterm

Figure 5.13: BitVise SFTP

If you use Mac or Linux, you can connect to MacPro by using Terminal.

– Connect to MacPro with your provided username (e.g., dan@203.159.29.212)

firmanhadi:~ firmanhadi$ ssh gic@203.159.29.212
Password:

Folder and file preparation

Go to Documents folder and create new folder (e.g., IDN3)

IMS:~ gic$ cd Documents
IMS:Documents gic$ mkdir IDN3

Go to IDN3 folder

IMS:Documents gic$ cd IDN3

Create new folder (raw, topo)

IMS:IDN3 gic$ mkdir raw && mkdir topo

Copy the config file from BOOTCAMP

IMS:IDN3 gic$ cp /Volumes/BOOTCAMP/IDN_SAR/config.alos2.slc.txt .

Confirm the structure of the folder

IMS:IDN3 gic$ ls -l
total 8
-rwxr-xr-x@ 1 gic  staff  3605 Sep  6 10:37 config.alos2.slc.txt
drwxr-xr-x  2 gic  staff    68 Sep  6 10:31 raw
drwxr-xr-x  2 gic  staff    68 Sep  6 10:31 topo

Download DEM from GMTSAR website

With the browser in your computer, go to this link http://topex.ucsd.edu/gmtsar/demgen/
Enter the coordinate for north, south, east and west. The boundary cannot span more than 4 longitude by 4 latitude degrees.
Click Generate to process.
If it has finished, click Download to get the result.
Extract the result to temporary folder in your computer

Figure 5.14: Downloading DEM for GMT5SAR processing

Copying the ALOS-2 data and DEM

Copy the ALOS-2 data (IMG, LED and VOL- files) to raw folder

Figure 5.15: Uploading ALOS-2 data to raw folder

Extract the files downloaded from DEM Generation process in GMTSAR website and copy all of the files to topo folder.

Figure 5.16: Uploading DEM files to topo folder

Modifying config file

If it is the first time in processing, set the value for proc_stage to 1 (proc_stage = 1).
Set dummy value for SLC_factor, 4.0 (SLC_factor = 4.0). This is important as we will receive a warning message to change the value to the correct one.
Set topo_phase to 1, to subtract topo_ra from the phase (topo_phase = 1)
Set shift_topo to 0.
Set switch_master to 1 if we assume master as repeat and slave as reference. Set it to 0, if we assume master as reference and slave image as repeat. The default value is 0 (switch_master = 0)
For filtering, set filter_wavelength = 100, and filter1 = gauss_alos_100m.
Set decimation of images to 1 if we want higher resolution image and set it to 2 if we want smaller resolution (dec_factor = 1). For training purpose, set it to 2.
Set correlation threshold for snapu to 0.1 (threshold_snaphu = 0.1)
Leave the option blank to unwrap the whole region (region_cut = ). Note that, in the latest version of GMT5SAR, this option will be executed if we write the config for interpolation (near_interp = 1). If we don’t put this option for interpolation, the result will be unwrapped to half of the image.
For masking the Lake or Ocean, set switch_land to 1 (switch_land = 1).
Set defomax = 0 for smoother unwrapped phase.
Set threshold_geocode = .10

Running the InSAR process

Under IDN3 folder, by using Command Line, run this command “p2p_ALOS2_SLC.csh IMG-master IMG-slave config.alos2.slc.txt”. Change IMG-master with the full file name of IMG as master and IMG-slave with the full file name of slave IMG.
Wait for the process to finish. It will take some time depend on the CPU speed and also the type of data. SM1 data will take longer than SM3.
The results will be created in intf folder, under IND folder.
KML files can be opened using Google Earth. Files without _ll suffix are still in azimuth and range coordinates, those with _ll suffix mean already in Geographic Coordinate System WGS84.

Converting to GeoTIFF

Files with .grd extension can be opened using QGIS and then exported to GeoTIFF format.
The results in TIFF extension are used in ArcMap for layouting.

Sentinel Asia Handbook

Chapter 5 Standard Operational Procedure (SOP)

5.1 Data Download

5.1.1 Sentinel Asia Website

5.1.2 IDC Data Download via FTP

5.2 Data processing

5.2.0.1 SAR

5.2.0.1.1 ArcGIS

5.2.0.1.2 QGIS

5.2.0.2 Optical data

5.2.0.3 Google Earth Engine (GEE)

5.2.0.4 Permanent Water

5.2.1 Earthquake

5.2.1.1 GMT5SAR

5.2.2 Volcano

5.2.3 Landslide

5.2.4 Glacier Lake Outburst Flood

5.2.5 Oil Spill

5.2.6 Forest Fire

5.3 Data Layout

5.3.1 Check and re-check