{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Cloud Masks\n",
"\n",
"The products contain several raster and vector files that represent cloud masks or cloud propabilities.\n",
"\n",
"This notebook explores the contents of these files and evaluates how a helper for dealing with cloud masks can be implemented."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"with r.open(raster_cloud_mask) as mask:\n",
" rplt.show(mask.read() > 64, transform=mask.transform)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The vector cloud mask path can be obtained by manipulating the path to the raster cloud mask:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'zip+file://input/tempelhofer_feld/S2A_MSIL2A_20190114T101351_N0211_R022_T32UQD_20190114T113404.zip!/S2A_MSIL2A_20190114T101351_N0211_R022_T32UQD_20190114T113404.SAFE/GRANULE/L2A_T32UQD_A018609_20190114T101352/QI_DATA/MSK_CLOUDS_B00.gml'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"vector_cloud_mask = raster_cloud_mask.replace(raster_cloud_mask.split('/')[-1], 'MSK_CLOUDS_B00.gml')\n",
"vector_cloud_mask"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Trying to read the cloud mask from the zip file"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"import geopandas as gpd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Fiona needs very long to open the GML file, the following lines are commented out because they did not terminate in an acceptable time (< 5min):"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"#with fiona.open(vector_cloud_mask) as f:\n",
"# pass"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"# gpd.read_file(vector_cloud_mask)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The workaround is to unpack the vector file before opening it:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/opt/conda/lib/python3.8/site-packages/geopandas/geodataframe.py:422: RuntimeWarning: Sequential read of iterator was interrupted. Resetting iterator. This can negatively impact the performance.\n",
" for feature in features_lst:\n",
":21: RuntimeWarning: Sequential read of iterator was interrupted. Resetting iterator. This can negatively impact the performance.\n",
" cloud_mask_fiona = unary_union([shape(f['geometry']) for f in features])\n"
]
}
],
"source": [
"import fiona\n",
"from pathlib import Path\n",
"from shapely.geometry import shape\n",
"from shapely.ops import unary_union\n",
"from tempfile import TemporaryDirectory\n",
"from zipfile import ZipFile\n",
"\n",
"tmp_path = vector_cloud_mask.replace('zip+file://', '')\n",
"zip_path, file_path = tmp_path.split('!/')\n",
"\n",
"with TemporaryDirectory() as tmp_dir, ZipFile(zip_path) as f:\n",
" f.extract(file_path, tmp_dir)\n",
" tmp_path = Path(tmp_dir) / file_path\n",
" \n",
" # read with geopandas\n",
" cloud_mask_geopandas = gpd.read_file(tmp_path)['geometry'].unary_union\n",
" \n",
" # skip geopandas and read with fiona; this generates a RuntimeWarning\n",
" # about performance\n",
" with fiona.open(tmp_path) as features:\n",
" cloud_mask_fiona = unary_union([shape(f['geometry']) for f in features])"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
""
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cloud_mask_geopandas"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`geopandas` can be skipped and instead `fiona` directly used:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
""
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cloud_mask_fiona"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the aspect ratio is different than the raster file plotted above."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using the helper function"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
""
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sentinel_helpers import scihub_cloud_mask\n",
"scihub_cloud_mask(product_path)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}