rio-cogeo can also be integrated directly in your custom script. See rio_cogeo.cogeo.cog_translate function.
+e.g:
+from rio_cogeo.cogeo import cog_translate
+
+def _translate(src_path, dst_path, profile="webp", profile_options={}, **options):
+ """Convert image to COG."""
+ # Format creation option (see gdalwarp `-co` option)
+ output_profile = cog_profiles.get(profile)
+ output_profile.update(dict(BIGTIFF="IF_SAFER"))
+ output_profile.update(profile_options)
+
+ # Dataset Open option (see gdalwarp `-oo` option)
+ config = dict(
+ GDAL_NUM_THREADS="ALL_CPUS",
+ GDAL_TIFF_INTERNAL_MASK=True,
+ GDAL_TIFF_OVR_BLOCKSIZE="128",
+ )
+
+ cog_translate(
+ src_path,
+ dst_path,
+ output_profile,
+ config=config,
+ in_memory=False,
+ quiet=True,
+ **options,
+ )
+ return True
+import numpy
+
+import mercantile
+
+from rasterio.io import MemoryFile
+from rasterio.transform import from_bounds
+
+from rio_cogeo.cogeo import cog_translate
+from rio_cogeo.profiles import cog_profiles
+
+# Create GeoTIFF profile
+bounds = mercantile.bounds(mercantile.Tile(0,0,0))
+
+# Rasterio uses numpy array of shape of `(bands, height, width)`
+width = 1024
+height = 1024
+nbands = 3
+
+img_array = tile = numpy.random.rand(nbands, height, width).astype(numpy.float32)
+
+src_transform = from_bounds(*bounds, width=width, height=height)
+
+src_profile = dict(
+ driver="GTiff",
+ dtype="float32",
+ count=nbands,
+ height=height,
+ width=width,
+ crs="epsg:4326",
+ transform=src_transform,
+)
+
+
+with MemoryFile() as memfile:
+ with memfile.open(**src_profile) as mem:
+ # Populate the input file with numpy array
+ mem.write(img_array)
+
+ dst_profile = cog_profiles.get("deflate")
+ cog_translate(
+ mem,
+ "my-output-cog.tif",
+ dst_profile,
+ in_memory=True,
+ quiet=True,
+ )
+from rasterio.io import MemoryFile
+
+from rio_cogeo.cogeo import cog_translate
+from rio_cogeo.profiles import cog_profiles
+
+from boto3.session import Session as boto3_session
+
+dst_profile = cog_profiles.get("deflate")
+
+with MemoryFile() as mem_dst:
+ # Important, we pass `mem_dst.name` as output dataset path
+ cog_translate("my-input-file.tif", mem_dst.name, dst_profile, in_memory=True)
+
+ # You can then use the memoryfile to do something else like
+ # upload to AWS S3
+ client = boto3_session.client("s3")
+ client.upload_fileobj(mem_dst, "my-bucket", "my-key")
+rio-cogeo provide a --web-optimized option which aims to create a web-tiling friendly COG.
+Output dataset features:
+Important
+Because it will certainly create a larger file (with padding tiles on the side of the file), a nodata value, an alpha band or an internal mask should +be present in the input dataset. If not the original data will be surrounded by black (0) data.
+By default rio cogeo will create a dataset with 512x512 internal tile size.
+This can be updated by passing --co BLOCKXSIZE=64 --co BLOCKYSIZE=64 options.
Web tiling optimization
+Creating a Web-Optimized COG, means you'll get a file which is perfectly aligned (bounds and internal tiles) with the mercator grid and with resolution (for the raw data and overview) which map the mercator zoom level resolution. This enable to reduce the number of GET request a dynamic tiling service needs to do to create a map tile from your COG.
+if the input dataset is not aligned to web mercator grid, the tiler will need +to fetch multiple internal tiles.
+By default rio cogeo will calculate the optimal overview level based on dataset +size and internal tile size (overview should not be smaller than internal tile +size (e.g 512px). Overview level will be translated to decimation level of +power of two:
+overview_level = 3
+overviews = [2 ** j for j in range(1, overview_level + 1)]
+print(overviews)
+[2, 4, 8]
+By default rio cogeo DO NOT forward band metadata (e.g statistics) to the output dataset.
+$ gdalinfo my_file.tif
+...
+Band 1 Block=576x1 Type=Float64, ColorInterp=Gray
+ NoData Value=999999986991104
+ Unit Type: mol mol-1
+ Metadata:
+ long_name=CO2 Dry-Air Column Average
+ missing_value=9.9999999e+14
+ NETCDF_DIM_time=0
+ NETCDF_VARNAME=XCO2MEAN
+ units=mol mol-1
+ _FillValue=9.9999999e+14
+
+$ rio cogeo my_file.tif my_cog.tif --blocksize 256
+
+$ gdalinfo my_cog.tif
+...
+Band 1 Block=256x256 Type=Float64, ColorInterp=Gray
+ NoData Value=999999986991104
+ Overviews: 288x181
+You can use --forward-band-tags to forwards the band metadata to output dataset.
$ rio cogeo create my_file.tif my_cog.tif --blocksize 256 --forward-band-tags
+$ gdalinfo my_cog.tif
+...
+Band 1 Block=256x256 Type=Float64, ColorInterp=Gray
+ NoData Value=999999986991104
+ Overviews: 288x181
+ Metadata:
+ long_name=CO2 Dry-Air Column Average
+ missing_value=9.9999999e+14
+ NETCDF_DIM_time=0
+ NETCDF_VARNAME=XCO2MEAN
+ units=mol mol-1
+ _FillValue=9.9999999e+14
+By default rio-cogeo will forward any nodata value or alpha channel to the +output COG.
+If your dataset type is Byte or Unit16, you could use internal bit mask
+(with the --add-mask option) to replace the Nodata value or Alpha band in
+output dataset (supported by most GDAL based backends).
Note: when adding a mask with an input dataset having an alpha band you'll
+need to use the bidx options to remove it from the output dataset.
# Replace the alpha band by an internal mask
+$ rio cogeo mydataset_withalpha.tif mydataset_withmask.tif --cog-profile raw --add-mask --bidx 1,2,3
+Important
+Using internal nodata value with lossy compression (webp, jpeg) is not
+recommended. Please use internal masking (or alpha band if using webp).
rio-cogeo modules adds a cogeo sub-command to your rasterio (rio) CLI. Three (3) commands are then available: create, info and validate.
$ rio cogeo --help
+ Usage: rio cogeo [OPTIONS] COMMAND [ARGS]...
+
+ Rasterio cogeo subcommands.
+
+ Options:
+ --version Show the version and exit.
+ --help Show this message and exit.
+
+ Commands:
+ create Create COGEO
+ info Lists information about a raster dataset.
+ validate Validate COGEO
+$ rio cogeo create --help
+ Usage: rio cogeo create [OPTIONS] INPUT OUTPUT
+
+ Create Cloud Optimized Geotiff.
+
+ Options:
+ --bidx, -b Band indexes to copy.
+ --cog-profile, -p CloudOptimized GeoTIFF profile (default: deflate). [jpeg|webp|zstd|lzw|deflate|packbits|lzma|lerc|lerc_deflate|lerc_zstd|raw]
+ --nodata Set nodata masking values for input dataset.
+ --add-mask Force output dataset creation with an internal mask (convert alpha band or nodata to mask).
+ --blocksize Overwrite profile's tile size.
+ --dtype, -t Output data type. [ubyte|uint8|uint16|int16|uint32|int32|float32|float64]
+ --overview-level Overview level (if not provided, appropriate overview level will be selected until the smallest overview is smaller than the value of the internal blocksize)
+ --overview-resampling Overview creation resampling algorithm (default: nearest). [nearest|bilinear|cubic|cubic_spline|lanczos|average|mode|gauss]
+ --overview-blocksize Overview's internal tile size (default defined by GDAL_TIFF_OVR_BLOCKSIZE env or 128)
+ --web-optimized, -w Create COGEO optimized for Web.
+ --zoom-level-strategy Strategy to determine zoom level (default: auto). [lower|upper|auto]
+ --zoom-level Zoom level number for the highest resolution. If this option is specified, `--zoom-level-strategy` is ignored.
+ --aligned-levels Number of overview levels for which GeoTIFF tile and tiles defined in the tiling scheme match.
+ --resampling, -r Resampling algorithm (default: nearest). Will only be applied with the `--web-optimized` option. [nearest|bilinear|cubic|cubic_spline|lanczos|average|mode|max|min|med|q1|q3|sum]
+ --in-memory / --no-in-memory Force processing raster in memory / not in memory (default: process in memory if smaller than 120 million pixels)
+ --allow-intermediate-compression Allow intermediate file compression to reduce memory/disk footprint.
+ --forward-band-tags Forward band tags to output bands.
+ --forward-ns-tags Forward namespaced tags to output dataset.
+ --threads Number of worker threads for multi-threaded compression (default: ALL_CPUS)
+ --use-cog-driver Use GDAL COG Driver (require GDAL>=3.1).
+ --tms PATH Path to TileMatrixSet JSON file.
+ --co, --profile Driver specific creation options. See the documentation for the selected output driver for more information.
+ --config GDAL configuration options.
+ --quiet, -q Remove progressbar and other non-error output.
+ --help Show this message and exit.
+$ rio cogeo validate --help
+Usage: rio cogeo validate [OPTIONS] INPUT
+
+ Validate Cloud Optimized Geotiff.
+
+Options:
+ --strict Treat warnings as errors.
+ --config NAME=VALUE GDAL configuration options.
+ --help Show this message and exit.
+The strict options will treat warnings (e.g missing overviews) as errors.
Using the --config option can be useful to restrict GDAL environment. By default GDAL will check for external files (such as overviews), which could make a COG invalid. To force GDAL to only consider the input file you can use --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR.
e.g: +
$ rio cogeo validate s3://bucket/geo.tif --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR --config AWS_NO_SIGN_REQUEST=YES
+(extended version or rio info).
$ rio cogeo info --help
+Usage: rio cogeo info [OPTIONS] INPUT
+
+ Dataset info.
+
+Options:
+ --json Print as JSON.
+ --help Show this message and exit.
+# Create a COGEO with DEFLATE compression (Using default `Deflate` profile)
+$ rio cogeo create mydataset.tif mydataset_jpeg.tif
+
+# Validate COGEO
+$ rio cogeo validate mydataset_jpeg.tif
+
+# Create a COGEO with JPEG profile and the first 3 bands of the data and add internal mask
+$ rio cogeo create mydataset.tif mydataset_jpeg.tif -b 1,2,3 --add-mask --cog-profile jpeg
+
+# List Raster info
+$ rio cogeo info mydataset_jpeg.tif
+Driver: GTiff
+File: mydataset_jpeg.tif
+COG: True
+Compression: DEFLATE
+ColorSpace: None
+
+Profile
+ Width: 10980
+ Height: 10980
+ Bands: 1
+ Tiled: True
+ Dtype: uint16
+ NoData: 0.0
+ Alpha Band: False
+ Internal Mask: False
+ Interleave: BAND
+ Colormap: False
+
+Geo
+ Crs: EPSG:32634
+ Origin: (699960.0, 3600000.0)
+ Resolution: (10.0, -10.0)
+ BoundingBox: (699960.0, 3490200.0, 809760.0, 3600000.0)
+ MinZoom: 10
+ MaxZoom: 19
+
+IFD
+ Id Size BlockSize Decimation
+ 0 10980x10980 1024x1024 0
+ 1 5490x5490 128x128 2
+ 2 2745x2745 128x128 4
+ 3 1373x1373 128x128 8
+ 4 687x687 128x128 16
+Requirements
+$ pip install rio-cogeo
The COG Specification is pretty basic
+++A cloud optimized GeoTIFF is a regular GeoTIFF file, aimed at being hosted on a HTTP file server, whose internal organization is friendly for consumption by clients issuing HTTP GET range request ("bytes: start_offset-end_offset" HTTP header). +It contains at its beginning the metadata of the full resolution imagery, followed by the optional presence of overview metadata, and finally the imagery itself. To make it friendly with streaming and progressive rendering, we recommend starting with the imagery of the smallest overview and finishing with the imagery of the full resolution level.
+
Ref: github.com/cogeotiff/cog-spec/blob/master/spec.md
+In Short, the specification just means you MUST create a GeoTIFF with internal block (tile) and the header must be ordered.
+From a command line point of view, it just means you need to add --co TILED=TRUE in a gdal_translate command.
Natural Earth web site host really neat raster and vector datasets. Let's download a large scale raster image: www.naturalearthdata.com/downloads/50m-raster-data/50m-cross-blend-hypso/
+$ wget https://naciscdn.org/naturalearth/50m/raster/HYP_50M_SR.zip
+Here is what we want to look at:
+$ rio cogeo info HYP_50M_SR.tif
+Driver: GTiff
+File: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR.tif
+Compression: None
+ColorSpace: None
+
+Profile
+ Width: 10800
+ Height: 5400
+ Bands: 3
+ Tiled: False
+ Dtype: uint8
+ NoData: None
+ Alpha Band: False
+ Internal Mask: False
+ Interleave: PIXEL
+ ColorMap: False
+
+Geo
+ Crs: EPSG:4326
+ Origin: (-179.99999999999997, 90.0)
+ Resolution: (0.03333333333333, -0.03333333333333)
+ BoundingBox: (-179.99999999999997, -89.99999999998201, 179.99999999996405, 90.0)
+
+IFD
+ Id Size BlockSize Decimation
+ 0 10800x5400 10800x1 0
+What we can see from the rio cogeo info output:
+Tiled: false and BlockSize=10800x1)With those informations we already know the GeoTIFF is not a COG (no internal blocks), but let's confirm with the validation script.
+$ rio cogeo validate HYP_50M_SR.tif
+The following warnings were found:
+- The file is greater than 512xH or 512xW, it is recommended to include internal overviews
+
+The following errors were found:
+- The file is greater than 512xH or 512xW, but is not tiled
+- The offset of the main IFD should be 8 for ClassicTIFF or 16 for BigTIFF. It is 174982088 instead
+- The offset of the first block of the image should be after its IFD
+/Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR.tif is NOT a valid cloud optimized GeoTIFF
+As mentioned earlier, the validation script confirms the GeoTIFF is not internally tiled and doesn't have overviews.
+Creating a valid Cloud Optimized GeoTIFF, is not just about creating internal tiles and/or internal overviews. The file internal structure has to be specific and require a complete copy of a file, which is what rio-cogeo does internally.
+$ rio cogeo create HYP_50M_SR.tif HYP_50M_SR_COG.tif
+Reading input: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR.tif
+ [####################################] 100%
+Adding overviews...
+Updating dataset tags...
+Writing output to: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR_COG.tif
+You could get the same COG with GDAL commands
+$ gdal_translate HYP_50M_SR.tif tmp.tif -co TILED=YES -co COMPRESS=DEFLATE
+$ gdaladdo -r nearest tmp.tif 2 4 8 16 32
+$ gdal_translate tmp.tif HYP_50M_SR_COG.tif -co TILED=YES -co COMPRESS=DEFLATE -co COPY_SRC_OVERVIEWS=YES
+By default rio-cogeo will create a COG with 512x512 blocksize (for the raw resolution) and use DEFLATE compression to reduce file size.
$ rio cogeo info HYP_50M_SR_COG.tif
+Driver: GTiff
+File: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR_COG.tif
+Compression: DEFLATE
+ColorSpace: None
+
+Profile
+ Width: 10800
+ Height: 5400
+ Bands: 3
+ Tiled: True
+ Dtype: uint8
+ NoData: None
+ Alpha Band: False
+ Internal Mask: False
+ Interleave: PIXEL
+ ColorMap: False
+
+Geo
+ Crs: EPSG:4326
+ Origin: (-179.99999999999997, 90.0)
+ Resolution: (0.03333333333333001, -0.03333333333333001)
+ BoundingBox: (-179.99999999999997, -89.99999999998204, 179.9999999999641, 90.0)
+
+IFD
+ Id Size BlockSize Decimation
+ 0 10800x5400 512x512 0
+ 1 5400x2700 128x128 2
+ 2 2700x1350 128x128 4
+ 3 1350x675 128x128 8
+ 4 675x338 128x128 16
+The importance of the compression
+$ ls -lah
+-rw-r--r--@ 1 youpi staff 167M Oct 18 2014 HYP_50M_SR.tif
+-rw-r--r-- 1 youpi staff 58M Jun 12 14:56 HYP_50M_SR_COG.tif
+By using rio-cogeo, we are not only creating a valid COG with internal tiling but we are also adding internal overviews (which let us get previews of the raw resolution with few GET requests).
Even with the addition of 4 levels of overviews (see IFD section in previous rio cogeo info output), we managed to reduce the file size by 3 (167Mb → 58Mb), and this is because rio cogeo applies Deflate compression by default to the COG.
More Magic ?
+As seen in the first rio cogeo info output, the data has 3 bands (RGB) and is of Uint8 data type. Because of this configuration, we can use even more efficient compression like JPEG or WEBP.
$ rio cogeo create HYP_50M_SR.tif HYP_50M_SR_COG_jpeg.tif -p jpeg
+Reading input: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR.tif
+ [####################################] 100%
+Adding overviews...
+Updating dataset tags...
+Writing output to: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR_COG_jpeg.tif
+
+$ ls -lah
+-rw-r--r--@ 1 vincentsarago staff 167M Oct 18 2014 HYP_50M_SR.tif
+-rw-r--r-- 1 vincentsarago staff 58M Jun 12 14:56 HYP_50M_SR_COG.tif
+-rw-r--r-- 1 vincentsarago staff 4.8M Jun 15 11:08 HYP_50M_SR_COG_jpeg.tif
+Now, our output file is only 4.8Mb, which is only ~3% of the original size 😱.
+Note:
+You can either load the COG in QGIS or use our plugin (rio-viz) to load it in a web browser.
+$ pip install rio-viz
+$ rio viz HYP_50M_SR_COG.tif
+
Issues and pull requests are more than welcome.
+dev install
+$ git clone https://github.com/cogeotiff/rio-cogeo.git
+$ cd rio-cogeo
+$ pip install -e .["test","dev"]
+python -m pytest --cov rio_cogeo --cov-report term-missing
+This repo is set to use pre-commit to run isort, flake8, pydocstring, black ("uncompromising Python code formatter") and mypy when committing new code.
$ pre-commit install
+$ git clone https://github.com/cogeotiff/rio-cogeo.git
+$ cd rio-cogeo
+$ pip install -e .["docs"]
+Hot-reloading docs:
+$ mkdocs serve
+To manually deploy docs (note you should never need to do this because Github +Actions deploys automatically for new commits.):
+$ mkdocs gh-deploy
+
+ 
+
+ Cloud Optimized GeoTIFF (COG) creation and validation plugin for Rasterio. +
+
+
+ 
+
+
+ 
+
+
+ 
+
+
+ 
+
+
+ 
+
+
Documentation: cogeotiff.github.io/rio-cogeo/
+Source Code: cogeotiff/rio-cogeo
+This plugin aims to facilitate the creation and validation of Cloud Optimized +GeoTIFF (COG or COGEO). While it respects the +COG specifications, this plugin also +enforces several features:
+--overview-level 0)Important: in GDAL 3.1 a new COG driver has been added (doc, discussion), starting with rio-cogeo version 2.2, --use-cog-driver option was added to create COG using the COG driver.
$ pip install -U pip
+$ pip install rio-cogeo
+Or install from source:
+$ pip install -U pip
+$ pip install git+https://github.com/cogeotiff/rio-cogeo.git
+It is recommended to use GDAL > 2.3.2. Previous versions might not be able to +create proper COGs (ref: OSGeo/gdal#754).
+More info in cogeotiff/rio-cogeo#55
+Blog post on good and bad COG formats: medium.com/@_VincentS_/do-you-really-want-people-using-your-data-ec94cd94dc3f
+Checkout rio-glui or rio-viz rasterio plugins to explore COG locally in your web browser.
+See CONTRIBUTING.md
+See CHANGES.md.
+See LICENSE
+ + + + + + +rio-cogeo defines default Cloud Optimized GeoTIFF profiles which user can use with the CLI or directly with the API.
+Default profiles are tiled (tiled=True) with 512x512 blocksizes.
from rio_cogeo.profiles import cog_profiles
+
+cog_profiles
+> {
+ 'jpeg': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'JPEG', 'photometric': 'YCbCr'},
+ 'webp': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'WEBP'},
+ 'zstd': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'ZSTD'},
+ 'lzw': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LZW'},
+ 'deflate': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'DEFLATE'}
+ 'packbits': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'PACKBITS'},
+ 'lzma': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LZMA'},
+ 'lerc': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LERC'},
+ 'lerc_deflate': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LERC_DEFLATE'},
+ 'lerc_zstd': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LERC_ZSTD'},
+ 'raw': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512}
+}
+JPEG
+WEBP
+ZSTD
+Note in Nov 2018, there was a change in libtiff's ZSTD tags which create incompatibility for old ZSTD compressed GeoTIFF (link)
+LZW
+DEFLATE
+PACKBITS
+LZMA
+LERC
+LERC_DEFLATE
+LERC_ZSTD
+RAW
+Profiles can be extended by providing '--co' option in command line
+# Create a COGEO without compression and with 1024x1024 block size and 256 overview blocksize
+$ rio cogeo create mydataset.tif mydataset_raw.tif --co BLOCKXSIZE=1024 --co BLOCKYSIZE=1024 --cog-profile raw --overview-blocksize 256
+See gdal.org/drivers/raster/gtiff.html#creation-options for full details of creation options.
+ + + + + + +morecantile requirement to >=5.0,<6.0 (author @mentaljam, cogeotiff/rio-cogeo!267)pydantic requirement to ~=2.0 (author @mentaljam, cogeotiff/rio-cogeo!267)pydantic deprecation warnings (author @mentaljam, cogeotiff/rio-cogeo!267)BaseModel.json with BaseModel.model_dump_jsonBaseModel.dict with BaseModel.model_dumpclass Config with model_config class variables>=4.0.0--tms option to specify a path to a TileMatrixSet JSON filebreaking change
+Web optimization is now done in rio-cogeo instead of GDAL, when using --web-optimized and --use-cog-driver options
switch from using TILING_SCHEME namespaced tags to simple TILING_SCHEME_ prefixed metadata
# before
+with rasterio.open("cog_web.tif") as src:
+ print(src.tags(ns="TILING_SCHEME"))
+>>> {
+ "NAME": "WebMercatorQuad",
+ "ZOOM_LEVEL": "18",
+}
+
+# now
+with rasterio.open("cog_web.tif") as src:
+ print(src.tags())
+>>> {
+ "TILING_SCHEME_NAME": "WebMercatorQuad",
+ "TILING_SCHEME_ZOOM_LEVEL": "18",
+}
+Breaking Changes
+height or width) and be tiled even if they are smaller than 512x512# before
+rio cogeo create image_51x51.tif cog.tif
+rio cogeo info cog.tif --json | jq '.IFD'
+>>> [
+ {
+ "Level": 0,
+ "Width": 51,
+ "Height": 51,
+ "Blocksize": [
+ 51,
+ 51
+ ],
+ "Decimation": 0
+ }
+]
+rio cogeo info cog.tif --json | jq '.Profile.Tiled'
+>>> false
+
+# now
+rio cogeo create image_51x51.tif cog.tif
+rio cogeo info cog.tif --json | jq '.IFD'
+>>> [
+ {
+ "Level": 0,
+ "Width": 51,
+ "Height": 51,
+ "Blocksize": [
+ 512,
+ 512
+ ],
+ "Decimation": 0
+ }
+]
+rio cogeo info cog.tif --json | jq '.Profile.Tiled'
+>>> true
+cog_info() when previously passing *kwargs to cog_validateinfo methods (previously GDAL's configs were only use in the COG validation step)cog_validateconfig option in rio cogeo info CLIpyproject.toml (cogeotiff/rio-cogeo!232)--zoom-level option to define dataset coarsest zoom level, when creating web optimized COG.aligned_levelsBreaking Changes:
+>=3.1,<4.0. WebOptimized COGs will is now aligned with GDAL and Mercantile TMS definition.rasterio.vrt.WarpedVRT in utils.get_web_optimized_params to better handle dataset with GEOS projection (crossing dateline).rasterio.path (cogeotiff/rio-cogeo!222)cog_info output and update rio_cogeo.models.Info (cogeotiff/rio-cogeo!223)test and dev dependencies to rio-tiler>=3.0.0a0morecantile>=3.0click version requirement to >=7.0 to make sure click.Choice supports the case_sensitive option.cog_validate (cogeotiff/rio-cogeo!206)from rio_cogeo import cog_validate
+
+assert cog_validate("cog.tif", congig={"GDAL_DISABLE_READDIR_ON_OPEN": "EMPTY_DIR"})[0]
+In previous version we were forcing GDAL_DISABLE_READDIR_ON_OPEN=FALSE in cog_validate function to check for external overviews.
Starting with version 2.3, it's up to the user to set the wanted GDAL configuration (e.g EMPTY_DIR: no external file check, FALSE: check for external files)
Info model (author @geospatial-jeff, cogeotiff/rio-cogeo!201)info output (cogeotiff/rio-cogeo#191)use_cog_driver option to create COG using new GDAL COG Driver (cogeotiff/rio-cogeo!194)Breaking Changes:
+rio_cogeo.cogeo.cog_info now returns a pydantic modelfrom rio_cogeo.cogeo import cog_info
+
+# before
+info = cog_info("my.tif")
+assert isinstance(info, dict)
+assert info["GEO"]["CRS"]
+
+# now
+assert isinstance(info, rio_cogeo.models.Info)
+assert info.GEO.CRS
+TILING SCHEME in dataset namespaced metadata when creating WebOptimized COG (cogeotiff/rio-cogeo!193)Tags (cogeotiff/rio-cogeo!193)# before
+$ rio cogeo create in.tif out.tif -w
+$ rio cogeo info out.tif | jq .Tags
+
+>>> {
+ "AREA_OR_POINT": "Area",
+ "OVR_RESAMPLING_ALG": "NEAREST"
+}
+
+# now
+$ rio cogeo create in.tif out.tif -w
+$ rio cogeo info out.tif | jq .Tags
+>> {
+ "Image Metadata": {
+ "AREA_OR_POINT": "Area",
+ "DataType": "Generic",
+ "OVR_RESAMPLING_ALG": "NEAREST"
+ },
+ "Image Structure": {
+ "COMPRESSION": "DEFLATE",
+ "INTERLEAVE": "BAND",
+ "LAYOUT": "COG"
+ },
+ "Tiling Scheme": {
+ "NAME": "WEBMERCATORQUAD",
+ "ZOOM_LEVEL": "17"
+ }
+}
+Web-Optimized configuration to match GDAL COG Driver (cogeotiff/rio-cogeo!193)By default only the raw data will be aligned to the grid. To align overviews, the aligned_levels option can be used (wasn't really working in previous version).
rio_cogeo.utils.get_web_optimized_params has been refactored (cogeotiff/rio-cogeo!193)
cog_translate will now materialize Nodata or Alpha band to an internal mask automatically for JPEG compresssed output (cogeotiff/rio-cogeo!196)
# before
+cog_translate(raster_path_rgba, "cogeo.tif", jpeg_profile)
+with rasterio.open("cogeo.tif") as src:
+ assert src.count == 4
+ assert src.compression.value == "JPEG"
+ assert has_alpha_band(src)
+ assert not has_mask_band(src)
+
+# now
+cog_translate(raster_path_rgba, "cogeo.tif", jpeg_profile)
+with rasterio.open("cogeo.tif") as src:
+ assert src.count == 3
+ assert src.compression.value == "JPEG"
+ assert has_mask_band(src)
+cog_translate to allow a user to set or update a colormapcmap = {0: (0, 0, 0, 0), 1: (1, 2, 3, 255)}
+cog_translate("boring.tif", "cogeo.tif", deflate_profile, colormap=cmap)
+with rasterio.open("cogeo.tif") as cog:
+ print(cog.colormap(1)[1])
+
+>>> (1, 2, 3, 255)
+cog_translate to allow the user to add more dataset metadatascog_translate("boring.tif", "cogeo.tif", deflate_profile, additional_cog_metadata={"comments": "I made this tiff with rio-cogeo"})
+
+with rasterio.open("cogeo.tif") as cog:
+ print(cog.tags()["comment"])
+
+>>> "I made this tiff with rio-cogeo"
+# before
+from rio_cogeo.cogeo import cog_translate, cog_validate, cog_info
+from rio_cogeo.profiles import cog_profiles
+
+# now
+from rio_cogeo import cog_translate, cog_validate, cog_info, cog_profiles
+morecantile and update the web-optimized creation method to better match GDAL 3.2.zoom_level_strategy options to match GDAL 3.2 COG driver.aligned_levels (cli and api) to select the level of overview to align with the TMS grid.Breaking Changes:
+* removed --latitude-adjustment/--global-maxzoom option in the CLI
+* removed latitude_adjustment option in rio_cogeo.cogeo.cog_translate
+* updated overview blocksize to match the blocksize of the high resolution data (instead of default to 128)
+* for web-optimized COG, the highest overview level will be aligned with the TMS grid.
pkg_resources (pypa/setuptools#510)There have been no changes since 2.0a9
+utils.has_mask_band (#156)rio_cogeo.cogeo.cog_infocount information in rio_cogeo.cogeo.cog_info output (#150)rio_cogeo.cogeo.cog_infoBreaking Changes: +* rio_cogeo.utils.get_max_zoom renamed rio_cogeo.utils.get_zooms and now return min/max zoom.
+info CLI (#134)Deflate as default temporary compression (#137)--config CLI option to pass additional GDAL Configuration options (#135)--blocksize option in CLI (#131)rio_cogeo.utils.get_maximum_overview_level and use rasterio.rio.overview.get_maximum_overview_level (#132)dataset_mask returned by rasterio to uint8 to overcome a bug in rasterio 1.1.2 (#115)strict option to cog_validate to treat warnings as error (#109) * Thanks @pierotofyPHOTOMETRIC=MINISBLACK when PHOTOMETRIC is set to YCBCR for 1 band dataset (#41)-forward-band-tags options (#115)--allow-intermediate-compression option to reduce the memory/disk footprint (#103)non-standard compression (#97)CLI Changes:
+* add -t, --dtype datatype option.
API Changes: +* add datatype option +* update for rasterio>=1.0.28 +* allow rasterio.io.DatasetReader input (#89)
+Note: This release was deleted in PyPi.
+--web-optimized option to create a web optimized COG (#10)--latitude-adjustment/--global-maxzoom option to adjust MAX_ZOOM for global datasets--resampling option to select the resampling algorithm when using --web-optimized--in-memory/--no-in-memory options to use temporyNamedd file instead of in-memory temp file.Breaking Changes:
+Breaking Changes:
+Breaking Changes:
+--quiet option (#46)--overview-blocksize to set overview's internal tile size (#60)Bug fixes:
+Breaking Changes:
+Breaking Changes: +* replace "BAND" by "PIXEL" interleave in PACKBITS profile (#16)
+Cloud Optimized GeoTIFF (COG) creation and validation plugin for Rasterio.
Documentation: cogeotiff.github.io/rio-cogeo/
Source Code: cogeotiff/rio-cogeo
"},{"location":"#cloud-optimized-geotiff","title":"Cloud Optimized GeoTIFF","text":"This plugin aims to facilitate the creation and validation of Cloud Optimized GeoTIFF (COG or COGEO). While it respects the COG specifications, this plugin also enforces several features:
--overview-level 0)Important: in GDAL 3.1 a new COG driver has been added (doc, discussion), starting with rio-cogeo version 2.2, --use-cog-driver option was added to create COG using the COG driver.
$ pip install -U pip\n$ pip install rio-cogeo\nOr install from source:
$ pip install -U pip\n$ pip install git+https://github.com/cogeotiff/rio-cogeo.git\nIt is recommended to use GDAL > 2.3.2. Previous versions might not be able to create proper COGs (ref: OSGeo/gdal#754).
More info in cogeotiff/rio-cogeo#55
"},{"location":"#more","title":"More","text":"Blog post on good and bad COG formats: medium.com/@_VincentS_/do-you-really-want-people-using-your-data-ec94cd94dc3f
Checkout rio-glui or rio-viz rasterio plugins to explore COG locally in your web browser.
"},{"location":"#contribution-development","title":"Contribution & Development","text":"See CONTRIBUTING.md
"},{"location":"#changes","title":"Changes","text":"See CHANGES.md.
"},{"location":"#license","title":"License","text":"See LICENSE
"},{"location":"API/","title":"Application programming interface (API)","text":"rio-cogeo can also be integrated directly in your custom script. See rio_cogeo.cogeo.cog_translate function.
e.g:
from rio_cogeo.cogeo import cog_translate\n\ndef _translate(src_path, dst_path, profile=\"webp\", profile_options={}, **options):\n\"\"\"Convert image to COG.\"\"\"\n # Format creation option (see gdalwarp `-co` option)\n output_profile = cog_profiles.get(profile)\n output_profile.update(dict(BIGTIFF=\"IF_SAFER\"))\n output_profile.update(profile_options)\n\n # Dataset Open option (see gdalwarp `-oo` option)\n config = dict(\n GDAL_NUM_THREADS=\"ALL_CPUS\",\n GDAL_TIFF_INTERNAL_MASK=True,\n GDAL_TIFF_OVR_BLOCKSIZE=\"128\",\n )\n\n cog_translate(\n src_path,\n dst_path,\n output_profile,\n config=config,\n in_memory=False,\n quiet=True,\n **options,\n )\n return True\nimport numpy\n\nimport mercantile\n\nfrom rasterio.io import MemoryFile\nfrom rasterio.transform import from_bounds\n\nfrom rio_cogeo.cogeo import cog_translate\nfrom rio_cogeo.profiles import cog_profiles\n\n# Create GeoTIFF profile\nbounds = mercantile.bounds(mercantile.Tile(0,0,0))\n\n# Rasterio uses numpy array of shape of `(bands, height, width)`\nwidth = 1024\nheight = 1024\nnbands = 3\n\nimg_array = tile = numpy.random.rand(nbands, height, width).astype(numpy.float32)\n\nsrc_transform = from_bounds(*bounds, width=width, height=height)\n\nsrc_profile = dict(\n driver=\"GTiff\",\n dtype=\"float32\",\n count=nbands,\n height=height,\n width=width,\n crs=\"epsg:4326\",\n transform=src_transform,\n)\n\n\nwith MemoryFile() as memfile:\n with memfile.open(**src_profile) as mem:\n # Populate the input file with numpy array\n mem.write(img_array)\n\n dst_profile = cog_profiles.get(\"deflate\")\n cog_translate(\n mem,\n \"my-output-cog.tif\",\n dst_profile,\n in_memory=True,\n quiet=True,\n )\nfrom rasterio.io import MemoryFile\n\nfrom rio_cogeo.cogeo import cog_translate\nfrom rio_cogeo.profiles import cog_profiles\n\nfrom boto3.session import Session as boto3_session\n\ndst_profile = cog_profiles.get(\"deflate\")\n\nwith MemoryFile() as mem_dst:\n # Important, we pass `mem_dst.name` as output dataset path\n cog_translate(\"my-input-file.tif\", mem_dst.name, dst_profile, in_memory=True)\n\n # You can then use the memoryfile to do something else like\n # upload to AWS S3\n client = boto3_session.client(\"s3\")\n client.upload_fileobj(mem_dst, \"my-bucket\", \"my-key\")\nrio-cogeo provide a --web-optimized option which aims to create a web-tiling friendly COG.
Output dataset features:
Important
Because it will certainly create a larger file (with padding tiles on the side of the file), a nodata value, an alpha band or an internal mask should be present in the input dataset. If not the original data will be surrounded by black (0) data.
"},{"location":"Advanced/#internal-tile-size","title":"Internal tile size","text":"By default rio cogeo will create a dataset with 512x512 internal tile size. This can be updated by passing --co BLOCKXSIZE=64 --co BLOCKYSIZE=64 options.
Web tiling optimization
Creating a Web-Optimized COG, means you'll get a file which is perfectly aligned (bounds and internal tiles) with the mercator grid and with resolution (for the raw data and overview) which map the mercator zoom level resolution. This enable to reduce the number of GET request a dynamic tiling service needs to do to create a map tile from your COG.
if the input dataset is not aligned to web mercator grid, the tiler will need to fetch multiple internal tiles.
"},{"location":"Advanced/#overview-levels","title":"Overview levels","text":"By default rio cogeo will calculate the optimal overview level based on dataset size and internal tile size (overview should not be smaller than internal tile size (e.g 512px). Overview level will be translated to decimation level of power of two:
overview_level = 3\noverviews = [2 ** j for j in range(1, overview_level + 1)]\nprint(overviews)\n[2, 4, 8]\nBy default rio cogeo DO NOT forward band metadata (e.g statistics) to the output dataset.
$ gdalinfo my_file.tif\n...\nBand 1 Block=576x1 Type=Float64, ColorInterp=Gray\n NoData Value=999999986991104\n Unit Type: mol mol-1\n Metadata:\n long_name=CO2 Dry-Air Column Average\n missing_value=9.9999999e+14\n NETCDF_DIM_time=0\n NETCDF_VARNAME=XCO2MEAN\n units=mol mol-1\n _FillValue=9.9999999e+14\n\n$ rio cogeo my_file.tif my_cog.tif --blocksize 256\n\n$ gdalinfo my_cog.tif\n...\nBand 1 Block=256x256 Type=Float64, ColorInterp=Gray\n NoData Value=999999986991104\n Overviews: 288x181\nYou can use --forward-band-tags to forwards the band metadata to output dataset.
$ rio cogeo create my_file.tif my_cog.tif --blocksize 256 --forward-band-tags\n$ gdalinfo my_cog.tif\n...\nBand 1 Block=256x256 Type=Float64, ColorInterp=Gray\n NoData Value=999999986991104\n Overviews: 288x181\n Metadata:\n long_name=CO2 Dry-Air Column Average\n missing_value=9.9999999e+14\n NETCDF_DIM_time=0\n NETCDF_VARNAME=XCO2MEAN\n units=mol mol-1\n _FillValue=9.9999999e+14\nBy default rio-cogeo will forward any nodata value or alpha channel to the output COG.
If your dataset type is Byte or Unit16, you could use internal bit mask (with the --add-mask option) to replace the Nodata value or Alpha band in output dataset (supported by most GDAL based backends).
Note: when adding a mask with an input dataset having an alpha band you'll need to use the bidx options to remove it from the output dataset.
# Replace the alpha band by an internal mask\n$ rio cogeo mydataset_withalpha.tif mydataset_withmask.tif --cog-profile raw --add-mask --bidx 1,2,3\nImportant
Using internal nodata value with lossy compression (webp, jpeg) is not recommended. Please use internal masking (or alpha band if using webp).
rio-cogeo modules adds a cogeo sub-command to your rasterio (rio) CLI. Three (3) commands are then available: create, info and validate.
$ rio cogeo --help\n Usage: rio cogeo [OPTIONS] COMMAND [ARGS]...\n\n Rasterio cogeo subcommands.\n\n Options:\n --version Show the version and exit.\n --help Show this message and exit.\n\n Commands:\n create Create COGEO\n info Lists information about a raster dataset.\n validate Validate COGEO\n$ rio cogeo create --help\n Usage: rio cogeo create [OPTIONS] INPUT OUTPUT\n\n Create Cloud Optimized Geotiff.\n\n Options:\n --bidx, -b Band indexes to copy.\n --cog-profile, -p CloudOptimized GeoTIFF profile (default: deflate). [jpeg|webp|zstd|lzw|deflate|packbits|lzma|lerc|lerc_deflate|lerc_zstd|raw]\n --nodata Set nodata masking values for input dataset.\n --add-mask Force output dataset creation with an internal mask (convert alpha band or nodata to mask).\n --blocksize Overwrite profile's tile size.\n --dtype, -t Output data type. [ubyte|uint8|uint16|int16|uint32|int32|float32|float64]\n --overview-level Overview level (if not provided, appropriate overview level will be selected until the smallest overview is smaller than the value of the internal blocksize)\n --overview-resampling Overview creation resampling algorithm (default: nearest). [nearest|bilinear|cubic|cubic_spline|lanczos|average|mode|gauss]\n --overview-blocksize Overview's internal tile size (default defined by GDAL_TIFF_OVR_BLOCKSIZE env or 128)\n --web-optimized, -w Create COGEO optimized for Web.\n --zoom-level-strategy Strategy to determine zoom level (default: auto). [lower|upper|auto]\n --zoom-level Zoom level number for the highest resolution. If this option is specified, `--zoom-level-strategy` is ignored.\n --aligned-levels Number of overview levels for which GeoTIFF tile and tiles defined in the tiling scheme match.\n --resampling, -r Resampling algorithm (default: nearest). Will only be applied with the `--web-optimized` option. [nearest|bilinear|cubic|cubic_spline|lanczos|average|mode|max|min|med|q1|q3|sum]\n --in-memory / --no-in-memory Force processing raster in memory / not in memory (default: process in memory if smaller than 120 million pixels)\n --allow-intermediate-compression Allow intermediate file compression to reduce memory/disk footprint.\n --forward-band-tags Forward band tags to output bands.\n --forward-ns-tags Forward namespaced tags to output dataset.\n --threads Number of worker threads for multi-threaded compression (default: ALL_CPUS)\n --use-cog-driver Use GDAL COG Driver (require GDAL>=3.1).\n --tms PATH Path to TileMatrixSet JSON file.\n --co, --profile Driver specific creation options. See the documentation for the selected output driver for more information.\n --config GDAL configuration options.\n --quiet, -q Remove progressbar and other non-error output.\n --help Show this message and exit.\n$ rio cogeo validate --help\nUsage: rio cogeo validate [OPTIONS] INPUT\n\n Validate Cloud Optimized Geotiff.\n\nOptions:\n --strict Treat warnings as errors.\n --config NAME=VALUE GDAL configuration options.\n --help Show this message and exit.\nThe strict options will treat warnings (e.g missing overviews) as errors.
Using the --config option can be useful to restrict GDAL environment. By default GDAL will check for external files (such as overviews), which could make a COG invalid. To force GDAL to only consider the input file you can use --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR.
e.g:
$ rio cogeo validate s3://bucket/geo.tif --config GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR --config AWS_NO_SIGN_REQUEST=YES\n(extended version or rio info).
$ rio cogeo info --help\nUsage: rio cogeo info [OPTIONS] INPUT\n\n Dataset info.\n\nOptions:\n --json Print as JSON.\n --help Show this message and exit.\n# Create a COGEO with DEFLATE compression (Using default `Deflate` profile)\n$ rio cogeo create mydataset.tif mydataset_jpeg.tif\n\n# Validate COGEO\n$ rio cogeo validate mydataset_jpeg.tif\n\n# Create a COGEO with JPEG profile and the first 3 bands of the data and add internal mask\n$ rio cogeo create mydataset.tif mydataset_jpeg.tif -b 1,2,3 --add-mask --cog-profile jpeg\n\n# List Raster info\n$ rio cogeo info mydataset_jpeg.tif\nDriver: GTiff\nFile: mydataset_jpeg.tif\nCOG: True\nCompression: DEFLATE\nColorSpace: None\n\nProfile\n Width: 10980\nHeight: 10980\nBands: 1\nTiled: True\n Dtype: uint16\n NoData: 0.0\n Alpha Band: False\n Internal Mask: False\n Interleave: BAND\n Colormap: False\n\nGeo\n Crs: EPSG:32634\n Origin: (699960.0, 3600000.0)\nResolution: (10.0, -10.0)\nBoundingBox: (699960.0, 3490200.0, 809760.0, 3600000.0)\nMinZoom: 10\nMaxZoom: 19\n\nIFD\n Id Size BlockSize Decimation\n 0 10980x10980 1024x1024 0\n1 5490x5490 128x128 2\n2 2745x2745 128x128 4\n3 1373x1373 128x128 8\n4 687x687 128x128 16\nRequirements
$ pip install rio-cogeo
The COG Specification is pretty basic
A cloud optimized GeoTIFF is a regular GeoTIFF file, aimed at being hosted on a HTTP file server, whose internal organization is friendly for consumption by clients issuing HTTP GET range request (\"bytes: start_offset-end_offset\" HTTP header). It contains at its beginning the metadata of the full resolution imagery, followed by the optional presence of overview metadata, and finally the imagery itself. To make it friendly with streaming and progressive rendering, we recommend starting with the imagery of the smallest overview and finishing with the imagery of the full resolution level.
Ref: github.com/cogeotiff/cog-spec/blob/master/spec.md
In Short, the specification just means you MUST create a GeoTIFF with internal block (tile) and the header must be ordered.
From a command line point of view, it just means you need to add --co TILED=TRUE in a gdal_translate command.
Natural Earth web site host really neat raster and vector datasets. Let's download a large scale raster image: www.naturalearthdata.com/downloads/50m-raster-data/50m-cross-blend-hypso/
$ wget https://naciscdn.org/naturalearth/50m/raster/HYP_50M_SR.zip\nHere is what we want to look at:
$ rio cogeo info HYP_50M_SR.tif\nDriver: GTiff\nFile: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR.tif\nCompression: None\nColorSpace: None\n\nProfile\n Width: 10800\n Height: 5400\n Bands: 3\n Tiled: False\n Dtype: uint8\n NoData: None\n Alpha Band: False\n Internal Mask: False\n Interleave: PIXEL\n ColorMap: False\n\nGeo\n Crs: EPSG:4326\n Origin: (-179.99999999999997, 90.0)\n Resolution: (0.03333333333333, -0.03333333333333)\n BoundingBox: (-179.99999999999997, -89.99999999998201, 179.99999999996405, 90.0)\n\nIFD\n Id Size BlockSize Decimation\n 0 10800x5400 10800x1 0\nWhat we can see from the rio cogeo info output:
Tiled: false and BlockSize=10800x1)With those informations we already know the GeoTIFF is not a COG (no internal blocks), but let's confirm with the validation script.
"},{"location":"Is_it_a_COG/#3-cog-validation","title":"3. COG validation","text":"$ rio cogeo validate HYP_50M_SR.tif\nThe following warnings were found:\n- The file is greater than 512xH or 512xW, it is recommended to include internal overviews\n\nThe following errors were found:\n- The file is greater than 512xH or 512xW, but is not tiled\n- The offset of the main IFD should be 8 for ClassicTIFF or 16 for BigTIFF. It is 174982088 instead\n- The offset of the first block of the image should be after its IFD\n/Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR.tif is NOT a valid cloud optimized GeoTIFF\nAs mentioned earlier, the validation script confirms the GeoTIFF is not internally tiled and doesn't have overviews.
"},{"location":"Is_it_a_COG/#4-cog-creation","title":"4. COG creation","text":"Creating a valid Cloud Optimized GeoTIFF, is not just about creating internal tiles and/or internal overviews. The file internal structure has to be specific and require a complete copy of a file, which is what rio-cogeo does internally.
$ rio cogeo create HYP_50M_SR.tif HYP_50M_SR_COG.tif\nReading input: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR.tif\n [####################################] 100%\nAdding overviews...\nUpdating dataset tags...\nWriting output to: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR_COG.tif\nYou could get the same COG with GDAL commands
$ gdal_translate HYP_50M_SR.tif tmp.tif -co TILED=YES -co COMPRESS=DEFLATE\n$ gdaladdo -r nearest tmp.tif 2 4 8 16 32\n$ gdal_translate tmp.tif HYP_50M_SR_COG.tif -co TILED=YES -co COMPRESS=DEFLATE -co COPY_SRC_OVERVIEWS=YES\nBy default rio-cogeo will create a COG with 512x512 blocksize (for the raw resolution) and use DEFLATE compression to reduce file size.
$ rio cogeo info HYP_50M_SR_COG.tif\nDriver: GTiff\nFile: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR_COG.tif\nCompression: DEFLATE\nColorSpace: None\n\nProfile\n Width: 10800\n Height: 5400\n Bands: 3\n Tiled: True\n Dtype: uint8\n NoData: None\n Alpha Band: False\n Internal Mask: False\n Interleave: PIXEL\n ColorMap: False\n\nGeo\n Crs: EPSG:4326\n Origin: (-179.99999999999997, 90.0)\n Resolution: (0.03333333333333001, -0.03333333333333001)\n BoundingBox: (-179.99999999999997, -89.99999999998204, 179.9999999999641, 90.0)\n\nIFD\n Id Size BlockSize Decimation\n 0 10800x5400 512x512 0\n 1 5400x2700 128x128 2\n 2 2700x1350 128x128 4\n 3 1350x675 128x128 8\n 4 675x338 128x128 16\nThe importance of the compression
$ ls -lah\n-rw-r--r--@ 1 youpi staff 167M Oct 18 2014 HYP_50M_SR.tif\n-rw-r--r-- 1 youpi staff 58M Jun 12 14:56 HYP_50M_SR_COG.tif\nBy using rio-cogeo, we are not only creating a valid COG with internal tiling but we are also adding internal overviews (which let us get previews of the raw resolution with few GET requests).
Even with the addition of 4 levels of overviews (see IFD section in previous rio cogeo info output), we managed to reduce the file size by 3 (167Mb \u2192 58Mb), and this is because rio cogeo applies Deflate compression by default to the COG.
More Magic ?
As seen in the first rio cogeo info output, the data has 3 bands (RGB) and is of Uint8 data type. Because of this configuration, we can use even more efficient compression like JPEG or WEBP.
$ rio cogeo create HYP_50M_SR.tif HYP_50M_SR_COG_jpeg.tif -p jpeg\nReading input: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR.tif\n [####################################] 100%\nAdding overviews...\nUpdating dataset tags...\nWriting output to: /Users/vincentsarago/Downloads/HYP_50M_SR/HYP_50M_SR_COG_jpeg.tif\n\n$ ls -lah\n-rw-r--r--@ 1 vincentsarago staff 167M Oct 18 2014 HYP_50M_SR.tif\n-rw-r--r-- 1 vincentsarago staff 58M Jun 12 14:56 HYP_50M_SR_COG.tif\n-rw-r--r-- 1 vincentsarago staff 4.8M Jun 15 11:08 HYP_50M_SR_COG_jpeg.tif\nNow, our output file is only 4.8Mb, which is only ~3% of the original size \ud83d\ude31.
Note:
You can either load the COG in QGIS or use our plugin (rio-viz) to load it in a web browser.
$ pip install rio-viz\n$ rio viz HYP_50M_SR_COG.tif\nIssues and pull requests are more than welcome.
dev install
$ git clone https://github.com/cogeotiff/rio-cogeo.git\n$ cd rio-cogeo\n$ pip install -e .[\"test\",\"dev\"]\npython -m pytest --cov rio_cogeo --cov-report term-missing\nThis repo is set to use pre-commit to run isort, flake8, pydocstring, black (\"uncompromising Python code formatter\") and mypy when committing new code.
$ pre-commit install\n$ git clone https://github.com/cogeotiff/rio-cogeo.git\n$ cd rio-cogeo\n$ pip install -e .[\"docs\"]\nHot-reloading docs:
$ mkdocs serve\nTo manually deploy docs (note you should never need to do this because Github Actions deploys automatically for new commits.):
$ mkdocs gh-deploy\nrio-cogeo defines default Cloud Optimized GeoTIFF profiles which user can use with the CLI or directly with the API.
Default profiles are tiled (tiled=True) with 512x512 blocksizes.
from rio_cogeo.profiles import cog_profiles\n\ncog_profiles\n> {\n 'jpeg': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'JPEG', 'photometric': 'YCbCr'},\n 'webp': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'WEBP'},\n 'zstd': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'ZSTD'},\n 'lzw': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LZW'},\n 'deflate': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'DEFLATE'}\n 'packbits': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'PACKBITS'},\n 'lzma': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LZMA'},\n 'lerc': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LERC'},\n 'lerc_deflate': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LERC_DEFLATE'},\n 'lerc_zstd': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512, 'compress': 'LERC_ZSTD'},\n 'raw': {'driver': 'GTiff', 'interleave': 'pixel', 'tiled': True, 'blockxsize': 512, 'blockysize': 512}\n}\nJPEG
WEBP
ZSTD
Note in Nov 2018, there was a change in libtiff's ZSTD tags which create incompatibility for old ZSTD compressed GeoTIFF (link)
LZW
DEFLATE
PACKBITS
LZMA
LERC
LERC_DEFLATE
LERC_ZSTD
RAW
Profiles can be extended by providing '--co' option in command line
# Create a COGEO without compression and with 1024x1024 block size and 256 overview blocksize\n$ rio cogeo create mydataset.tif mydataset_raw.tif --co BLOCKXSIZE=1024 --co BLOCKYSIZE=1024 --cog-profile raw --overview-blocksize 256\nSee gdal.org/drivers/raster/gtiff.html#creation-options for full details of creation options.
"},{"location":"release-notes/","title":"Release Notes","text":""},{"location":"release-notes/#500-2023-07-25","title":"5.0.0 (2023-07-25)","text":"morecantile requirement to >=5.0,<6.0 (author @mentaljam, cogeotiff/rio-cogeo!267)pydantic requirement to ~=2.0 (author @mentaljam, cogeotiff/rio-cogeo!267)pydantic deprecation warnings (author @mentaljam, cogeotiff/rio-cogeo!267)BaseModel.json with BaseModel.model_dump_jsonBaseModel.dict with BaseModel.model_dumpclass Config with model_config class variables>=4.0.0--tms option to specify a path to a TileMatrixSet JSON filebreaking change
Web optimization is now done in rio-cogeo instead of GDAL, when using --web-optimized and --use-cog-driver options
switch from using TILING_SCHEME namespaced tags to simple TILING_SCHEME_ prefixed metadata
# before\nwith rasterio.open(\"cog_web.tif\") as src:\n print(src.tags(ns=\"TILING_SCHEME\"))\n>>> {\n \"NAME\": \"WebMercatorQuad\",\n \"ZOOM_LEVEL\": \"18\",\n}\n\n# now\nwith rasterio.open(\"cog_web.tif\") as src:\n print(src.tags())\n>>> {\n \"TILING_SCHEME_NAME\": \"WebMercatorQuad\",\n \"TILING_SCHEME_ZOOM_LEVEL\": \"18\",\n}\nBreaking Changes
height or width) and be tiled even if they are smaller than 512x512# before\nrio cogeo create image_51x51.tif cog.tif\nrio cogeo info cog.tif --json | jq '.IFD'\n>>> [\n{\n\"Level\": 0,\n \"Width\": 51,\n \"Height\": 51,\n \"Blocksize\": [\n51,\n 51\n],\n \"Decimation\": 0\n}\n]\nrio cogeo info cog.tif --json | jq '.Profile.Tiled'\n>>> false\n\n# now\nrio cogeo create image_51x51.tif cog.tif\nrio cogeo info cog.tif --json | jq '.IFD'\n>>> [\n{\n\"Level\": 0,\n \"Width\": 51,\n \"Height\": 51,\n \"Blocksize\": [\n512,\n 512\n],\n \"Decimation\": 0\n}\n]\nrio cogeo info cog.tif --json | jq '.Profile.Tiled'\n>>> true\ncog_info() when previously passing *kwargs to cog_validateinfo methods (previously GDAL's configs were only use in the COG validation step)cog_validateconfig option in rio cogeo info CLIpyproject.toml (cogeotiff/rio-cogeo!232)--zoom-level option to define dataset coarsest zoom level, when creating web optimized COG.aligned_levelsBreaking Changes:
>=3.1,<4.0. WebOptimized COGs will is now aligned with GDAL and Mercantile TMS definition.rasterio.vrt.WarpedVRT in utils.get_web_optimized_params to better handle dataset with GEOS projection (crossing dateline).rasterio.path (cogeotiff/rio-cogeo!222)cog_info output and update rio_cogeo.models.Info (cogeotiff/rio-cogeo!223)test and dev dependencies to rio-tiler>=3.0.0a0morecantile>=3.0click version requirement to >=7.0 to make sure click.Choice supports the case_sensitive option.cog_validate (cogeotiff/rio-cogeo!206)from rio_cogeo import cog_validate\n\nassert cog_validate(\"cog.tif\", congig={\"GDAL_DISABLE_READDIR_ON_OPEN\": \"EMPTY_DIR\"})[0]\nIn previous version we were forcing GDAL_DISABLE_READDIR_ON_OPEN=FALSE in cog_validate function to check for external overviews.
Starting with version 2.3, it's up to the user to set the wanted GDAL configuration (e.g EMPTY_DIR: no external file check, FALSE: check for external files)
Info model (author @geospatial-jeff, cogeotiff/rio-cogeo!201)info output (cogeotiff/rio-cogeo#191)use_cog_driver option to create COG using new GDAL COG Driver (cogeotiff/rio-cogeo!194)Breaking Changes:
rio_cogeo.cogeo.cog_info now returns a pydantic modelfrom rio_cogeo.cogeo import cog_info\n\n# before\ninfo = cog_info(\"my.tif\")\nassert isinstance(info, dict)\nassert info[\"GEO\"][\"CRS\"]\n\n# now\nassert isinstance(info, rio_cogeo.models.Info)\nassert info.GEO.CRS\nTILING SCHEME in dataset namespaced metadata when creating WebOptimized COG (cogeotiff/rio-cogeo!193)Tags (cogeotiff/rio-cogeo!193)# before\n$ rio cogeo create in.tif out.tif -w\n$ rio cogeo info out.tif | jq .Tags\n\n>>> {\n \"AREA_OR_POINT\": \"Area\",\n \"OVR_RESAMPLING_ALG\": \"NEAREST\"\n}\n\n# now\n$ rio cogeo create in.tif out.tif -w\n$ rio cogeo info out.tif | jq .Tags\n>> {\n \"Image Metadata\": {\n \"AREA_OR_POINT\": \"Area\",\n \"DataType\": \"Generic\",\n \"OVR_RESAMPLING_ALG\": \"NEAREST\"\n },\n \"Image Structure\": {\n \"COMPRESSION\": \"DEFLATE\",\n \"INTERLEAVE\": \"BAND\",\n \"LAYOUT\": \"COG\"\n },\n \"Tiling Scheme\": {\n \"NAME\": \"WEBMERCATORQUAD\",\n \"ZOOM_LEVEL\": \"17\"\n }\n}\nWeb-Optimized configuration to match GDAL COG Driver (cogeotiff/rio-cogeo!193)By default only the raw data will be aligned to the grid. To align overviews, the aligned_levels option can be used (wasn't really working in previous version).
rio_cogeo.utils.get_web_optimized_params has been refactored (cogeotiff/rio-cogeo!193)
cog_translate will now materialize Nodata or Alpha band to an internal mask automatically for JPEG compresssed output (cogeotiff/rio-cogeo!196)
# before\ncog_translate(raster_path_rgba, \"cogeo.tif\", jpeg_profile)\nwith rasterio.open(\"cogeo.tif\") as src:\n assert src.count == 4\n assert src.compression.value == \"JPEG\"\n assert has_alpha_band(src)\n assert not has_mask_band(src)\n\n# now\ncog_translate(raster_path_rgba, \"cogeo.tif\", jpeg_profile)\nwith rasterio.open(\"cogeo.tif\") as src:\n assert src.count == 3\n assert src.compression.value == \"JPEG\"\n assert has_mask_band(src)\ncog_translate to allow a user to set or update a colormapcmap = {0: (0, 0, 0, 0), 1: (1, 2, 3, 255)}\ncog_translate(\"boring.tif\", \"cogeo.tif\", deflate_profile, colormap=cmap)\nwith rasterio.open(\"cogeo.tif\") as cog:\n print(cog.colormap(1)[1])\n\n>>> (1, 2, 3, 255)\ncog_translate to allow the user to add more dataset metadatascog_translate(\"boring.tif\", \"cogeo.tif\", deflate_profile, additional_cog_metadata={\"comments\": \"I made this tiff with rio-cogeo\"})\n\nwith rasterio.open(\"cogeo.tif\") as cog:\n print(cog.tags()[\"comment\"])\n\n>>> \"I made this tiff with rio-cogeo\"\n# before\nfrom rio_cogeo.cogeo import cog_translate, cog_validate, cog_info\nfrom rio_cogeo.profiles import cog_profiles\n\n# now\nfrom rio_cogeo import cog_translate, cog_validate, cog_info, cog_profiles\nmorecantile and update the web-optimized creation method to better match GDAL 3.2.zoom_level_strategy options to match GDAL 3.2 COG driver.aligned_levels (cli and api) to select the level of overview to align with the TMS grid.Breaking Changes: * removed --latitude-adjustment/--global-maxzoom option in the CLI * removed latitude_adjustment option in rio_cogeo.cogeo.cog_translate * updated overview blocksize to match the blocksize of the high resolution data (instead of default to 128) * for web-optimized COG, the highest overview level will be aligned with the TMS grid.
pkg_resources (pypa/setuptools#510)There have been no changes since 2.0a9
"},{"location":"release-notes/#20a9-2020-10-03","title":"2.0a9 (2020-10-03)","text":"utils.has_mask_band (#156)rio_cogeo.cogeo.cog_infocount information in rio_cogeo.cogeo.cog_info output (#150)rio_cogeo.cogeo.cog_infoBreaking Changes: * rio_cogeo.utils.get_max_zoom renamed rio_cogeo.utils.get_zooms and now return min/max zoom.
"},{"location":"release-notes/#20a5-2020-07-31","title":"2.0a5 (2020-07-31)","text":"info CLI (#134)Deflate as default temporary compression (#137)--config CLI option to pass additional GDAL Configuration options (#135)--blocksize option in CLI (#131)rio_cogeo.utils.get_maximum_overview_level and use rasterio.rio.overview.get_maximum_overview_level (#132)dataset_mask returned by rasterio to uint8 to overcome a bug in rasterio 1.1.2 (#115)strict option to cog_validate to treat warnings as error (#109) * Thanks @pierotofyPHOTOMETRIC=MINISBLACK when PHOTOMETRIC is set to YCBCR for 1 band dataset (#41)-forward-band-tags options (#115)--allow-intermediate-compression option to reduce the memory/disk footprint (#103)non-standard compression (#97)CLI Changes: * add -t, --dtype datatype option.
API Changes: * add datatype option * update for rasterio>=1.0.28 * allow rasterio.io.DatasetReader input (#89)
Note: This release was deleted in PyPi.
"},{"location":"release-notes/#110-2019-07-16","title":"1.1.0 (2019-07-16)","text":"--web-optimized option to create a web optimized COG (#10)--latitude-adjustment/--global-maxzoom option to adjust MAX_ZOOM for global datasets--resampling option to select the resampling algorithm when using --web-optimized--in-memory/--no-in-memory options to use temporyNamedd file instead of in-memory temp file.Breaking Changes:
Breaking Changes:
Breaking Changes:
--quiet option (#46)--overview-blocksize to set overview's internal tile size (#60)Bug fixes:
Breaking Changes:
Breaking Changes: * replace \"BAND\" by \"PIXEL\" interleave in PACKBITS profile (#16)
"},{"location":"release-notes/#10dev12018-16-13","title":"1.0dev1(2018-16-13)","text":"