2.1.  Terms and definitions

2.1.1. BigTIFF

a file that uses modified TIFF headers to allow for internal 64 bit offset (adding support for TIFF files larger than 4 Gbytes)

2.1.2. Cloud

an on-demand availability of computer system resources, especially data storage (cloud storage) and computing power, that are exposed in the web by cloud providers that do not require direct active management by the user

2.1.3. Coverage

feature that acts as a function to return values from its range for any direct position within its spatiotemporal domain [OGC Abstract Topic 6]

2.1.4. Cloud Optimized GeoTIFF

a regular GeoTIFF file, aimed at being hosted on a HTTP file server, with an internal organization that enables more efficient workflows on the cloud [https://www.cogeo.org/]

2.1.5. Geokey

an equivalent in function to a TIFF tag, but with a different storage mechanism defined by the GeoTIFF [GeoTIFF Format Specification 1.0]

2.1.6. GeoTIFF

Standard for storing georeference and geocoding information in a TIFF 6.0 compliant raster file. [GeoTIFF Format Specification 1.0]

2.1.7. Imagery

representation of phenomena as images produced electronically and/or optical techniques. [ISO 19101-2:2018, 3.14]

2.1.8. Overview

Image File Directory that contains a reduced resolution image

2.1.9. Range

a HTTP GET request type that lets clients ask for the portions of a web resource that they need

2.1.10. Raster

usually rectangular pattern of parallel scanning lines forming or corresponding to the display on a cathode ray tube [ISO 19123:2005, 4.1.30]

a continuous planar space in which pixel values are visually realized [GeoTIFF v1.0]

2.1.11. Regular grid

grid whose grid lines have a constant distance along each grid axis [OGC 09-146r8, Coverage Implementation Schema with Corrigendum]

2.1.12. Subfile

Image File Directory (a part of a TIFF file) that contains one raster image

2.1.13. Tag

a packet of numerical or ASCII values, which have a numerical “Tag” ID indicating their information content in a TIFF file [GeoTIFF Format Specification 1.0]

2.1.14. Tile

geometric shape with known properties that may or may not be the result of a tiling (tessellation) process. A tile consists of a single connected “piece” (topological disc) without “holes” or “lines” [OGC 19-014r3]

small rectangular representation of geographic data, often part of a set of such elements, covering a tiling scheme and sharing similar information content and graphical styling. A tile can be uniquely defined in a tile matrix by one integer index in each dimension. [OGC 17-083r3, fragment]

2.2.  Abbreviated terms

COG

Cloud Optimized GeoTIFF

HTTP

Hypertext Transfer Protocol

IFD

Image File Directory

TIFF

Tagged Image File Format

5.1.  Tiles

As shown in Figure 2, typical raster images store data row by row. To avoid having a stream of bytes that is a too long, a TIFF image is divided into strips. The client must read most of the file to get the piece of the image it is interested in.

Figure 2 — Traditional raster image storage is row by row as indicated by the green path (from https://www.element84.com/blog/cloud-optimized-geotiff-vs-the-meta-raster-format)

Instead, COG stores image data in tiles (a schema introduced in TIFF v6). With a tile matrix (commonly known as tiling in the COG community), only the tiles covering the area of interest need to be read by the client making data extraction and visualization faster (see Figure 3)

Figure 3 — COGs store images tile by tile instead of row by row (from https://www.element84.com/blog/cloud-optimized-geotiff-vs-the-meta-raster-format)

5.2.  Reduced-Resolution Subfiles

Clients do not always need to show a full resolution image. Reduced-Resolution Subfiles (commonly known as overviews in the COG community) are down-sampled versions of the original image. They represent “zoomed out” versions of the image (see Figure 4).

Figure 4 — Image reduction of resolution to generate overviews (from https://www.element84.com/blog/cloud-optimized-geotiff-vs-the-meta-raster-format)

Multiple overviews can be stored in a COG file to match multiple zoom levels. Overviews are stored as tiles just like the original image and they share the same georeference. So an application that supports zooming only needs to retrieve the tiles for the overview associated with the given zoom level.

The use of tiles is not new and is used in several circumstances, as shown in Table 1, which identifies use of tiles in servers implementing Web Feature Service (WFS), Web Map Service (WMS), Web Coverage Service (WCS), and Web Map Tile Service (WMTS) Standards. Servers can structure their internal data in tiles (that are invisible to the user) to produce faster responses or have rendered tiles directly that can be presented to the user. When data is transmitted to the client and rendered on the client side, COG is an ideal solution for gridded data.

Table 1 — Use of tiles in different services and approaches

(inspired by https://github.com/openlayers/openlayers/issues/10733)

To make the COG format an efficient data container available over the web, there is a need to be able to request a fragment of a file. Fortunately, HTTP 1.1 introduced support for range requests. Range requests enable a client to request only a portion of an HTTP message from a server. A server indicates support for range requests by returning the header Accept-Ranges: bytes. In the case of COGs, this enables the client to request the TIFF file header, the GeoTIFF tags, and the relevant individual tiles or tile ranges without downloading the entire file, as illustrated by Figure 5.

Figure 5 — Image transmission of only one tile

After this introduction, the ER is structured into the following sections:

Section 5 discusses the level of adoption of the COG format by clients and data providers

Section 6 discusses how COG is implemented in practice by providing examples

Section 7 proposes a set of requirements classes that could constitute the starting point for a future COG standard in the OGC Standards program

Section 8 discusses some issues found in the current approach for COG and proposes solutions

Section 9 proposes a set of requirements classes that could constitute the starting point for a future BigTIFF standard in the OGC Standards program

6.1.  TIFF v6 and BigTIFF

GeoTIFF (and, by extension COG) is dependent on TIFF v6 which is limited to files sizes of less than 4 GBytes. However, implementations of GeoTIFF (and COG) overcome this limitation by adopting BigTIFF as an alternative to the TIFF v6 headers. There is currently a misalignment between the GeoTIFF Standard (that ignores BigTIFF) and its implementations (that implement BigTIFF) that could be solved by formalizing BigTIFF as an official standard and referencing BigTIFF in the GeoTIFF Standard.

6.2.  Georeference in COG

From all of the tile matrices provided at different resolutions in a COG file, only the highest resolution is georeferenced by GeoTIFF tags. The georeference of the lower resolutions depends on this georeference. Clients have to deduce the georeference of the reduced resolution subfiles by computing a ratio between the number of columns of the highest resolution and the reduced resolution (and another ratio for the number of rows). Instead, COG could take advantage of the tile matrix set data structure defined in the OGC 17-083r2 OGC Two Dimensional Tile Matrix Set Standard (2d-TMS). COG could also include a Tile Matrix Set definition and tile indices as new GeoTIFF tags. More work and discussions are needed to assess the convenience of this alignment between OGC and 2d-TMS.

8.1.  Efficient organization of data in a TIFF file

The Tiling and Reduced-Resolution Subfiles (sometimes called Overviews) in the GeoTIFF format supports the necessary structure for COG files so that the HTTP GET Range queries can request just the part of the file that is relevant.

Reduced-Resolution Subfiles come into play when the client wants to render a quick image of the whole or a big part of the area represented in the file. Instead of downloading every pixel, the software can just request a smaller, already created, lower-resolution version. The structure of the COG file on an HTTP Range supporting web server enables client software to easily find just the part of the whole file that is needed.

Tiles come into play when some small area of the overall extent of the COG file needs to be processed or visualized. This could be part of a reduced-resolution subfile, or it could be at full resolution. Tile organization makes all the relevant bytes of an area (a tile) to be in the same part of the file, so the software can use HTTP GET Range request to get only the tiles it needs.

8.2.  Relation to OGC Tile Set Standards

The combined use of tiles and resolution levels is not new in OGC Standards. In fact the OGC Two-dimensional Tile Matrix Set standard (and the older OGC WMTS 1.0) use exactly the same approach. However, the draft OGC API — Tiles specification and the older WMTS 1.0 Standard require either a service to be installed in the web server provided or thousands of pre-generated independent tiles to be created. None of this is necessary in the COG approach as most of the modern web services natively support HTTP range.

Improving the relationship between COG and 2DTMS can be beneficial, so this document includes an extra requirement class for COG that could support using a list of COGs to store a Tile Set based on Common Tile Matrix Sets.

9.1.  How to support COG in browsers

COG is not supported natively in web browsers. However, EOX (https://eox.at) has developed a set of JavaScript files that are used by the COG explorer library https://github.com/geotiffjs/cog-explorer. The library adds COG support to any modern browser. This demo portal: https://geotiffjs.github.io/cog-explorer provides a live example of the use of this library.

To demonstrate the use of the COG explorer with any COG file, the participants tried to connect the COG explorer to the Spanish CNIG download center http://centrodedescargas.cnig.es/CentroDescargas/catalogo.do?Serie=LANDS#. In this site, the historical Landsat national mosaics are available as COG files. Unfortunately, the CNIG has decided to adopt a multipart file format as a response to an HTTP GET request. This approach has the advantage of suggesting a file name to save the file when it is downloaded. However, there are two disadvantages: it forces encoding the COG file in base64 (inflating the size of the transmission) and it does not support COG HTTP GET range requests, thereby making the random access over HTTP impossible. This also makes the CNIG website incompatible with the COG explorer. Instead, the participants decided to download one of the available files and copy it to one of the CREAF servers. Once copies in the web shared folder, a normal HTTP GET request with range was possible. The file was exposed in the following url: https://joanma.uab.cat/temp/cog/landsat5_1991.tif.

9.1.1.  COG and CORS in browsers

The COG-explorer is only available as an HTTPS endpoint. The rules recently auto-imposed in web browsers prevent a page that comes from a HTTPS domain to read data from an HTTP service. To be compatible with COG-Explorer, servers have to expose COG files in a URL starting with https://. To meet this requirement, an Internet Information Service (IIS) in Windows 2012r2 as a web server was selected. IIS in Windows 2012r2 supports “HTTP range” by default. However, that is not enough due to other Cross-Origin Resource Sharing (CORS) restrictions imposed by browsers. To “authorize” the COG-explorer domain (geotiffjs.github.io) to read the COG file in another domain (joanma.uab.cat), adding these two headers in the IIS configuration was necessary.

Access-Control-Allow-Headers: range
Access-Control-Allow-Origin: *

This way, the client knows that the http://joanma.uab.cat authorizes any other domain to read data and check for range headers. Now, server data can be read by the COG-explorer application directly (see Figure 6).

Figure 6 — Result of loading a COG file in the COG explorer that can be reproduced with https://geotiffjs.github.io/cog-explorer/#long=2.131&lat=41.347&zoom=8&scene=https://joanma.uab.es/temp/cog/landsat5_1991.tif

To demonstrate that library is able to request only the data belonging to the necessary tiles at the level of resolution needed to populate the screen without having to download the whole file, the Chrome Developer Tools can be used. The Network tab in the Chrome Developer Tools provides a list of the HTTP requests that are happening behind the scenes and reveals that the same COG file is accessed in several requests (see Figure 7).

Figure 7 — COG Explorer does several requests to the server

Some of the requests are HTTP HEAD requests that check if the connection will work. The HEAD request is also used to communicate the full size of the COG file (see Figure 8). This is necessary to know the valid range of a HTTP range header.

Figure 8 — COG Explorer HEAD request sample

Others requests are GET requests with the range header that limits the volume of data retrieved to a few kilobytes (see Figure 9).

Figure 9 — COG Explorer GET request sample asking data in the 494403584-494469120 interval of bytes.

9.2.  COG support in desktop solutions

Support for geospatial raster formats in many desktop solutions is driven by the GDAL libraries. There is a GDAL driver to read and write COG files. This driver was introduced in GDAL version 3.1 and now version 3.3 has the following high level characteristics:

• Support for TIFF or BigTIFF file;

• Create tiles (512 pixels by default) for imagery, mask, and overviews;

• Create lower resolution overviews until the maximum size of the smallest overview level is lower than 512 pixels;

• Compress the tiles if requested;

• Support pixel interleaving for multi-band dataset; and

• Create an optimized layout of TIFF sections to minimize the number of data requests needed by a reader doing random read access.

More about this driver is here: https://gdal.org/drivers/raster/cog.html

9.3.  COG support in map services

The GeoServer community has developed an extension to support COG: https://docs.geoserver.org/latest/en/user/community/cog/cog.html

The COG plugin does not add new stores. Instead, the plugin adds COG support to existing stores. When configuring a GeoTIFF store, a new checkbox is available: “Cloud Optimized GeoTIFF (COG).” The plugin gives support for HTTP range and provides an optimized implementation for Amazon S3 buckets.

9.4.  COG support in data cubes

The Open Data Cube (ODC) is an Open Source Geospatial Data Management and Analysis Software project that helps the user harness the power of Satellite data. At its core, the ODC is a set of Python libraries and a PostgreSQL database that helps the user work with geospatial raster data. In the past, in order to accelerate access to long time series, the Open Data Cube had a double-step process for adding remote sensing satellite data in the system. After downloading the data, the files were first indexed and latter transformed and ingested as netCDF tiles. Indexing is a process of including a reference to a downloaded file in a database. Even if reading the indexed version of the files is possible, depending on the format, such an index could not be optimal. Ingesting is a process of taking the indexed files and transforming those files into a set of netCDFs files, each one representing a big tile. Ingesting requires time and requires disk space as it creates a duplicate of the data but results in an increase in data retrieval performance. If original data is downloaded as COG, there is no need for ingesting the data since the indexed version is performing optimally. That is why the new version of ODC replaces by a direct indexing of COG data files, avoiding the need for data ingestion and transformation. The documentations says that the limitation of netCDF reading to a single thread means that reading from COG files on disk could be faster than ingested netCDF in some situations. The single thread limitation is originated by the library used and not by the file format.

9.5.  COG data adoption

At the time of writing this ER, a two-speed level of adoption was observed. Some cloud-based data repositories are duplicates of the official space agencies’ distribution systems created by some Earth observation companies. This is showing that Earth observation industry sees a business opportunity in simplifying access to remote sensing data using COG. These repositories have catalogues based on SpatioTemporal Asset Catalog (STAC) that are exposing the remote sensing data transformed to COG format. These are two examples:

• Accessing Landsat 8 Collection 2 Level 2 data with the Planetary Computer STAC API (https://planetarycomputer.microsoft.com/dataset/landsat-8-c2-l2#Example-Notebook)

• Sentinel-2 Cloud-Optimized GeoTIFFs (https://registry.opendata.aws/sentinel-2-l2a-cogs/)

On the other hand, there are efforts from the remote sensing agencies themselves to migrate to COG format. A significant example is the USGS, which is making the whole Landsat archive available directly in COG (https://prd-wret.s3.us-west-2.amazonaws.com/assets/palladium/production/atoms/files/LSDS-1388-Landsat-Cloud-Optimized-GeoTIFF_DFCB-v2.0.pdf) as well as other products such as the USGS Digital Elevation Model (https://www.usgs.gov/news/usgs-digital-elevation-models-dem-switching-new-distribution-format). NASA also considers COG an appropriate format to stimulate participation in a hackathon with the objective of reusing remote sensing data (https://earthdata.nasa.gov/collaborate/cloud-optimized-geotiffs).

9.6.  Criticism to the approach

One of the main impediments to COG adoption is the lack of support in browsers. This means that libraries are needed to make COG readable in browsers and it will be desirable to have native support to maps in web browsers (see Figure 10 and Figure 11)

Figure 10 — Twitter about COG not being natively supported by browsers (Part 1)

Figure 11 — Twitter about COG not being natively supported by browsers (Part 2)

For example, a WMS service that uses COG data in the backend could overcome this limitation by providing on-the-fly translation from COG tiles to PNG or JPEG formats.

Other criticisms derive from the TIFF original design. For example, COG cannot easily be used for multidimensional data beyond 2D as the TIFF internal structure is limited to one or more 2D data subfiles.

9.7.  Alternatives to COG

There are other formats that are alternatives to COG.

• Meta Raster Format (MRF) is an image and data storage format designed for fast access to imagery within a georeferenced tile pyramid at discrete resolutions. (https://github.com/nasa-gibs/mrf/blob/master/spec/mrf_spec.md)

• GeoPngDB is a tiled geospatial data format capable of representing global-scale data sets at high resolution in a format natively supported by web browsers. (https://github.com/sasakiassociates/geo-png-db)

• The Cloud Raster Format (CRF) is an Esri-created raster format that is optimized for writing and reading large files in a distributed processing and storage environment.

• GeoPackage Tiled Gridded Coverages is a way to store tiles in a GeoPackage. The tiles are stored as TIFF files. In this case the TIFF should not have internal tiles but they are independent TIFF files each one representing a single tile. This way you can extract the desired tile (one TIFF tile) using a SQLite queries. This approach does not support HTTP GET Range to request a set of bytes in the GeoPackage, but nothing prevents setting up a web service that uses GeoPackage in the backend to provide access to the tiles, one by one.

• netCDF-4 / HDF5 are formats for scientific data. Data can be structured in several tables that represent 2-dimensional arrays or in cubes of n-dimensional arrays. This tables or cubes could be internally arranged in tiles and overviews. Some metadata will be needed to make the structure of tables discoverable. With the right libraries, netCDF-4 / HDF5 could be accessed over the network using HTTP range.

It is worth mention that optimized organizations for the Cloud are also being defined for other types of coverages. For example, the Cloud Optimized Point Cloud (COPC) (https://copc.io/) is a LAZ 1.4 file that stores a point cloud organized in a clustered octree. COPC contains a Variable Length Record (VLR) that describes the octree organization of data that are stored in standard LAZ 1.4 chunks. It was inspired by the Entwine Point Tile format (https://entwine.io/entwine-point-tile.html) but instead of using some JSON files, the data is stored in binary structures in the LAZ format. A reader does not know about COPC can still read the LAZ without taking advantage of the octree organization.

10.1.  Current specification

The COG website at https://www.cogeo.org/ provides some help on how COG works and directs to the current specification in GitHub: https://github.com/cogeotiff/cog-spec/blob/master/spec.md. The specification is surprisingly short and consists of one page text and a description of a reference implementation in the GDAL library. The document is contributed by Jeff Albrecht, Kyle Barron, and Vincent Sarago.

10.2.  Common organization of a COG file

In a COG file created by the GDAL library using libtiff >= 4.0.11, the information for a COG file results in a TIFF file (or a BigTIFF file) organized in sections following the sequence below.

1. TIFF/BigTIFF header/signature and pointer to first IFD (Image File Directory)

2. Internal GDAL information called “ghost area”

3. IFD of the full resolution image, followed by GeoTIFF tags values, excluding the TileOffsets and TileByteCounts arrays

4. IFD of the mask of the full resolution image, if present, followed by GeoTIFF tags values, excluding the TileOffsets and TileByteCounts arrays

5. IFD of the first (largest in dimensions) overview level of the full resolution image, if present

6. IFD of the following overviews (repeated)

7. IFD of the last (smallest) overview level, if present

8. IFD of the first (largest in dimensions) overview level of the mask, if present

9. IFD of the following mask overviews (repeated)

10. IFD of the last (smallest) overview level of the mask, if present

11. TileOffsets and TileByteCounts arrays of the above IFDs

12. tile data of the smallest overview, if present (with each tile followed by the corresponding tile of mask data, if present), with leader and trailer bytes

13. tile data of the following overviews (repeated)

14. tile data of the largest overview, if present (interleaved with mask data if present)

15. tile data of the full resolution image, if present (interleaved with corresponding mask data if present)

Note that the IFD or full-resolution image and the overviews and the actual tile data are in opposite order. The full-resolution data is at the end to the file while the smallest overview is at the beginning of the file (after the IFDs). Please note that only the full resolution image contains GeoTIFF. The georeference of the overviews need to be derived from the relation between number of columns and rows between the full resolution image and the overviews.

10.3.  Validating and extracting GeoTIFF metadata from COG files

There is an online tool to validate if a COG is well formed based on the GDAL library. If COG becomes an OGC standard in the future, it could be the basis for testing implementations: http://cog-validate.radiant.earth/html

This online tool has a user interface and can also be invoked as an API with the url as a single query parameter. In the latter case, the response is a JSON document containing several details extracted from the file including the bounding box, the tile count for each overview and band, the Well Known Text (WKT) CRS information, the geotransform matrix, and the offset to each overview.

{
	"status": "success",
	"gdal_info": {
		"files": [ "/vsicurl/https://joanma.uab.es/temp/cog/landsat5_1991.tif" ],
		"cornerCoordinates": {
			"upperRight": [ 1132533.893, 4883529.188 ],
			"lowerLeft": [ -74126.107, 3870639.188 ],
			"lowerRight": [ 1132533.893, 3870639.188 ],
			"upperLeft": [ -74126.107, 4883529.188 ],
			"center": [ 529203.893, 4377084.188 ]
		},
		"wgs84Extent": {
			"type": "Polygon",
			"coordinates": [
				[
					[ -10.1459972, 43.8809094 ],
					[ -9.273279, 34.8161184 ],
					[ 3.9075608, 34.7816184 ],
					[ 4.8665274, 43.8333089 ],
					[ -10.1459972, 43.8809094 ]
				]
			]
		},
		"description": "https://joanma.uab.es/temp/cog/landsat5_1991.tif",
		"driverShortName": "GTiff",
		"driverLongName": "GeoTIFF",
		"bands": [
			{
				"mask": {
					"overviews": [
						{
							"size": [ 20111, 16882 ]
						},
						{
							"size": [ 10056, 8441 ]
						},
						{
							"size": [ 5028, 4221 ]
						},
						{
							"size": [ 2514, 2111 ]
						},
						{
							"size": [ 1257, 1056 ]
						},
						{
							"size": [ 629, 528 ]
						},
						{
							"size": [ 315, 264 ]
						},
						{
							"size": [ 158, 132 ]
						},
						{
							"size": [ 79, 66 ]
						}
					],
					"flags": [ "PER_DATASET", "ALPHA" ]
				},
				"band": 1,
				"colorInterpretation": "Red",
				"overviews": [
					{
						"size": [ 20111, 16882 ]
					},
					{
						"size": [ 10056, 8441 ]
					},
					{
						"size": [ 5028, 4221 ]
					},
					{
						"size": [ 2514, 2111 ]
					},
					{
						"size": [ 1257, 1056 ]
					},
					{
						"size": [ 629, 528 ]
					},
					{
						"size": [ 315, 264 ]
					},
					{
						"size": [ 158, 132 ]
					},
					{
						"size": [ 79, 66 ]
					}
				],
				"type": "Byte",
				"block": [ 256, 256 ],
				"metadata": {

				}
			},
			{
				"mask": {
					"overviews": [
						{
							"size": [ 20111, 16882 ]
						},
						{
							"size": [ 10056, 8441 ]
						},
						{
							"size": [ 5028, 4221 ]
						},
						{
							"size": [ 2514, 2111 ]
						},
						{
							"size": [ 1257, 1056 ]
						},
						{
							"size": [ 629, 528 ]
						},
						{
							"size": [ 315, 264 ]
						},
						{
							"size": [ 158, 132 ]
						},
						{
							"size": [ 79, 66 ]
						}
					],
					"flags": [ "PER_DATASET", "ALPHA" ]
				},
				"band": 2,
				"colorInterpretation": "Green",
				"overviews": [
					{
						"size": [ 20111, 16882 ]
					},
					{
						"size": [ 10056, 8441 ]
					},
					{
						"size": [ 5028, 4221 ]
					},
					{
						"size": [ 2514, 2111 ]
					},
					{
						"size": [ 1257, 1056 ]
					},
					{
						"size": [ 629, 528 ]
					},
					{
						"size": [ 315, 264 ]
					},
					{
						"size": [ 158, 132 ]
					},
					{
						"size": [ 79, 66 ]
					}
				],
				"type": "Byte",
				"block": [ 256, 256 ],
				"metadata": {

				}
			},
			{
				"mask": {
					"overviews": [
						{
							"size": [ 20111, 16882 ]
						},
						{
							"size": [ 10056, 8441 ]
						},
						{
							"size": [ 5028, 4221 ]
						},
						{
							"size": [ 2514, 2111 ]
						},
						{
							"size": [ 1257, 1056 ]
						},
						{
							"size": [ 629, 528 ]
						},
						{
							"size": [ 315, 264 ]
						},
						{
							"size": [ 158, 132 ]
						},
						{
							"size": [ 79, 66 ]
						}
					],
					"flags": [ "PER_DATASET", "ALPHA" ]
				},
				"band": 3,
				"colorInterpretation": "Blue",
				"overviews": [
					{
						"size": [ 20111, 16882 ]
					},
					{
						"size": [ 10056, 8441 ]
					},
					{
						"size": [ 5028, 4221 ]
					},
					{
						"size": [ 2514, 2111 ]
					},
					{
						"size": [ 1257, 1056 ]
					},
					{
						"size": [ 629, 528 ]
					},
					{
						"size": [ 315, 264 ]
					},
					{
						"size": [ 158, 132 ]
					},
					{
						"size": [ 79, 66 ]
					}
				],
				"type": "Byte",
				"block": [ 256, 256 ],
				"metadata": {

				}
			},
			{
				"band": 4,
				"colorInterpretation": "Alpha",
				"overviews": [
					{
						"size": [ 20111, 16882 ]
					},
					{
						"size": [ 10056, 8441 ]
					},
					{
						"size": [ 5028, 4221 ]
					},
					{
						"size": [ 2514, 2111 ]
					},
					{
						"size": [ 1257, 1056 ]
					},
					{
						"size": [ 629, 528 ]
					},
					{
						"size": [ 315, 264 ]
					},
					{
						"size": [ 158, 132 ]
					},
					{
						"size": [ 79, 66 ]
					}
				],
				"type": "Byte",
				"block": [ 256, 256 ],
				"metadata": {

				}
			}
		],
		"coordinateSystem": {
			"wkt": "PROJCS[\"WGS 84 / UTM zone 30N\",\n    GEOGCS[\"WGS 84\",\n        DATUM[\"WGS_1984\",\n            SPHEROID[\"WGS 84\",6378137,298.257223563,\n                AUTHORITY[\"EPSG\",\"7030\"]],\n            AUTHORITY[\"EPSG\",\"6326\"]],\n        PRIMEM[\"Greenwich\",0,\n            AUTHORITY[\"EPSG\",\"8901\"]],\n        UNIT[\"degree\",0.0174532925199433,\n            AUTHORITY[\"EPSG\",\"9122\"]],\n        AUTHORITY[\"EPSG\",\"4326\"]],\n    PROJECTION[\"Transverse_Mercator\"],\n    PARAMETER[\"latitude_of_origin\",0],\n    PARAMETER[\"central_meridian\",-3],\n    PARAMETER[\"scale_factor\",0.9996],\n    PARAMETER[\"false_easting\",500000],\n    PARAMETER[\"false_northing\",0],\n    UNIT[\"metre\",1,\n        AUTHORITY[\"EPSG\",\"9001\"]],\n    AXIS[\"Easting\",EAST],\n    AXIS[\"Northing\",NORTH],\n    AUTHORITY[\"EPSG\",\"32630\"]]"
		},
		"geoTransform": [ -74126.10725287361, 30.0, 0.0, 4883529.1883060075, 0.0, -30.0 ],
		"metadata": {
			"": {
				"TIFFTAG_DATETIME": "Tue Feb  2 12:16:23 2021\n",
				"BIGTIFF": "YES",
				"BLOCKYSIZE": "256",
				"PHOTOMETRIC": "RGB",
				"COMPRESS": "LZW",
				"AREA_OR_POINT": "Area",
				"INTERLEAVE": "PIXEL",
				"BLOCKXSIZE": "256",
				"TILED": "NO",
				"ALPHA": "NON-PREMULTIPLIED",
				"TIFFTAG_SOFTWARE": "BlueMarble Geographics GeoCore"
			},
			"IMAGE_STRUCTURE": {
				"INTERLEAVE": "PIXEL",
				"COMPRESSION": "JPEG"
			}
		},
		"size": [ 40222, 33763 ]
	},
	"details": {
		"ifd_offsets": {
			"overview_8": 784204,
			"overview_6": 783216,
			"overview_7": 783750,
			"overview_4": 780596,
			"overview_5": 782426,
			"overview_2": 753256,
			"overview_3": 774766,
			"overview_0": 335356,
			"overview_1": 669442,
			"main": 16
		},
		"data_offsets": {
			"overview_8": 784593,
			"overview_6": 823612,
			"overview_7": 794429,
			"overview_4": 1165093,
			"overview_5": 905154,
			"overview_2": 5677864,
			"overview_3": 2093193,
			"overview_0": 73529801,
			"overview_1": 19658902,
			"main": 274599054
		}
	}
}

Another way to extract the internal metadata of a COG file is to use the EXIF tool: https://metacpan.org/pod/release/EXIFTOOL/Image-ExifTool-5.67/exiftool.

The syntax as follows generates a report of the file metadata.

exiftool
c:\COG\pnt_landsat5_2006_mosaico_islas_canarias_b321_80porciento_b543_20porciento_hu28_8bits_COG_90.tif -a -U -g1

Adding -U in the command line results in descriptions instead of tag names:

---- ExifTool ----
ExifTool Version Number         : 12.25
---- System ----
File Name                       : pnt_landsat5_2006_mosaico_islas_canarias_b321_80porciento_b543_20porciento_hu28_8bits_COG_90.tif
Directory                       : D:/docs/Recerca/COG
File Size                       : 13 MiB
File Modification Date/Time     : 2021:05:02 10:00:26+02:00
File Access Date/Time           : 2021:05:07 23:03:06+02:00
File Creation Date/Time         : 2021:05:02 10:00:24+02:00
File Permissions                : -rw-rw-rw-
---- File ----
File Type                       : BTF
File Type Extension             : btf
MIME Type                       : image/x-tiff-big
---- IFD0 ----
Image Width                     : 15829
Image Height                    : 6520
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
X Resolution                    : 72
Y Resolution                    : 72
Planar Configuration            : Chunky
Resolution Unit                 : inches
Software                        : Adobe Photoshop CC 2018 (Windows)
Modify Date                     : 2020:06:25 10:30:36
Tile Width                      : 256
Tile Length                     : 256
Tile Offsets                    : (Binary data 13380 bytes, use -b option to extract)
Tile Byte Counts                : (Binary data 6873 bytes, use -b option to extract)
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
Pixel Scale                     : 30 30 0
Model Tie Point                 : 0 0 0 187333.999630584 3255440 0
Geo Tiff Directory              : (Binary data 106 bytes, use -b option to extract)
Geo Tiff Ascii Params           : (Binary data 29 bytes, use -b option to extract)
---- ICC-header ----
Profile CMM Type                : Adobe Systems Inc.
Profile Version                 : 2.1.0
Profile Class                   : Display Device Profile
Color Space Data                : RGB
Profile Connection Space        : XYZ
Profile Date Time               : 1999:06:03 00:00:00
Profile File Signature          : acsp
Primary Platform                : Apple Computer Inc.
CMM Flags                       : Not Embedded, Independent
Device Manufacturer             : none
Device Model                    :
Device Attributes               : Reflective, Glossy, Positive, Color
Rendering Intent                : Perceptual
Connection Space Illuminant     : 0.9642 1 0.82491
Profile Creator                 : Adobe Systems Inc.
Profile ID                      : 0
---- ICC_Profile ----
Profile Copyright               : Copyright 1999 Adobe Systems Incorporated
Profile Description             : Adobe RGB (1998)
Media White Point               : 0.95045 1 1.08905
Media Black Point               : 0 0 0
Red Tone Reproduction Curve     : (Binary data 14 bytes, use -b option to extract)
Green Tone Reproduction Curve   : (Binary data 14 bytes, use -b option to extract)
Blue Tone Reproduction Curve    : (Binary data 14 bytes, use -b option to extract)
Red Matrix Column               : 0.60974 0.31111 0.01947
Green Matrix Column             : 0.20528 0.62567 0.06087
Blue Matrix Column              : 0.14919 0.06322 0.74457
---- IFD1 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 7915
Image Height                    : 3260
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : (Binary data 12895 bytes, use -b option to extract)
Tile Byte Counts                : (Binary data 6789 bytes, use -b option to extract)
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- IFD2 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 3958
Image Height                    : 1630
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : (Binary data 2880 bytes, use -b option to extract)
Tile Byte Counts                : (Binary data 1710 bytes, use -b option to extract)
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- IFD3 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 1979
Image Height                    : 815
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : (Binary data 783 bytes, use -b option to extract)
Tile Byte Counts                : (Binary data 483 bytes, use -b option to extract)
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- IFD4 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 990
Image Height                    : 408
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : (Binary data 222 bytes, use -b option to extract)
Tile Byte Counts                : (Binary data 139 bytes, use -b option to extract)
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- IFD5 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 495
Image Height                    : 204
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : (Binary data 47 bytes, use -b option to extract)
Tile Byte Counts                : (Binary data 38 bytes, use -b option to extract)
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- IFD6 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 248
Image Height                    : 102
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : 72539 76095
Tile Byte Counts                : 3556 3300
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- IFD7 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 124
Image Height                    : 51
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : 69071
Tile Byte Counts                : 3468
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- IFD8 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 62
Image Height                    : 26
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : 67168
Tile Byte Counts                : 1903
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- IFD9 ----
Subfile Type                    : Reduced-resolution image
Image Width                     : 31
Image Height                    : 13
Bits Per Sample                 : 8 8 8
Compression                     : JPEG
Photometric Interpretation      : RGB
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 128
Tile Length                     : 128
Tile Offsets                    : 65171
Tile Byte Counts                : 1997
Sample Format                   : Unsigned; Unsigned; Unsigned
JPEG Tables                     : (Binary data 73 bytes, use -b option to extract)
---- Composite ----
Image Size                      : 15829x6520
Megapixels                      : 103.2

Some extra sentences helps us to extract the GeoTiff entries

exiftool c:\COG\pnt_landsat5_2006_mosaico_islas_canarias_b321_80porciento_b543_20porciento_hu28_8bits_COG_90.tif -b -GeoTiffAsciiParams

exiftool c:\COG\pnt_landsat5_2006_mosaico_islas_canarias_b321_80porciento_b543_20porciento_hu28_8bits_COG_90.tif -b -GeoTiffDirectory

Resulting in:

WGS 84 / UTM zone 28N|WGS 84|

1 1 0 7 1024 0 1 1 1025 0 1 1 1026 34737 22 0 2049 34737 7 22 2054 0 1 9102 3072 0 1 32628 3076 0 1 9001II

10.4.  What is the georeference of the overviews?

In a COG file, there is only one set of GeoTIFF keys for each full-resolution image or band. That means that the georeference applies to the full-resolution subfile leaving the georeference of the reduced-resolution subfiles to interpretation. In the existing COG documentation, there is no information on how to infer the georeference of the subfiles. However, the implementation of the GDAL COG library provides guidance, as follows. The full-resolution subfile and the reduced-resolution subfiles share exactly the same geospatial extent. If the number of pixels of the full-resolution image and the overviews are carefully selected, this generates non-rounded pixel sizes in the lower resolution. If the number of columns is even and the number of rows is odd in the full-resolution image and the relations between the full-resolution and the overview is 2, this results in a non-square pixels in the lower resolution. Non square pixels are not necessary a bad thing but they are uncommon and not well-supported in some implementations.

In order to know about georeference used in the GDAL library for the reduced resolution subfiles, use the following instruction: gdalinfo my.tif -oo OVERVIEW_LEVEL={ol}, where {ol} refers to the reduced resolution subfile starting, and being 0 the largest in dimension overview (omitting the —oo OVERVIEW_LEVEL parameter gives us the georeference of the full resolution image). These are the results of an arbitrary example over the Canary Islands downloaded from the Spanish CNIG website: pnt_landsat5_2006_mosaico_islas_canarias_b321_80porciento_b543_20porciento_hu28_8bits_COG_90.tiff

Table 2 — Resolutions in the COG canary islands example

The assumption that overviews have the exact same bounding box results in image artifacts. With the current convention, it is very difficult for a COG to match with a common TileMatrixSet that gives priority to maintaining square pixels. For example, a square image with a very particular extent configuration that has number of columns and rows based on a power of 2 can fit with a TileMatrixSet that use scale denominators based on powers of 2. In this particular case, a COG file could be used as a container for tiles following the 2D-TMS standard.

10.5.  How to define a COG standard based on conformance classes

An initial examination of the COG specification concluded that there are two standardization targets for the requirements defining COG: COG creation libraries and COG HTTP servers. This should result in two requirement classes, one for the TIFF specification (tiling+overviews) and one for the HTTP range services. Additional discussion identified the need to separate the first class into two classes. A more granular consideration of the requirements concluded that some COG files could be used in analytical tools that primarily require the maximum resolution level so they does not need “overviews.” That is why, in the end, three conformance classes were defined.

However, there is nothing in the three conformance classes that has anything particularly related to GeoTIFF, so a fourth conformance class was defined. This class deals with the current approach to georeference based on the GeoTIFF keys that does the georeference based on a common bounding box. This approach suggests that an alternative fifth conformance class could be defined in the future to deal with georeference using TileMatrixSet indices instead allowing for overviews with slightly different bounding boxes. However, this last requirement class requires the acceptance by the community that there is agreement on how important the use case it represents is (convergence with common TileMatrixSets).

11.1.  GeoTIFF format requirements

A COG file is a TIFF file or a BigTIFF file that uses two main organization techniques available in the TIFF version 6.0 specification: Tiling and Reduced-Resolution Subfiles (sometimes called Overviews). The tiled data can also be compressed for more efficient passage online. In addition, a COG file uses the GeoTIFF conventions for storing the relevant geospatial metadata.

IFDn_PixelScaleX=IFD0_PixelScaleX*IFD0_ImageWidth/IFDn_ImageWidth
IFDn_PixelScaleY=IFD0_PixelScaleY*IFD0_ImageHeight/IFDn_ImageHeight

ImageWidth = 15829
ImageHeight = 6520
ModelTiepointTag = (0 0 0 187334 3255440 0)
ModelPixelScaleTag = (30 30 0)
GeoKeyDirectoryTag:
-    GTModelTypeGeoKey = 1 (ModelTypeProjected 2D)
-    GTRasterTypeGeoKey = 1 (RasterPixelIsArea)
-    ProjectedCRSGeoKey = 32628 (Projected 2D CRS WGS 84 / UTM zone 28N)

11.2.  HTTP range support requirements

HTTP Version 1.1 introduced a range header in the GET requests that supports requesting only a fragment of a resource. If the server advertises with an “Accept-Ranges: bytes” header in its response, the server is telling the client that bytes of data can be requested in parts in separated requests. The client can request just the bytes that it needs from the server at any time. In a web environment, this is very useful for serving things such as video as clients do not need to download the entire file to begin playing it. In the COG use case, it is useful to get only the tiles needed at a particular time. A client showing a COG file on the screen can request the resolutions needed and only the tiles needed to cover the screen. Once the user pans or zooms, other GET range requests will query for other resolutions and tiles.

12.1.  How manage TIFF tags?.

TIFF tags are defined in a way that is extensible. TIFF v.6 defines a set of baseline tags.

However, there is no evidence that Adobe had published any list of private tags. There is a list of private tags maintained by https://www.awaresystems.be/imaging/tiff/tifftags/private.html that include the GeoTIFF tags as well as some of the ones introduced by ISO12234-2 for the photographic industry. This situation complicates the extensibility and affects the capacity to define new tags for the new versions of GeoTIFF.

12.2.  Need for a LONG type in the GeoTIFF standard

GeoTIFF Keys are available for SHORT and for FLOAT numbers but there is no key to define long numbers. To be able to define a COG subfile as related to some common TileMatrixSets the ranges of long indices need to be defined. The indices cannot be a SHORT because tilecol and tilerow can easily be larger than 65K if the scale denominator is small. Creating a GeoLongParamsTag and associating a TIFF tag number 34738 to it is proposed. There is no evidence that this number has been used for other purposes. If this extension to GeoTIFF is accepted and included, it is possible to define a GeoTIFF Key to indicate the TileMatrixSetLimits covered by a COG file.

12.3.  Media type for COG

Whether a specific media type for COG is needed or if image/tiff is sufficient given that fact that an classic TIFF and a BigTIFF can be identified by the version in the first bytes of the header is not clear (this was discussed in the past and recorded here: https://github.com/opengeospatial/geotiff/issues/34).

Many possibilities have been proposed but no agreement has been reached so far.

• image/tiff; application=geotiff, profile=COG

• image/tiff; application=geotiff; profile=cloud-optimized (proposed in https://github.com/radiantearth/stac-spec/blob/master/best-practices.md#common-media-types-in-stac)

• image/tiff; application=geotiff; cloud-optimized=true

• image/tiff; application=geotiff; cog=true

• image/cogeo+tiff

• image/cog+geotiff

The issue continues the discussions with elaborated media type proposals to make explicit or negotiate the characteristics of a COG file:

• image/tiff; application=geotiff; tiled=true; tile-size=256; overviews=true; compress=DEFLATE;

The specification for the tiff media type in the IANA https://www.iana.org/assignments/media-types/image/tiff only specifies a single optional parameter “application.” Adding a new official parameter such as “profile” will require a modification to the IANA registration.

Considering that a COG is a particular case of GeoTIFF, specifying that the application is COG could be enough. Writing the full name instead of the acronym makes visible the dependency with GeoTIFF.

• image/tiff; application=cloud-optimized-geotiff

However, this approach will prevent applications looking for image/tiff; application=geotiff to detect the media type as compatible to them.

Recently, a new W3C standard proposal named “Negotiation by profile” (https://www.w3.org/TR/dx-prof-conneg/) adds completely different possibility. Instead of extending media types, they are used in combination with the profile concept. The “Content-Profile” new entry in the headers of requests can be added to indicate the name of a profile in addition to the current header Content-Type. This allows us to respect the originally standardized media type indicated in the GeoTIFF standard. Please note that the value of the profile name should be a URI. Having values as full URIs avoids the need for a centralized registry of parameter values. Applying this suggestion, a header of a HTTP response containing a COG file could look like this:

Content-Type: image/tiff; application=geotiff;
Content-Profile: http://www.ogc.org/profile/image/tiff/cog

12.4.  BigTIFF

All versions of TIFF define a format that uses 32-bit offsets and, as such, the file size is limited to 4 GBytes. This limit has been sufficient for many years. Today, however, there is a need for a good multi-purpose open image file format that can handle huge images, or very large arrays of bands, beyond the 4 GByte boundary.

BigTIFF is a variant that closely resembles TIFF. However BigTIFF redefines a couple of internal data structures to support 64-bit offsets and this way extends support for very large files.

The BigTIFF specification is available on the Aware Systems website: https://www.awaresystems.be/imaging/tiff/bigtiff.html. Aware Systems is managed by Joris Van Damme. In the register of data formats maintained by the Library of the Congress (USA), the BigTIFF format is included and in the “licensing and patents” section it says that: “The BigTIFF enhancements in Aperio’s libtiff were made by Ole Eichhorn (formerly of Aperio Technologies, acquired by Leica Biosystems in 2012).” In the words of the bigtiff.org web page: “are donated to the public domain, in gratitude to Sam Leffler, Silicon Graphics, Joris Van Damme, AWare Systems, Frank Warmerdam, Andrey Kisley, Mike Welles, and all who have worked on libtiff over the years to provide such a great tool. The aperio libtiff changes were published on an ‘as is’ basis and neither Ole Eichhorn nor Aperio Technologies make any warranty as to their fitness for any intended use.” (from: https://www.loc.gov/preservation/digital/formats/fdd/fdd000328.shtml).

In an email conversation (June, 16th 2021, 15:05), Joris Van Damme declared that “I don’t mind people building on from my BigTIFF page, and I think you [OGC] should. You [OGC] can quote me on this, this is my official stance.” “As long as you don’t flat out copy and paste, and I much like to see things move forward.”

The OGC staff and participants in the Testbed 17 discussed the possibility to look for consensus in the community and adopt BigTIFF as a Standard and no legal barriers were found. This way the OGC will be able to maintain the specification and reference it from other work such as future versions of the GeoTIFF Standard. The GeoTIFF SWG has accepted the work item and started to explore this possibility.

The OGC GeoTIFF SWG should add credits or a similar section in section to the draft standard to credit Aware Systems for the material that served as starting point, and more generally to all past contributors of the BigTIFF design (possibly pointing to the initial discussion thread at https://www.asmail.be/msg0055453930.html).

13.1.  Overview

13.2.  BigTIFF requirements

The BigTIFF specification provides a solution for a TIFF file that requires a file size larger than 4 Gigabytes. In principle the BigTIFF specification will also support smaller files. However, files that do not require file size bigger than 4 Gigabytes or not using the new data types, should use TIFF version 6 specification instead. This is in order to ensure backwards compatibility with old implementations not supporting BigTIFF.

A BigTIFF file (and a TIFF) is defined to be a sequence of 8-bit bytes, where the bytes are numbered from 0 to N. In a BigTIFF file, the largest possible TIFF file is 2⁶⁴ bytes in length.