From 4da5a36df7e38e2294d1a0d10e8a7a9b000e31d1 Mon Sep 17 00:00:00 2001 From: Jon Duckworth Date: Wed, 26 Jan 2022 13:53:38 -0500 Subject: [PATCH 1/5] Update extensions tutorial --- .../adding-new-and-custom-extensions.ipynb | 572 +++++++----------- 1 file changed, 219 insertions(+), 353 deletions(-) diff --git a/docs/tutorials/adding-new-and-custom-extensions.ipynb b/docs/tutorials/adding-new-and-custom-extensions.ipynb index 636765a68..457fab0e3 100644 --- a/docs/tutorials/adding-new-and-custom-extensions.ipynb +++ b/docs/tutorials/adding-new-and-custom-extensions.ipynb @@ -6,508 +6,374 @@ "source": [ "## Adding New and Custom Extensions\n", "\n", - "This tutorial will cover how to add new extensions to PySTAC. It will go over how to contribute a common extension (one found in the [stac-spec repo](https://github.com/radiantearth/stac-spec/tree/v1.0.0-beta.2/extensions)), as well as how to register a custom extension with PySTAC.\n", + "This tutorial will cover how to implement a new extensions to PySTAC. It will go over:\n", "\n", - "We'll work on implementing the [Satellite Extension](https://github.com/radiantearth/stac-spec/tree/v1.0.0-beta.2/extensions/sat) with a modified extension ID, registering it as `space_camera` instead of `sat`. " - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "import pystac" + "- Using the `PropertiesExtension` and `ExtensionManagementMixin` classes to extend STAC objects\n", + "- Using the `SummariesExtension` class to extend Collection summaries\n", + "\n", + "For this exercise, we will implement an imaginary Order Request Extension that allows us to track an internal order ID associated with a given satellite image, as well as the history of that imagery order. This use-case is specific enough that it would probably not be a good candidate for an actual STAC Extension, but it gives us an opportunity to highlight some of the key aspects and patterns used in implementing STAC Extensions in PySTAC." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "If we were implementing the `sat` extension for real, we would make sure there is an entry for our extension in the `pystac.extensions.Extensions` object [found here](https://github.com/azavea/pystac/blob/v0.5.1/pystac/extensions/__init__.py#L10-L27) with the relevant entry. Here we'll just use our own `Extensions` class to define our fake `SPACE_CAMERA` extension ID:" + "First, we import the PySTAC modules and classes that we will be using throughout the tutorial." ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ - "class Extensions:\n", - " SPACE_CAMERA = 'space_camera'" + "from datetime import datetime, timedelta\n", + "from pprint import pprint\n", + "from typing import Any, Dict, List, Optional, Union\n", + "from uuid import uuid4\n", + "\n", + "import pystac\n", + "from pystac.utils import (\n", + " StringEnum,\n", + " datetime_to_str,\n", + " get_required,\n", + " map_opt,\n", + " str_to_datetime\n", + ")\n", + "from pystac.extensions.base import (\n", + " PropertiesExtension,\n", + " ExtensionManagementMixin\n", + ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "For this tutorial we'll use some code below to read in an item and modify the real `sat` extension ID into our tutorial `space_camera` ID. If we didn't need to do this modification, we could simply read in the item from the URI using `pystac.read_file`." + "## Define the Extension" ] }, { - "cell_type": "code", - "execution_count": 3, + "cell_type": "markdown", "metadata": {}, - "outputs": [], "source": [ - "import json\n", + "Our extension will extend STAC Items by adding the following properties:\n", "\n", - "def modify_sat_extension_id(item_json):\n", - " item_json['stac_extensions'].remove(pystac.extensions.Extensions.SAT)\n", - " item_json['stac_extensions'].append(Extensions.SPACE_CAMERA)\n", - " \n", - "def read_item(href):\n", - " item_json = json.loads(pystac.STAC_IO.read_text(href))\n", - " modify_sat_extension_id(item_json)\n", - " return pystac.read_dict(item_json)" + "- `order:id`: A unique string ID associated with the internal order for this image. This field will be required.\n", + "- `order:history`: A chronological list of events associated with this order. Each of these \"events\" will have a timestamp and an event type, which will be one of the following: `submitted`, `started_processing`, `delivered`, `cancelled`. This field will be optional." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Here we read in an item that implements the `sat` extension, which based on the above code will modify to implement the `space_camera` extension:" + "## Create Extension Classes\n", + "\n", + "Let's start by creating a class to represent the order history events." ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ - "item_before = read_item('https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0-beta.2/extensions/sat/examples/example-landsat8.json')\n" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "True" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "item_before.ext.implements(Extensions.SPACE_CAMERA)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Even though the item reports it implements that extension, that extension isn't registered with PySTAC and if we try to access the extension functionality it will tell us so:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "ename": "ExtensionError", - "evalue": "'space_camera' is not an extension registered with PySTAC", - "output_type": "error", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mExtensionError\u001b[0m Traceback (most recent call last)", - "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mitem_before\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mext\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mExtensions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSPACE_CAMERA\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", - "\u001b[0;32m~/proj/stac/pystac/venv/lib/python3.6/site-packages/pystac-0.5.0-py3.6.egg/pystac/stac_object.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, extension_id)\u001b[0m\n\u001b[1;32m 39\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mpystac\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSTAC_EXTENSIONS\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_registered_extension\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mextension_id\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 40\u001b[0m raise ExtensionError(\"'{}' is not an extension \"\n\u001b[0;32m---> 41\u001b[0;31m \"registered with PySTAC\".format(extension_id))\n\u001b[0m\u001b[1;32m 42\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 43\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimplements\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mextension_id\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", - "\u001b[0;31mExtensionError\u001b[0m: 'space_camera' is not an extension registered with PySTAC" - ] - } - ], - "source": [ - "item_before.ext[Extensions.SPACE_CAMERA]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "So let's implement it!\n", + "class OrderEventType(StringEnum):\n", + " SUBMITTED = \"submitted\"\n", + " STARTED_PROCESSING = \"started_processing\"\n", + " DELIVERED = \"delivered\"\n", + " CANCELLED = \"cancelled\"\n", "\n", - "### Implementing an ItemExtension\n", "\n", - "We'll be referring to the [Satellite Extensions Specification](https://github.com/radiantearth/stac-spec/blob/v1.0.0-beta.2/extensions/sat/README.md) (referred to as the spec) to implement this extension.\n", + "class OrderEvent:\n", + " properties: Dict[str, Any]\n", "\n", - "The `sat` extension (or in our case `space_camera` extension) is scoped to `Item`. That information is found in the \"Scope\" line at the top of the spec. We'll want to implement an `CatalogExtension`, `CollectionExtension`, and `ItemExtension` for each of the STAC object types in the scope. In this case, we're only implementing an `ItemExtension`." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "from pystac.extensions.base import ItemExtension" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To implement the object extension, create a child class that implements each of the abstract methods of the relevant base class. For `ItemExtension`, the only required methods are listed below, along with an appropriate `__init__` method:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [], - "source": [ - "class SatItemExt(ItemExtension):\n", - " def __init__(self, item):\n", - " self.item = item\n", - " \n", - " @classmethod\n", - " def from_item(self, item):\n", - " return SatItemExt(item)\n", + " def __init__(self, properties: Dict[str, Any]) -> None:\n", + " self.properties = properties\n", + "\n", + " @property\n", + " def event_type(self) -> OrderEventType:\n", + " return get_required(\n", + " self.properties.get(\"type\"),\n", + " self,\n", + " \"event_type\"\n", + " )\n", + " \n", + " @event_type.setter\n", + " def event_type(self, v: OrderEventType) -> None:\n", + " self.properties[\"type\"] = str(v)\n", + " \n", + " @property\n", + " def timestamp(self) -> datetime:\n", + " return str_to_datetime(\n", + " get_required(\n", + " self.properties.get(\"timestamp\"),\n", + " self,\n", + " \"timestamp\"\n", + " )\n", + " )\n", + " \n", + " @timestamp.setter\n", + " def timestamp(self, v: datetime) -> None:\n", + " self.properties[\"timestamp\"] = datetime_to_str(v)\n", + "\n", + " def __repr__(self) -> str:\n", + " return \"\"\n", + "\n", + " def apply(\n", + " self,\n", + " event_type: OrderEventType,\n", + " timestamp: datetime,\n", + " ) -> None:\n", + " self.event_type = event_type\n", + " self.timestamp = timestamp\n", "\n", " @classmethod\n", - " def _object_links(cls):\n", - " return []" + " def create(\n", + " cls,\n", + " event_type: OrderEventType,\n", + " timestamp: datetime,\n", + " ) -> \"OrderEvent\":\n", + " oe = cls({})\n", + " oe.apply(\n", + " event_type=event_type,\n", + " timestamp=timestamp\n", + " )\n", + " return oe\n", + "\n", + " def to_dict(self) -> Dict[str, Any]:\n", + " return self.properties" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The `from_item` class method simply returns a new instance of the item extension given an item.\n", + "A few important notes about how we constructed this:\n", "\n", - "The `_object_links` class method returns the `rel` string for any links that point to STAC objects like Catalogs, Collections or Items. PySTAC needs to know which links point to STAC objects because it needs to consider them when fully resolving a STAC into in-memory objects. It also will use this information when deciding on whether to use absolute or relative HREFs for the links, based on the root catalog type. In a lot of cases, extensions don't add new links to STAC objects, so this is normally an empty list; however, if the extension does do this (like the `source` link in the [Label Extension](https://github.com/radiantearth/stac-spec/tree/v1.0.0-beta.2/extensions/label#links-source-imagery)), make sure to return the correct value (like the LabelItemExt is doing [here](https://github.com/azavea/pystac/blob/v0.5.0/pystac/extensions/label.py#L291-L293))." + "- We used PySTAC's [StringEnum class](https://pystac.readthedocs.io/en/latest/api/utils.html#pystac.utils.StringEnum), which inherits from the Python [Enum](https://docs.python.org/3/library/enum.html) class, to capture the allowed event type values. This class has built-in methods that will convert these instances to strings when serializing STAC Items to JSON.\n", + "- We use property getters and setters to manipulate a `properties` dictionary in our `OrderEvent` class. We will see later how this pattern allows us to mutate Item property dictionaries in-place so that updates to the `OrderEvent` object are synced to the Item they extend.\n", + "- The `timestamp` property is converted to a string before it is saved in the `properties` dictionary. This ensures that dictionary is always JSON-serializable but allows us to work with the values as a Python `datetime` instance when using the property getter.\n", + "- We use `event_type` as our property name so that we do not shadow the built-in `type` function in the `apply` method. However, this values is stored under the desired `\"type\"` key in the underlying `properties` dictionary." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Defining properties\n", - "\n", - "An extension object works by modifying the Item (or whichever STAC object is being extended) directly through Python [property getters and setters](https://docs.python.org/3/library/functions.html#property). The getter should read directly from the `properties` or `extra_fields` in the item and perform any transformations needed to convert to the relevant Python objects (e.g. transform a string into a `datetime` object). Likewise, the setter should take in Python objects and transform them to their serialized string, and set them in the appropriate place in item. This way the extension modifies the Item directly, and will not require any specialized serialization or deserialization logic. This also allows multiple extensions to be used to access and set information on the STAC object - a distinct advantage to the inheritance-based extension implementation that PySTAC used before 0.4.0.\n", - "\n", - "For the `sat` extension we have two properties to implement, both of which are straightforward and do not need any transformation in the getters and setters:" + "Next, we will create a new class inheriting from `PropertiesExtension` and `ExtensionManagementMixin`. Since this class only extends `pystac.Item` instance, we do not need to make it [generic](https://docs.python.org/3/library/typing.html#typing.Generic). If you were creating an extension that applied to multiple object types (e.g. `pystac.Item` and `pystac.Asset`) then you would need to inherit from `typing.Generic` as well and create concrete extension classed for each of these object types (see the [EOExtension](https://github.com/stac-utils/pystac/blob/3c5176f178a4345cb50d5dab83f1dab504ed2682/pystac/extensions/eo.py#L279), [ItemEOExtension](https://github.com/stac-utils/pystac/blob/3c5176f178a4345cb50d5dab83f1dab504ed2682/pystac/extensions/eo.py#L385), and [AssetEOExtension](https://github.com/stac-utils/pystac/blob/3c5176f178a4345cb50d5dab83f1dab504ed2682/pystac/extensions/eo.py#L429) classes for an example of this implementation)." ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ - "class SatItemExt(ItemExtension):\n", - " def __init__(self, item):\n", + "SCHEMA_URI: str = \"https://some-domain/image-order/v1.0.0/schema.json\"\n", + "PREFIX: str = \"order:\"\n", + "ID_PROP: str = PREFIX + \"id\"\n", + "HISTORY_PROP: str = PREFIX + \"history\"\n", + "\n", + "class OrderExtension(\n", + " PropertiesExtension,\n", + " ExtensionManagementMixin[Union[pystac.Item, pystac.Collection]]\n", + "):\n", + " def __init__(self, item: pystac.Item):\n", " self.item = item\n", - " \n", - " @property\n", - " def orbit_state(self):\n", - " \"\"\"\"ADD DOCSTRING!\"\"\"\n", - " return self.item.properties.get('sat:orbit_state')\n", - " \n", - " @orbit_state.setter\n", - " def orbit_state(self, v):\n", - " self.item.properties['sat:orbit_state'] = v\n", - " \n", - " @property\n", - " def relative_orbit(self):\n", - " \"\"\"\"ADD DOCSTRING!\"\"\"\n", - " return self.item.properties.get('sat:relative_orbit')\n", - " \n", - " @relative_orbit.setter\n", - " def relative_orbit(self, v):\n", - " self.item.properties['sat:relative_orbit'] = v\n", - " \n", - " @classmethod\n", - " def from_item(self, item):\n", - " return SatItemExt(item)\n", + " self.properties = item.properties\n", "\n", - " @classmethod\n", - " def _object_links(cls):\n", - " return []" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Extensions also define an `apply` method that encodes the optional and required values that go into the extension. The `apply` should list all of the values of the extension, and give default values of `None` to optional parameters. That way a users adding an extension to an object can easily tell what values are needed to implement the extension.\n", + " def apply(\n", + " self,\n", + " order_id: str = None,\n", + " history: Optional[List[OrderEvent]] = None\n", + " ) -> None:\n", + " self.order_id = order_id\n", + " self.history = history\n", "\n", - "Here we use the `apply` method to encode the requirement in the spec that at least one of `orbit_state` and `relative_orbit` need to be defined:" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [], - "source": [ - "class SatItemExt(ItemExtension):\n", - " def __init__(self, item):\n", - " self.item = item\n", - " \n", - " def apply(self, orbit_state=None, relative_orbit=None):\n", - " \"\"\"Applies Satellite extension properties to the extended Item.\n", - " \n", - " Args:\n", - " orbit_state (str): The state of the orbit. Either ascending or descending \n", - " for polar orbiting satellites, or geostationary for geosynchronous satellites\n", - " relative_orbit (int): The relative orbit number at the time of acquisition.\n", - " \n", - " Note:\n", - " At least one property must be supplied.\n", - " \"\"\"\n", - " if orbit_state is None and relative_orbit is None:\n", - " raise pystac.STACError(\"sat extension needs at least one property value.\")\n", - " \n", - " self.orbit_state = orbit_state\n", - " self.relative_orbit = relative_orbit\n", - " \n", " @property\n", - " def orbit_state(self):\n", - " return self.item.properties.get('sat:orbit_state')\n", + " def order_id(self) -> str:\n", + " return get_required(self._get_property(ID_PROP, str), self, ID_PROP)\n", " \n", - " @orbit_state.setter\n", - " def orbit_state(self, v):\n", - " self.item.properties['sat:orbit_state'] = v\n", - " \n", + " @order_id.setter\n", + " def order_id(self, v: str) -> None:\n", + " self._set_property(ID_PROP, v, pop_if_none=False)\n", + "\n", " @property\n", - " def relative_orbit(self):\n", - " return self.item.properties.get('sat:relative_orbit')\n", + " def history(self) -> Optional[List[OrderEvent]]:\n", + " return map_opt(\n", + " lambda history: [OrderEvent(d) for d in history],\n", + " self._get_property(HISTORY_PROP, List[OrderEvent])\n", + " )\n", " \n", - " @relative_orbit.setter\n", - " def relative_orbit(self, v):\n", - " self.item.properties['sat:relative_orbit'] = v\n", - " \n", + " @history.setter\n", + " def history(self, v: Optional[List[OrderEvent]]) -> None:\n", + " self._set_property(\n", + " HISTORY_PROP,\n", + " map_opt(lambda history: [event.to_dict() for event in history], v),\n", + " pop_if_none=True\n", + " )\n", + "\n", " @classmethod\n", - " def from_item(self, item):\n", - " return SatItemExt(item)\n", + " def get_schema_uri(cls) -> str:\n", + " return SCHEMA_URI\n", "\n", " @classmethod\n", - " def _object_links(cls):\n", - " return []" + " def ext(cls, obj: pystac.Item, add_if_missing: bool = False) -> \"OrderExtension\":\n", + " if isinstance(obj, pystac.Item):\n", + " cls.validate_has_extension(obj, add_if_missing)\n", + " return OrderExtension(obj)\n", + " else:\n", + " raise pystac.ExtensionTypeError(\n", + " f\"OrderExtension does not apply to type '{type(obj).__name__}'\"\n", + " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Now that we have our object extension we need to register it with PySTAC. To do so we'll need to define an `ExtendedObject` to tie together the PySTAC object we are extending and our `SatItemExt` class:" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": {}, - "outputs": [], - "source": [ - "from pystac.extensions.base import ExtendedObject" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": {}, - "outputs": [], - "source": [ - "extended_object = ExtendedObject(pystac.Item, SatItemExt)" + "As with the `OrderEvent` class, we use property getters and setters for our extension fields (the `PropertiesExtension` class has a `properties` attribute where these are stored). Rather than setting these values directly in the dictionary, we use the `_get_property` and `_set_property` methods that are built into the `PropertiesExtension` class). We also add an `ext` method that will be used to extend `pystac.Item` instances, and a `get_schema_uri` method that is required for all `PropertiesExtension` classes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Then we define an `ExtensionDefinition` that ties together our extension ID with the list of object extensions. In this case, we are only extending Item and so there's only a single entry in the list:" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [], - "source": [ - "from pystac.extensions.base import ExtensionDefinition" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [], - "source": [ - "extension_definition = ExtensionDefinition(Extensions.SPACE_CAMERA, [extended_object])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For common extensions this definition usually happens at the end of the extension file all in one line; see [this example](https://github.com/azavea/pystac/blob/v0.5.0/pystac/extensions/label.py#L656-L657)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we can register the extension definition with PySTAC. For common extensions defined in the library [you would add it to the list in the top level package __init__](https://github.com/azavea/pystac/blob/v0.5.1/pystac/__init__.py#L32-L43). However if you're creating a custom extension you can use the following method:" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": {}, - "outputs": [], - "source": [ - "pystac.STAC_EXTENSIONS.add_extension(extension_definition)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Remember, if you are implementing an extension in PySTAC, make sure to add thorough unit tests ([example](https://github.com/azavea/pystac/blob/v0.5.0/tests/extensions/test_view.py)) and add the extension to the documentation ([example](https://github.com/azavea/pystac/blob/v0.5.0/docs/api.rst#view-geometry-extension))!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Using the extension\n", + "## Use the Extension\n", "\n", - "When we read the item (again manipulating the JSON so that the `sat` extension ID turns into `space_camera`), we can now access the extension functionality through the same means as the other extensions:" + "Let's try using our new classes to extend and `Item` and access the extension properties. We'll start by loading the core Item example from the STAC spec examples [here](https://github.com/radiantearth/stac-spec/blob/master/examples/core-item.json) and printing the existing properties." ] }, { "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [], - "source": [ - "item_after = read_item('https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0-beta.2/extensions/sat/examples/example-landsat8.json')" - ] - }, - { - "cell_type": "code", - "execution_count": 17, + "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "'ascending'" + "{'title': 'Core Item',\n", + " 'description': 'A sample STAC Item that includes examples of all common metadata',\n", + " 'datetime': None,\n", + " 'start_datetime': '2020-12-11T22:38:32.125Z',\n", + " 'end_datetime': '2020-12-11T22:38:32.327Z',\n", + " 'created': '2020-12-12T01:48:13.725Z',\n", + " 'updated': '2020-12-12T01:48:13.725Z',\n", + " 'platform': 'cool_sat1',\n", + " 'instruments': ['cool_sensor_v1'],\n", + " 'constellation': 'ion',\n", + " 'mission': 'collection 5624',\n", + " 'gsd': 0.512}" ] }, - "execution_count": 17, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "item_after.ext[Extensions.SPACE_CAMERA].orbit_state" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": {}, - "outputs": [], - "source": [ - "item_after.ext.space_camera.relative_orbit = 5" + "item = pystac.read_file('https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/core-item.json')\n", + "item.properties\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Notice that setting the property value through the extension sets the correct item property:" + "Next, let's verify that this Item does not implement our new Order Extension yet and that it does not already contain any of our Order Extension properties." ] }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 5, "metadata": {}, "outputs": [ { - "data": { - "text/plain": [ - "5" - ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" + "name": "stdout", + "output_type": "stream", + "text": [ + "Implements Extension: False\n", + "Order ID: None\n", + "History:\n" + ] } ], "source": [ - "item_after.properties['sat:relative_orbit']" + "print(f\"Implements Extension: {OrderExtension.has_extension(item)}\")\n", + "print(f\"Order ID: {item.properties.get(ID_PROP)}\")\n", + "print(\"History:\")\n", + "for event in item.properties.get(HISTORY_PROP, []):\n", + " pprint(event)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "We can also read in an item that does not already implement the extension, enable it, and use the `apply` method to fill out the values:" + "As expected, this Item does not implement the extension (i.e. the schema URI is not in the Item's `stac_extensions` list). Let's add it, create an instance of `OrderExtension` that extends the `Item`, and add some values for our extension fields." ] }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 6, "metadata": {}, "outputs": [], "source": [ - "item3 = pystac.read_file('https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0-beta.2/item-spec/examples/sample-full.json')" + "order_ext = OrderExtension.ext(item, add_if_missing=True)\n", + "\n", + "# Create a unique string ID for the order ID\n", + "order_ext.order_id = str(uuid4())\n", + "\n", + "# Create some fake order history and set it using the extension\n", + "event_1 = OrderEvent.create(\n", + " event_type=OrderEventType.SUBMITTED,\n", + " timestamp=datetime.now() - timedelta(days=1)\n", + ")\n", + "event_2 = OrderEvent.create(\n", + " event_type=OrderEventType.STARTED_PROCESSING,\n", + " timestamp=datetime.now() - timedelta(hours=12)\n", + ")\n", + "event_3 = OrderEvent.create(\n", + " event_type=OrderEventType.DELIVERED,\n", + " timestamp=datetime.now() - timedelta(hours=1)\n", + ")\n", + "order_ext.history = [event_1, event_2, event_3]\n" ] }, { - "cell_type": "code", - "execution_count": 21, + "cell_type": "markdown", "metadata": {}, - "outputs": [], "source": [ - "item3.ext.enable(Extensions.SPACE_CAMERA)" + "Now let's check to see if these values were written to our Item properties." ] }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Help on method apply in module __main__:\n", - "\n", - "apply(orbit_state=None, relative_orbit=None) method of __main__.SatItemExt instance\n", - " Applies Satellite extension properties to the extended Item.\n", - " \n", - " Args:\n", - " orbit_state (str): The state of the orbit. Either ascending or descending \n", - " for polar orbiting satellites, or geostationary for geosynchronous satellites\n", - " relative_orbit (int): The relative orbit number at the time of acquisition.\n", - " \n", - " Note:\n", - " At least one property must be supplied.\n", - "\n" + "Implements Extension: True\n", + "Order ID: f6f367b6-a787-48de-941f-08a04be77683\n", + "History:\n", + "{'timestamp': '2022-01-20T11:44:01.803820Z', 'type': 'submitted'}\n", + "{'timestamp': '2022-01-20T23:44:01.803939Z', 'type': 'started_processing'}\n", + "{'timestamp': '2022-01-21T10:44:01.803999Z', 'type': 'delivered'}\n" ] } ], "source": [ - "help(item3.ext.space_camera.apply)" - ] - }, - { - "cell_type": "code", - "execution_count": 23, - "metadata": {}, - "outputs": [], - "source": [ - "item3.ext.space_camera.apply(relative_orbit='ascending')" + "print(f\"Implements Extension: {OrderExtension.has_extension(item)}\")\n", + "print(f\"Order ID: {item.properties.get(ID_PROP)}\")\n", + "print(\"History:\")\n", + "for event in item.properties.get(HISTORY_PROP, []):\n", + " pprint(event)" ] }, { @@ -534,9 +400,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.5" + "version": "3.8.9" } }, "nbformat": 4, "nbformat_minor": 2 -} \ No newline at end of file +} From 2a8080c4b5f118b299389c0a5bcf373ff03e468d Mon Sep 17 00:00:00 2001 From: Jon Duckworth Date: Wed, 26 Jan 2022 13:53:38 -0500 Subject: [PATCH 2/5] Update intro --- docs/tutorials/adding-new-and-custom-extensions.ipynb | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/tutorials/adding-new-and-custom-extensions.ipynb b/docs/tutorials/adding-new-and-custom-extensions.ipynb index 457fab0e3..f691e956d 100644 --- a/docs/tutorials/adding-new-and-custom-extensions.ipynb +++ b/docs/tutorials/adding-new-and-custom-extensions.ipynb @@ -6,10 +6,7 @@ "source": [ "## Adding New and Custom Extensions\n", "\n", - "This tutorial will cover how to implement a new extensions to PySTAC. It will go over:\n", - "\n", - "- Using the `PropertiesExtension` and `ExtensionManagementMixin` classes to extend STAC objects\n", - "- Using the `SummariesExtension` class to extend Collection summaries\n", + "This tutorial will cover using the `PropertiesExtension` and `ExtensionManagementMixin` classes in `pystac.extensions.base` to implement a new extension to PySTAC.\n", "\n", "For this exercise, we will implement an imaginary Order Request Extension that allows us to track an internal order ID associated with a given satellite image, as well as the history of that imagery order. This use-case is specific enough that it would probably not be a good candidate for an actual STAC Extension, but it gives us an opportunity to highlight some of the key aspects and patterns used in implementing STAC Extensions in PySTAC." ] From ac3c992b9daf51c9465e0bb00f8d066cf815095a Mon Sep 17 00:00:00 2001 From: Jon Duckworth Date: Wed, 26 Jan 2022 13:53:38 -0500 Subject: [PATCH 3/5] Match tutorials TOC order to page order --- docs/tutorials.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/tutorials.rst b/docs/tutorials.rst index 6c1dc5d74..d7e6a972e 100644 --- a/docs/tutorials.rst +++ b/docs/tutorials.rst @@ -55,4 +55,8 @@ implement your own custom extensions. :maxdepth: 2 :glob: - tutorials/** + tutorials/pystac-introduction.ipynb + tutorials/pystac-spacenet-tutorial.ipynb + tutorials/how-to-create-stac-catalogs.ipynb + tutorials/creating-a-landsat-stac.ipynb + tutorials/adding-new-and-custom-extensions.ipynb From 95ceb19ddb6083fc98bd1cbec4cf66e5868cb1ce Mon Sep 17 00:00:00 2001 From: Jon Duckworth Date: Wed, 26 Jan 2022 13:53:38 -0500 Subject: [PATCH 4/5] Update dummy schema URI and fix typo --- docs/tutorials/adding-new-and-custom-extensions.ipynb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/adding-new-and-custom-extensions.ipynb b/docs/tutorials/adding-new-and-custom-extensions.ipynb index f691e956d..9651ea566 100644 --- a/docs/tutorials/adding-new-and-custom-extensions.ipynb +++ b/docs/tutorials/adding-new-and-custom-extensions.ipynb @@ -169,7 +169,7 @@ "metadata": {}, "outputs": [], "source": [ - "SCHEMA_URI: str = \"https://some-domain/image-order/v1.0.0/schema.json\"\n", + "SCHEMA_URI: str = \"https://example.com/image-order/v1.0.0/schema.json\"\n", "PREFIX: str = \"order:\"\n", "ID_PROP: str = PREFIX + \"id\"\n", "HISTORY_PROP: str = PREFIX + \"history\"\n", @@ -241,7 +241,7 @@ "source": [ "## Use the Extension\n", "\n", - "Let's try using our new classes to extend and `Item` and access the extension properties. We'll start by loading the core Item example from the STAC spec examples [here](https://github.com/radiantearth/stac-spec/blob/master/examples/core-item.json) and printing the existing properties." + "Let's try using our new classes to extend an `Item` and access the extension properties. We'll start by loading the core Item example from the STAC spec examples [here](https://github.com/radiantearth/stac-spec/blob/master/examples/core-item.json) and printing the existing properties." ] }, { From 3495885fb1f520c3394d8c21d4eb081f5e862732 Mon Sep 17 00:00:00 2001 From: Jon Duckworth Date: Wed, 26 Jan 2022 13:54:15 -0500 Subject: [PATCH 5/5] Add CHANGELOG entry for #724 --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index d05ce5dff..6be86bb8e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ ### Fixed - Self links no longer included in Items for "relative published" catalogs ([#725](https://github.com/stac-utils/pystac/pull/725)) +- Adding New and Custom Extensions tutorial now up-to-date with new extensions API ([#724](https://github.com/stac-utils/pystac/pull/724)) ### Deprecated