Skip to content

main_index

The OPTIMADE Index Meta-Database server

The server is based on MongoDB, using either pymongo or mongomock.

This is an example implementation with example data. To implement your own index meta-database server see the documentation at https://optimade.org/optimade-python-tools.

BASE_URL_PREFIXES = {'major': f'/v{__api_version__.split('-')[0].split('+')[0].split('.')[0]}', 'minor': f'/v{'.'.join(__api_version__.split('-')[0].split('+')[0].split('.')[:2])}', 'patch': f'/v{'.'.join(__api_version__.split('-')[0].split('+')[0].split('.')[:3])}'} module-attribute

CONFIG: ServerConfig = ServerConfig() module-attribute

This singleton loads the config from a hierarchy of sources (see customise_sources) and makes it importable in the server code.

DEFAULT_CONFIG_FILE_PATH: str = str(Path.home().joinpath('.optimade.json')) module-attribute

Default configuration file path.

This variable is used as the fallback value if the environment variable OPTIMADE_CONFIG_FILE is not set.

Note

It is set to: pathlib.Path.home()/.optimade.json

For Unix-based systems (Linux) this will be equivalent to ~/.optimade.json.

LOGGER = logging.getLogger('optimade') module-attribute

OPTIMADE_EXCEPTIONS: Iterable[tuple[type[Exception], Callable[[Request, Exception], JSONAPIResponse]]] = [(StarletteHTTPException, http_exception_handler), (OptimadeHTTPException, http_exception_handler), (RequestValidationError, request_validation_exception_handler), (ValidationError, validation_exception_handler), (VisitError, grammar_not_implemented_handler), (NotImplementedError, not_implemented_handler), (Exception, general_exception_handler)] module-attribute

A tuple of all pairs of exceptions and handler functions that allow for appropriate responses to be returned in certain scenarios according to the OPTIMADE specification.

To use these in FastAPI app code:

from fastapi import FastAPI
app = FastAPI()
for exception, handler in OPTIMADE_EXCEPTIONS:
    app.add_exception_handler(exception, handler)

OPTIMADE_MIDDLEWARE: Iterable[BaseHTTPMiddleware] = (EnsureQueryParamIntegrity, CheckWronglyVersionedBaseUrls, HandleApiHint, AddWarnings) module-attribute

A tuple of all the middleware classes that implement certain required features of the OPTIMADE specification, e.g. warnings and URL versioning.

Note

The order in which middleware is added to an application matters.

As discussed in the docstring of AddWarnings, this middleware is the final entry to this list so that it is the first to be applied by the server. Any other middleware should therefore be added before iterating through this variable. This is the opposite way around to the example in the Starlette documentation which initialises the application with a pre-built middleware list in the reverse order to OPTIMADE_MIDDLEWARE.

To use this variable in FastAPI app code after initialisation:

from fastapi import FastAPI
app = FastAPI()
for middleware in OPTIMADE_MIDDLEWARE:
    app.add_middleware(middleware)

Alternatively, to use this variable on initialisation:

from fastapi import FastAPI
from starlette.middleware import Middleware
app = FastAPI(
    ...,
    middleware=[Middleware(m) for m in reversed(OPTIMADE_MIDDLEWARE)]
)

__api_version__ = '1.1.0' module-attribute

__version__ = '1.0.1' module-attribute

app = FastAPI(root_path=CONFIG.root_path, title='OPTIMADE API - Index meta-database', description=f'The [Open Databases Integration for Materials Design (OPTIMADE) consortium](https://www.optimade.org/) aims to make materials databases interoperational by developing a common REST API.This is the "special" index meta-database.This specification is generated using [`optimade-python-tools`](https://github.com/Materials-Consortia/optimade-python-tools/tree/v{__version__}) v{__version__}.', version=__api_version__, docs_url=f'{BASE_URL_PREFIXES['major']}/extensions/docs', redoc_url=f'{BASE_URL_PREFIXES['major']}/extensions/redoc', openapi_url=f'{BASE_URL_PREFIXES['major']}/extensions/openapi.json', default_response_class=JSONAPIResponse, separate_input_output_schemas=False, lifespan=lifespan) module-attribute

config_warnings = w module-attribute

data = json.load(f) module-attribute

processed = [] module-attribute

providers = get_providers(add_mongo_id=True) module-attribute

JSONAPIResponse

Bases: JSONResponse

This class simply patches fastapi.responses.JSONResponse to use the JSON:API 'application/vnd.api+json' MIME type.

Source code in optimade/server/routers/utils.py 
41
42
43
44
45
46
47
class JSONAPIResponse(JSONResponse):
    """This class simply patches `fastapi.responses.JSONResponse` to use the
    JSON:API 'application/vnd.api+json' MIME type.

    """

    media_type = "application/vnd.api+json"

add_major_version_base_url(app)

Add mandatory endpoints to /vMAJOR base URL.

Source code in optimade/server/main_index.py 
133
134
135
136
def add_major_version_base_url(app: FastAPI):
    """Add mandatory endpoints to `/vMAJOR` base URL."""
    for endpoint in (index_info, links):
        app.include_router(endpoint.router, prefix=BASE_URL_PREFIXES["major"])

add_optional_versioned_base_urls(app)

Add the following OPTIONAL prefixes/base URLs to server: 

    /vMajor.Minor
    /vMajor.Minor.Patch

Source code in optimade/server/main_index.py 
139
140
141
142
143
144
145
146
147
148
def add_optional_versioned_base_urls(app: FastAPI):
    """Add the following OPTIONAL prefixes/base URLs to server:
    ```
        /vMajor.Minor
        /vMajor.Minor.Patch
    ```
    """
    for version in ("minor", "patch"):
        app.include_router(index_info.router, prefix=BASE_URL_PREFIXES[version])
        app.include_router(links.router, prefix=BASE_URL_PREFIXES[version])

get_providers(add_mongo_id=False)

Retrieve Materials-Consortia providers (from https://providers.optimade.org/v1/links).

Fallback order if providers.optimade.org is not available:

  1. Try Materials-Consortia/providers on GitHub.
  2. Try submodule providers' list of providers.
  3. Log warning that providers list from Materials-Consortia is not included in the /links-endpoint.

Parameters:

Name Type Description Default
add_mongo_id bool

Whether to populate the _id field of the provider with MongoDB ObjectID.

 False

Returns:

Type Description
list

List of raw JSON-decoded providers including MongoDB object IDs.
Source code in optimade/utils.py 
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
def get_providers(add_mongo_id: bool = False) -> list:
    """Retrieve Materials-Consortia providers (from https://providers.optimade.org/v1/links).

    Fallback order if providers.optimade.org is not available:

    1. Try Materials-Consortia/providers on GitHub.
    2. Try submodule `providers`' list of providers.
    3. Log warning that providers list from Materials-Consortia is not included in the
       `/links`-endpoint.

    Arguments:
        add_mongo_id: Whether to populate the `_id` field of the provider with MongoDB
            ObjectID.

    Returns:
        List of raw JSON-decoded providers including MongoDB object IDs.

    """
    import json

    import requests

    for provider_list_url in PROVIDER_LIST_URLS:
        try:
            providers = requests.get(provider_list_url).json()
        except (
            requests.exceptions.ConnectionError,
            requests.exceptions.ConnectTimeout,
            json.JSONDecodeError,
        ):
            pass
        else:
            break
    else:
        try:
            from optimade.server.data import providers  # type: ignore
        except ImportError:
            from optimade.server.logger import LOGGER

            LOGGER.warning(
                """Could not retrieve a list of providers!

    Tried the following resources:

{}
    The list of providers will not be included in the `/links`-endpoint.
""".format(
                    "".join([f"    * {_}\n" for _ in PROVIDER_LIST_URLS])
                )
            )
            return []

    providers_list = []
    for provider in providers.get("data", []):
        # Remove/skip "exmpl"
        if provider["id"] == "exmpl":
            continue

        provider.update(provider.pop("attributes", {}))

        # Add MongoDB ObjectId
        if add_mongo_id:
            provider["_id"] = {
                "$oid": mongo_id_for_database(provider["id"], provider["type"])
            }

        providers_list.append(provider)

    return providers_list

lifespan(app) async

Add dynamic endpoints and adjust config on startup.

Source code in optimade/server/main_index.py 
43
44
45
46
47
48
49
50
51
52
53
@asynccontextmanager  # type: ignore[arg-type]
async def lifespan(app: FastAPI):
    """Add dynamic endpoints and adjust config on startup."""
    CONFIG.is_index = True
    # Add API endpoints for MANDATORY base URL `/vMAJOR`
    add_major_version_base_url(app)
    # Add API endpoints for OPTIONAL base URLs `/vMAJOR.MINOR` and `/vMAJOR.MINOR.PATCH`
    add_optional_versioned_base_urls(app)

    # Yield so that the app can start
    yield

mongo_id_for_database(database_id, database_type)

Produce a MongoDB ObjectId for a database

Source code in optimade/utils.py 
24
25
26
27
28
29
30
31
32
33
34
def mongo_id_for_database(database_id: str, database_type: str) -> str:
    """Produce a MongoDB ObjectId for a database"""
    from bson.objectid import ObjectId

    oid = f"{database_id}{database_type}"
    if len(oid) > 12:
        oid = oid[:12]
    elif len(oid) < 12:
        oid = f"{oid}{'0' * (12 - len(oid))}"

    return str(ObjectId(oid.encode("UTF-8")))