OpenApi Schema¶
CodeGenerator is able to generate OpenAPI specs for certain services by inspecting their code. This requires service package being installed in the environment where the generator is running. It then tries to initialize service application and for supported runtimes scans for the exposed operations. At the moment following services are covered:
Nova
Neutron
Cinder
Glance
Keystone
Octavia
Generator can be invoked after installing it as a regular python project with dependencies
openstack-codegenerator --target openapi-spec --work-dir wrk --service-type compute
The generator is having possibility to additionally parse rendered service API-REF HTML documentation and supplement descriptions in the generated OpenApi spec by trying to find corresponding information in the html.
openstack-codegenerator --target openapi-spec --work-dir wrk --service-type compute --api-ref-src <PATH_TO_RENDERED_DOC>.html
Another project for rendering generated OpenAPI specs in the style similar (but not the same way) to currently used os-api-ref: https://github.com/gtema/openstack-openapi. It implements a Sphinx extension that reads spec file and converts it to internal sphinx directives to get a proper structure in the rendered HTML and styles it using BootstrapV5 library. Sample rendering can be seen under https://gtema.github.io/openstack-openapi/
Highlevel description (for contributor)¶
Base generator
OpenStackServerSourceGenerator
is
supporting WSGI + Routes based application out of box. For such applications
it tries to get the main router from wich all exposed routes are being
analysed. During routes processing generator is searching for supported
decorators and frameworks in order to extract as most information about the
operation as possible:
url
method
path parameters
query parameters
expected body jsonschema
response jsonschema
expected response and error codes
operation description (docstrings)
Generator for every covered OpenStack service is inherits from the Base
generator (i.e. NovaGenerator
. It is
expected that init method will perform service setup activities (i.e.
database creation or config file preparation whenever required) and sets the
main application router. generate method of the class is then being invoked
and it reads current spec file (if present to update it) and loops over all
exposed routes. For each route a dedicated method _process_route is
invoked, which in turn invoke multiple additional methods for parameters or
body schemas processing.
After processing when api-ref html is available a dedicated method
merge_api_ref_doc
can be called to add
available descriptions (operation, parameters).
Note
Since all services use oslo_config and oslo_policy libraries which rely on global state they race with each other. In order to avoid this processing rely on multiprocessing to isolate services.
Nova¶
Source code of Nova currently provides full information about exposed routes
and query/path parameters, as well as jsonschema of request body. Sadly it does
not contain jsonschemas of the responses. CodeGenerator at the moment covers
those missing schemas directly in the code and injects them into the schema via
NovaGenerator:_get_schema_ref
After stabilization it is expected to move implemented schemas into the Nova source code base.
Cinder¶
Cinder is very similar to Nova so everything mentioned above is applicable here as well.
for Cinder at the moment all operations are duplicated under v3/${project_id}/… and v3/…. For the sake of standartization project_id urls are excluded from the produces spec file.
Glance¶
Glance is also using routes for exposing application. However in difference to Nova and Cinder it does not describe request parameters of bodies in an expected way. Current implementation of the Glance generator therefore is looking at the request serializer and deserializer attached to the operation controllers. When this information is present and contain usable jsonschema it is being used. In other cases similar approach to Nova with hardcoding response information is being used. But since Glance code base contain certain useful jsonschemas (not connected in the routes) generator gets those schemas directly from the code (where the mapping is known).
Keystone¶
This service is using Flask framework which gives similar capabilities to the routes. However here there are no body information at all (neither Request nor Response). Also here there are certain jsonschemas found directly in the Keystone code base and connected for the schema generation.
Neutron¶
This is where things are getting more challenging.
Neutron requires having DB provisioned and an in-memory DB seems not to be
possible due to technics for the DB communication. In addition to that config
file enabling desired extensions is expected. All this activities are covered
in NeutronGenerator:setup_neutron
.
According to the current information it is not possible to have all possible
Neutron extensions and plugins enabled at the same time. This is solved by
generator spinning multiple subprocesses that bootstrap Neutron with different
configuration and then merge results. This is handled by spinning up Neutron
few times with independent configurations and merging resulting spec.
Additional challenge in Neutron is that it does not use routes to expose operations directly, but is having a mix of routes based operations for extensions and pecan app for the base functionality. Since the pecan framework is based on a purely dynamic routing there is no possibility to extract information about exposed routes by doing code inspection. Luckily only base operations (router/net/subnet) are implemented this way. Therefore generator registers known pecan operations into the extensions router and normal generator flow is being invoked.
Next challenge is that for Neutron there is no description of bodies at all,
but certain controllers are having API_DEFINITION attached. While this is not
a jsonschema at all it can be used to create one where possible. Sadly there is
still sometime no possibility to properly estimate whether certain operation is
exposed and functioning or it is exposed but fails permanently due to the fact,
that API_DEFINITION extrapolation fails for this operation.
get_schema
method is responsible for
conversion of the API_DEFINITION into the jsonschema, but is not able to work
perfectly until additional work is invested.
Certain additional operations (addRouterInterface, addExtraRoute, …) are not having any information available and require to be also hardcodede in the generator.
Octavia¶
Octavia is also based on the pecan with its dynamic routing, but the
majority of controllers are available for scanning due to the source code
classes hierarchy. To keep the generation process close to generics
OctaviaGenerator
is constructing
routes router from this information and adds few known exceptions. For the
produced routing table generic process is being invoked which is then looking
at the WSME decorators attached to the exposed operations. Since WSME
schema is not a jsonschema on its own but it can be considered as an
alternative to jsonschema a naive conversion is implemented in
_convert_wsme_to_jsonschema
.