OpenStack Client Tools

Welcome to the world of next generation OpenStack client tools written in Rust.

As a programming language Rust is getting more and more traction in the low level programming. It has very unique programming safety features what makes it a very good fit in a complex world of OpenStack. As a compiled language it is also a very good fit for the CLI tools allowing users to escape the python dependency issues. In the containerization era placing a small size binary is so much easier.

Current focus of the project is at introducing Rust as a programming language into the ecosystem of OpenStack user facing tooling and provides SDK as well as CLI and TUI.

Design principles

After long time maintaining OpenStack client facing tools it became clear that it cannot continue the same way as before. OpenStack services are very different and are not behaving similar to each other. There were multiple attempts to standardize APIs across services, but it didn't worked. Then there were attempts to try to standardize services on the SDK level. This is partially working, but require a very high implementation effort and permanent maintenance. Reverse-engineering service API by looking at the API-REF documentation is very time consuming. A huge issue is also that API-REF is being a human-written document that very often diverges from the code what leads to the issues when covering those resources in SDK/CLI. Tracking the API evolving is another aspect of the maintenance effort.

As a solution a completely different approach has been chosen to reduce maintenance effort while at the same time guaranteeing that API bindings match to what service is supporting in reality. Instead of human reading the API-REF written by another human who maybe was involved in the implementation of the feature OpenAPI specs is being chosen as a source of truth. Since such specs were also not existing and multiple attempts to introduce OpenAPI in OpenStack failed the process was restarted again. Currently there is a lot of work happening in OpenStack to produce specs for majority of the services. Main component responsible for that is codegenerator. Apart of inspecting source code of the selected OpenStack services it is also capable of generating tools in this repository. There is of course a set of the framework code, but the REST API wrapping and commands implementation is fully generated.

At the end of the day it means that there is no need to touch the generated code at all. Once resources available in the OpenAPI spec of the service are being initially integrated into the subprojects here they become maintained by the generator. New features added into the resource by the service would be automatically updated once OpenAPI spec is being updated.

Generating code from the OpenAPI has another logical consequence: generated code is providing the same features as the API itself. So if API is doing thing not very logical the SDK/CLI will do it in the same way. Previously it was always landing on the shoulders of SDK/CLI maintainers to try to cope with it. Now if API is bad - API author is to blame.

Code being automatically generated from OpenAPI specs of the service APIs.
Unix philosophy: "do one thing well". Every resource/command coverage tries to focus only on the exact API. Combination of API calls is not in scope of the generated code. "Simple is better then complex" (The Zen of Python).
SDK/CLI bindings are wrapping the API with no additional guessing or normalization.
User is in full control of input and output. Microversion X.Y has a concrete body schema and with no faulty merges between different versions.

Components

openstack_sdk - SDK
openstack_cli - The new and shiny CLI for OpenStack
openstack_tui - Text (Terminal) User Interface
structable_derive - Helper crate for having Output in some way similar to old OpenStackClient

Trying out

Install binary

It is possible to install compiled version from the GitHub releases. It comes with a dedicated installer in every release and can be retrieved with the following command:

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/gtema/openstack/releases/download/openstack_cli-v0.8.2/openstack_cli-installer.sh | sh

TUI can be installed similarly:

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/gtema/openstack/releases/download/openstack_tui-v0.1.6/openstack_tui-installer.sh | sh

Build locally

Alternatively it is possible to compile project from sources. Since the project is a pure Rust it requires having a Rust compile suite.

cargo b

Run

Once the binary is available just start playing with it. If you already have your clouds.yaml config file from python-openstackclient you are free to go:

osc --help
osc --os-cloud devstack compute flavor list

Authentication and authorization

Understanding authentication in OpenStack is far away from being a trivial task. In general authentication and authorization are different things, but they are mixed into 1 API request. When trying to authenticate user is passing identification data (username and password or similar) together with the requested authorization (project scope, domain scope or similar). As a response to this API request a session Token is being returned to the user that needs to be always sent with any following request. When authorization scope need to be changed (i.e. perform API in a scope of a different project) a re-authorization need to be performed. That may be done with the same user identification data or using existing session token.

Existing python based OpenStack tools are keeping only one active session going through re-authorization whenever required. There is a support for the token caching which is bound to the used authorization request. When a new session is requested a search (when enabled) is being performed in the cache and a matching token is being returned which is then retried. When MFA or SSO are being used this process is introducing a very ugly user experience forcing user to re-entrer verification data every time a new scope is being requested with a new session.

In this project authentication and authorization in user facing applications is handled differently. Auth caching to the file system is enabled by default. User identification data (auth_url + user name + user domain name) is combined into the hash which is then used as a 1st level caching key and points to the hashmap of authorization data and corresponding successful authorization response data. A key in the 2nd level is a hash calculated from the scope information and a value is a returned token with catalog and expiration information. This way of handling information allows to immediately retrieve valid auth information for the requested scope if it already exists (which may be even shared between processes) or reuse valid authentication data to get new valid requested authorization saving user from need to re-process MFA or SSO requirements.

Configuration

Rust based tools support typical clouds.yaml/secure.yaml files for configuration.

Authentication methods

Currently only a subset of all possible authentication methods is covered with the work on adding further method ongoing

v3Token

A most basic auth method is an API token (X-Auth-Token). In the clouds.yaml this requires setting auth_type to one of the [v3token, token]. The token itself should be specified in the token attribute.

v3Password

A most common auth method is a username/password. In the clouds.yaml this requires setting auth_type to one of the [v3password, password] or leaving it empty.

Following attributes specify the authentication data:

username - The user name
user_id - The user ID
password - The user password
user_domain_name - The name of the domain user belongs to
user_domain_id - The ID of the domain user belongs to

It is required to specify username or user_id as well as user_domain_name or user_domain_id.

v3Totp

Once user login is protected with the MFA a OTP token must be specified. It is represented as passcode, but it not intended to be used directly in the clouds.yaml

v3Multifactor

A better way to handle MFA is by using a v3multifactor auth type. In this case configuration looks a little bit different:

auth_type = v3multifactor
auth_methods is a list of individual auth_types combined in the authentication flow (i.e ['v3password', 'v3totp'])

When a cloud connection is being established in an interactive mode and server responds that it require additional authentication methods those would be processed based on the available data.

v3WebSso

An authentication method that is getting more a more popular is a Single Sign On using remote Identity Data Provider. This flow requires user to authenticate itself in the browser by the IDP directly. It is required to provide following data in the configuration in order for this mode to be used:

auth_type = v3websso
identity_provider - identity provider as configured in the Keystone
protocol - IDP protocol as configured in the Keystone

Note: This authentication type only works in the interactive mode. That means in the case of the CLI that there must be a valid terminal (echo foo | osc identity user create will not work)

v3ApplicationCredential

Application credentials provide a way to delegate a user’s authorization to an application without sharing the user’s password authentication. This is a useful security measure, especially for situations where the user’s identification is provided by an external source, such as LDAP or a single-sign-on service. Instead of storing user passwords in config files, a user creates an application credential for a specific project, with all or a subset of the role assignments they have on that project, and then stores the application credential identifier and secret in the config file.

Multiple application credentials may be active at once, so you can easily rotate application credentials by creating a second one, converting your applications to use it one by one, and finally deleting the first one.

Application credentials are limited by the lifespan of the user that created them. If the user is deleted, disabled, or loses a role assignment on a project, the application credential is deleted.

Required configuration:

auth_type = v3applicationcredential
application_credential_secret - a secret part of the application credential
application_credential_id - application credential identity
application_credential_name - application credential name. Note: It is required to specify user data when using application credential name
user_id - user ID when application_credential_name is used
user_name - user name when application_credential_name is used
user_domain_id - User domain ID when application_credential_name is used
user_domain_name - User domain ID when application_credential_name is used

Either application_credential_id is required or application_credential_name in which case additionally the user information is required.

Caching

As described above in difference to the Python OpenStack tooling authentication caching is enabled by default. It can be disabled using cache.auth: false in the clouds.yaml.

Data is cached locally in the ~/.osc folder. It is represented by set of files where file name is constructed as a hash of authentication information (discarding sensitive data). Content of the file is a serialized map of authorization data (scope) with the token information (catalog, expiration, etc).

Every time a new connection need to be established first a search in the cache is performed to find an exact match using supplied authentication and authorization information. When there is no usable information (no information at all or cached token is already expired) a search is performed for any valid token ignoring the scope (authz). When a valid token is found in the cache it is used to obtain a new authorization with required scope. Otherwise a new authentication is being performed.

API operation mapping structure

Python based OpenStack API binding tools are structured on a resource base, where every API resource is a class/object having multiple methods for the resource CRUD and other operations. Moreover microversion differences are also being dealt inside this single object. This causes method typing being problematic and not definite (i.e. when create and get operations return different structures or microversions require modified types).

Since Rust is a strongly typed programming language the same approach is not going to work (neither this approach proved to be a good one). Every unique API call (url + method + payload type) is represented by a dedicated module (simple Enum are still mapped into same module). All RPC-like actions of OpenStack services are also represented by a dediated module. Also when the operation supports different request body schemas in different microversions it is also implemented by a dedicated module. This gives user a better control by using an object with definite constraints explicitly declaring support of a certain microversion.

OpenStack API bindings (SDK)

Every platform API requires SDK bindings to various programming languages. OpenStack API bindings for Rust are not an exception. The OpenStack comes with openstack_sdk crate providing an SDK with both synchronous and asynchronous interfaces.

API bindings are generated from the OpenAPI specs of corresponding services. That means that those are only wrapping the API and usually not providing additional convenience features. The major benefit is that no maintenance efforts are required for the code being generated. Once OpenStack service updates OpenAPI spec for the integrated resources the changes would be immediately available on the next regeneration.

Features

Sync and Async interface
Query, Find and Pagination interfaces implementing basic functionality
RawQuery interface providing more control over the API invocation with upload and download capabilities.
Every combination of URL + http method + body schema is represented by a dedicated module
User is in charge of return data schema.

Structure

Every single API call is represented by a dedicated module with a structure implementing REST Endpoint interface. That means that a GET operation is a dedicated implementation compared to a POST operation. Like described in the Structure document every RPC-like action and every microversion is implemented with a single module.

Using

The simplest example demonstrating how to list compute flavors:

#![allow(unused)]
fn main() {
use openstack_sdk::api::{paged, Pagination, QueryAsync};
use openstack_sdk::{AsyncOpenStack, config::ConfigFile, OpenStackError};
use openstack_sdk::types::ServiceType;
use openstack_sdk::api::compute::v2::flavor::list;

async fn list_flavors() -> Result<(), OpenStackError> {
    // Get the builder for the listing Flavors Endpoint
    let mut ep_builder = list::Request::builder();
    // Set the `min_disk` query param
    ep_builder.min_disk("15");
    let ep = ep_builder.build().unwrap();

    let cfg = ConfigFile::new().unwrap();
    // Get connection config from clouds.yaml/secure.yaml
    let profile = cfg.get_cloud_config("devstack".to_string()).unwrap().unwrap();
    // Establish connection
    let mut session = AsyncOpenStack::new(&profile).await?;

    // Invoke service discovery when desired.
    session.discover_service_endpoint(&ServiceType::Compute).await?;

    // Execute the call with pagination limiting maximum amount of entries to 1000
    let data: Vec<serde_json::Value> = paged(ep, Pagination::Limit(1000))
        .query_async(&session)
        .await.unwrap();

    println!("Data = {:?}", data);
    Ok(())
}
}

Documentation

Current crate documentation is known to be very poor. It will be addressed in future, but for now the best way to figure out how it works is to look at openstack_cli and openstack_tui using it.

Crate documentation is published here

Project documentation is available here

OpenStackClient

osc is a CLI for the OpenStack written in Rust. It is relying on the corresponding openstack_sdk crate (library) and is generated using OpenAPI specifications. That means that the maintenance effort for the tool is much lower compared to the fully human written python-openstackclient. Due to the fact of being auto-generated there are certain differences to the python cli but also an enforced UX consistency.

NOTE: As a new tool it tries to solve some issues with the original python-openstackclient. That means that it can not provide seamless migration from one tool to another.

Commands implementation code is being produced by codegenerator what means there is no maintenance required for that code.

Microversions

Initially python-openstackclient was using lowest microversion unless additional argument specifying microversion was passed. Later, during switching commands towards using of the OpenStackSDK a highest possible microversion started being used (again unless user explicitly requested microversion with --XXX-api-version Y.Z). One common thing both approaches use is to give user control over the version what is crucial to guarantee stability. The disadvantage of both approaches is that they come with certain opinions that does not necessarily match what user expects and make expectation on what will happen hard. For the end user reading help page of the command is pretty complex and error prone when certain parameters appear, disappear and re-appear with different types between microversion. Implementing (and using) the command is also both complex and error prone in this case.

osc is trying to get the best of 2 approaches and providing dedicated commands for microversions (i.e. create20, create294). Latest microversion command is always having a general alias (create in the above case) to let user explicitly use latest microversion, what, however, does not guarantee it can be invoked with requested parameters. This approach allows user to be very explicit in the requirement and have a guarantee of the expected parameters. When a newer microversion is required user should explicitly to do "migration" step adapting the invocation to a newer set of parameters. Microversion (or functionality) deprecation is also much simpler this way and is handled by marking the whole command deprecated and/or drop it completely.

Request timing

osc supports --timing argument that enables capturing of all HTTP requests and outputs timings grouped by URL (ignoring the query parameters) and method.

Command interface

`osc`

OpenStack command line interface.

Configuration

As all OpenStack tools it fully supports clouds.yaml

Features

osc api as an API wrapper allowing user to perform any direct API call specifying service type, url, method and payload. This can be used for example when certain resource is not currently implemented natively.
osc auth with subcommands for dealing explicitly with authentication (showing current auth info, renewing auth, MFA/SSO support)
Every resource is having a service type in the command solving confusions like user groups vs volume groups
Every multi-word resource name is "-" separated (i.e. floating-ip, access-rule)

Output

osc ... -o json as an explicit machine readable format output. It allows seeing raw resource json representation as send by the API without any processing on the client side.

Note: the result is not the raw json response, but the raw json resource information found underneath expected resource key. This mode can be used i.e. to see fields that are not expected by the osc and allows further easy machine processing with tools like jq

osc ... -o wide for list operations to return all known fields. By default list operation will only return a subset of known generic resource fields to prevent multiline tables. This mode (together with not specifying -o parameter at all) is considered as an output for humans. Field names are not generally renamed and are names as the API returns them.

Shell autocompletion

osc supports generation of the completion file for diverse shells. This can be enabled i.e. by executing

bash echo 'source <(osc completion bash)' >>~/.bashrc

Usage: osc [OPTIONS] <COMMAND>

Available subcommands:

osc api — Perform direct REST API requests with authorization
osc auth — Cloud Authentication operations
osc block-storage — Block Storage (Volume) service (Cinder) commands
osc catalog — Catalog commands args
osc compute — Compute service (Nova) operations
osc dns — DNS service (Designate) operations
osc identity — Identity (Keystone) commands
osc image — Image service operations
osc load-balancer — Load Balancer service operations
osc network — Network (Neutron) commands
osc object-store — Object Store service (Swift) commands
osc placement — The placement API service was introduced in the 14.0.0 Newton release within the nova repository and extracted to the placement repository in the 19.0.0 Stein release. This is a REST API stack and data model used to track resource provider inventories and usages, along with different classes of resources. For example, a resource provider can be a compute node, a shared storage pool, or an IP allocation pool. The placement service tracks the inventory and usage of each provider. For example, an instance created on a compute node may be a consumer of resources such as RAM and CPU from a compute node resource provider, disk from an external shared storage pool resource provider and IP addresses from an external IP pool resource provider
osc completion — Output shell completion code for the specified shell (bash, zsh, fish, or powershell). The shell code must be evaluated to provide interactive completion of osc commands. This can be done by sourcing it from the .bash_profile

Options:

--os-cloud <OS_CLOUD> — Name reference to the clouds.yaml entry for the cloud configuration
--os-project-id <OS_PROJECT_ID> — Project ID to use instead of the one in connection profile
--os-project-name <OS_PROJECT_NAME> — Project Name to use instead of the one in the connection profile
--os-client-config-file <OS_CLIENT_CONFIG_FILE> — Custom path to the clouds.yaml config file
--os-client-secure-file <OS_CLIENT_SECURE_FILE> — Custom path to the secure.yaml config file
-o, --output <OUTPUT> — Output format

Possible values:
- json: Json output
- wide: Wide (Human readable table with extra attributes). Note: this has effect only in list operations
-f, --fields <FIELDS> — Fields to return in the output (only in normal and wide mode)
-p, --pretty — Pretty print the output
-v, --verbose — Verbosity level. Repeat to increase level
--timing — Record HTTP request timings

`osc api`

Perform direct REST API requests with authorization

This command enables direct REST API call with the authorization and version discovery handled transparently. This may be used when required operation is not implemented by the osc or some of the parameters require special handling.

Example:

console osc --os-cloud devstack api compute flavors/detail | jq

Usage: osc api [OPTIONS] <SERVICE_TYPE> <URL>