Add automated OpenAPI specification for `mgmt_api` #309

amansinghoriginal · 2025-09-19T00:06:11Z

Description

This pull request introduces an utomated system for generating and maintaining an OpenAPI 3.0 specification for the Drasi Management API.

The goal is to have a version-controlled API contract that is always in sync with the implementation. This will improve developer experience and documentation, and will enable automated tooling for API validation and client generation.

Key Changes

1. Refactoring for a Valid OpenAPI Generation

Refactoring of the mgmt_api service to make it compatible with OpenAPI generation tools.

utoipa Integration: The utoipa crate has been integrated into the mgmt_api service to generate the OpenAPI spec directly from the Rust code.
Macro Removal & Handler Refactoring: The previous route-generation macros (v1_crud_api!) have been completely removed. They are replaced with explicit Actix handler functions (upsert, get, list, etc.) in new, dedicated modules for each resource (e.g., sources.rs, reactions.rs).
Introduction of Concrete DTOs: To solve a fundamental limitation where the utoipa tool could not correctly generate schemas for generic Rust types (e.g., ResourceDto<TSpec, TStatus>), new concrete DTOs have been introduced for each resource (e.g., SourceDto, ReactionDto). The handlers have been updated to use these explicit types, which makes the API contract unambiguous and allows utoipa to generate a valid spec.
Generated Spec: The resulting valid openapi.yaml is now checked into the repository at control-planes/mgmt_api/openapi.yaml.

2. Automation and CI/CD Integration

Pre-Commit Hook: A new Git pre-commit hook (.githooks/pre-commit) has been added. It automatically regenerates the OpenAPI spec before each commit and fails if the generated spec has changed, forcing the developer to include the updated spec.
CI Verification: The build-test.yml GitHub Actions workflow has been updated with two new jobs:
- verify-openapi-spec: Ensures the committed spec is never out of sync with the code.
- detect-breaking-api-changes: Uses openapi-diff to prevent backward-incompatible changes from being merged into main, thus enforcing the v1 API contract.

3. API Versioning Alignment

Package Version: The mgmt_api crate version in Cargo.toml has been updated from 0.1.0 to 1.0.0. This aligns the package's major version with the /v1 API path.
Spec Version: The version in the generated openapi.yaml is set to 1.0.0.

4. Tooling and Refinements

Makefile Targets: New openapi-* targets have been added to the control-planes/mgmt_api/Makefile for developers to easily generate, validate, and serve the spec locally.
Dead Code Removal: The now-unused route-generation macros (v1_crud_api!) have been removed, cleaning up the codebase.
WebSocket Documentation: A placeholder has been added to the OpenAPI spec to document the /v1/debug WebSocket endpoint.

Reasons for refactoring to concrete DTOs

The macro-based approach with v1_crud_api!(SourceDomainService, SourceSpecDto, SourceStatusDto) generated handlers using ResourceDto<$spec_dto, $status_dto>
ResourceDto<TSpec, TStatus> generic pattern cannot be properly represented in OpenAPI specs
Utoipa cannot resolve these generic type parameters at compile time for OpenAPI generation
Benefits of the Refactoring:
- Produces a proper, parseable OpenAPI 3.0 specification
- Better documentation: Each resource type has explicit, documented endpoints
- CI/CD integration: Enables automated spec validation and breaking change detection
- Type safety: Concrete types provide clearer API contracts

Type of change

This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Drasi.

control-planes/mgmt_api/openapi.yaml

control-planes/mgmt_api/src/api/v1/openapi.rs

.github/workflows/build-test.yml

Copilot

Pull Request Overview

This PR introduces automated OpenAPI 3.0 specification generation for the Drasi Management API, enabling automatic documentation and API contract validation. The implementation uses the utoipa crate to generate OpenAPI specs directly from Rust code annotations, ensuring the documentation stays synchronized with the implementation.

Key Changes:

Replaced macro-based route generation with explicit handler modules annotated with #[utoipa::path] for OpenAPI documentation
Added automated CI/CD verification to prevent API specification drift and breaking changes
Integrated Swagger UI for interactive API documentation

Reviewed Changes

Copilot reviewed 27 out of 29 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
control-planes/mgmt_api/src/main.rs	Updated routing configuration to use new explicit handler modules and added Swagger UI service
control-planes/mgmt_api/src/lib.rs	New library file exposing API modules for the spec generation binary
control-planes/mgmt_api/src/api/v1/*.rs	New explicit handler modules replacing macro-generated routes with utoipa annotations
control-planes/mgmt_api/src/api/v1/openapi.rs	Central OpenAPI documentation configuration
control-planes/mgmt_api/src/api/v1/models/*.rs	Added ToSchema derives for OpenAPI schema generation
control-planes/mgmt_api/openapi.yaml	Generated OpenAPI 3.0 specification
control-planes/mgmt_api/Makefile	Added OpenAPI generation, validation, and serving targets
control-planes/mgmt_api/Dockerfile.*	Updated to specify mgmt_api binary explicitly
control-planes/mgmt_api/Cargo.toml	Added utoipa dependencies and spec generation binary
.github/workflows/build-test.yml	Added CI jobs for OpenAPI spec verification and breaking change detection
.githooks/pre-commit	Added pre-commit hook to ensure API spec stays current

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

control-planes/mgmt_api/src/api/v1/models/providers.rs

control-planes/mgmt_api/src/api/v1/source_providers.rs

control-planes/mgmt_api/src/api/v1/reaction_providers.rs

Copilot

Pull Request Overview

Copilot reviewed 32 out of 34 changed files in this pull request and generated 4 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

control-planes/mgmt_api/src/api/v1/models/providers.rs

control-planes/mgmt_api/src/api/v1/debug.rs

control-planes/mgmt_api/src/main.rs

control-planes/mgmt_api/src/api/v1/models/providers.rs

This commit introduces a robust, automated system for generating and maintaining an OpenAPI 3.0 specification for the management API. This provides a version-controlled API contract that is always in sync with the implementation. The previous macro-based routes have been refactored into explicit Actix handlers. Generic DTOs in the API layer were replaced with concrete structs (e.g., SourceDto) to resolve a utoipa limitation and produce a valid spec. Key changes include: - Integrated the utoipa crate to generate the spec from Rust code. - Aligns the mgmt_api crate version to 1.0.0 to match the /v1 API path. - Adds a pre-commit hook to enforce local spec updates. - Adds CI jobs to verify spec sync and detect breaking changes. - verify-openapi-spec: Fails the build if the committed spec is out of date. - detect-breaking-api-changes: Uses openapi-diff to prevent merging backward-incompatible changes. Signed-off-by: Aman Singh <aman.singh.original@gmail.com>

danielgerlag

Great contribution! 👍 👍

danielgerlag · 2025-09-23T19:58:19Z

.githooks/pre-commit

        ;;
 esac

+echo "Running pre-commit hook: Checking for API spec changes..."


In my experience, pre-commit hooks are not reliable, as devs often don't set execution permissions or there are cross platform issues.

danielgerlag · 2025-09-23T20:00:48Z

control-planes/mgmt_api/src/api/v1/continuous_queries.rs

+    ),
+    request_body = QuerySpecDto,
+    responses(
+        (status = 200, description = "Query created or updated", body = ContinuousQueryDto),


We don't support updating queries

danielgerlag · 2025-09-23T20:05:26Z

control-planes/mgmt_api/src/api/v1/continuous_queries.rs

+        (status = 500, description = "Internal server error")
+    )
+)]
+pub async fn watch(service: web::Data<DebugService>, id: web::Path<String>) -> impl Responder {


I don't think we have any of the E2E tests that hit the watch or debug endpoints. We'll need to test these.

danielgerlag · 2025-09-23T20:08:27Z

control-planes/mgmt_api/src/api/v1/models/result.rs

+
+#[derive(Serialize, ToSchema)]
+#[serde(tag = "kind")]
+pub enum ResultEventDto {


Sine the ResultEvent originates from the QueryHost, would it make sense for the QueryHost to be the source of truth for that schema and publish an OpenAPI spec that the Management API consumes?

EDIT: we have a TypeSpec definition here: https://github.com/drasi-project/drasi-platform/blob/main/typespec/query-output/main.tsp

danielgerlag · 2025-09-23T20:10:12Z

control-planes/mgmt_api/src/api/v1/openapi.rs

+    ),
+    info(
+        title = "Drasi Management API",
+        version = "1.0.0",


Should we harmonize thus version with our release versions?

danielgerlag · 2025-09-23T20:11:37Z

control-planes/mgmt_api/src/api/v1/query_containers.rs

+) -> impl Responder {
+    log::debug!("readywait_resource: {:?}", id);
+
+    if params.timeout > 300 {


I see this timeout in multiple places, should we manage it from a config file or constant?

amansinghoriginal requested a review from a team as a code owner September 19, 2025 00:06

amansinghoriginal force-pushed the api_spec branch from bd43733 to 24a03c6 Compare September 19, 2025 00:06

github-advanced-security bot found potential problems Sep 19, 2025

View reviewed changes

control-planes/mgmt_api/openapi.yaml Fixed Show fixed Hide fixed

control-planes/mgmt_api/src/api/v1/openapi.rs Fixed Show fixed Hide fixed

amansinghoriginal force-pushed the api_spec branch from 0094c82 to 94ef57e Compare September 19, 2025 00:44

github-advanced-security bot found potential problems Sep 19, 2025

View reviewed changes

.github/workflows/build-test.yml Show resolved Hide resolved

amansinghoriginal force-pushed the api_spec branch 2 times, most recently from a71724a to 8c6f729 Compare September 19, 2025 00:53

amansinghoriginal requested a review from Copilot September 19, 2025 03:37

Copilot AI reviewed Sep 19, 2025

View reviewed changes

control-planes/mgmt_api/src/api/v1/models/providers.rs Outdated Show resolved Hide resolved

control-planes/mgmt_api/src/api/v1/source_providers.rs Outdated Show resolved Hide resolved

control-planes/mgmt_api/src/api/v1/reaction_providers.rs Outdated Show resolved Hide resolved

amansinghoriginal self-assigned this Sep 19, 2025

amansinghoriginal force-pushed the api_spec branch from 8c6f729 to c81c4dc Compare September 19, 2025 16:02

amansinghoriginal requested a review from Copilot September 19, 2025 16:06

Copilot AI reviewed Sep 19, 2025

View reviewed changes

amansinghoriginal force-pushed the api_spec branch from c81c4dc to 3d73a50 Compare September 19, 2025 16:31

amansinghoriginal force-pushed the api_spec branch from 3d73a50 to c1ecc91 Compare September 19, 2025 22:06

amansinghoriginal requested review from agentofreality, danielgerlag and ruokun-niu September 19, 2025 23:37

amansinghoriginal linked an issue Sep 22, 2025 that may be closed by this pull request

Kubernetes Operator for Drasi #312

Open

1 task

danielgerlag reviewed Sep 23, 2025

View reviewed changes

Add automated OpenAPI specification for mgmt_api #309

Are you sure you want to change the base?

Add automated OpenAPI specification for mgmt_api #309

Uh oh!

Conversation

amansinghoriginal commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key Changes

1. Refactoring for a Valid OpenAPI Generation

2. Automation and CI/CD Integration

3. API Versioning Alignment

4. Tooling and Refinements

Reasons for refactoring to concrete DTOs

Type of change

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Key Changes:

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

danielgerlag left a comment

Choose a reason for hiding this comment

Uh oh!

danielgerlag Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

danielgerlag Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

danielgerlag Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

danielgerlag Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

danielgerlag Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

danielgerlag Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

danielgerlag Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add automated OpenAPI specification for `mgmt_api` #309

Add automated OpenAPI specification for `mgmt_api` #309

amansinghoriginal commented Sep 19, 2025 •

edited

Loading