From Hundreds of API Methods to One Conversation

In 2025, a company developing public and private cloud platforms asked me to explore how AI could become part of its customer-facing product. Its cloud already offered a web interface, a REST API, and Terraform integrations. But creating several connected resources still required either extensive manual configuration or infrastructure-as-code expertise.

The REST API exposed the broadest set of capabilities, while the web and Terraform interfaces covered different subsets. That made the platform powerful, but gave new users a steep learning curve.

I proposed adding an AI assistant directly to the cloud interface. It would not be another documentation chatbot. A user could describe an outcome, review a proposed configuration and its price, and authorize the agent to create the required resources.

A working proof of concept in under a week

After presenting the idea, I received access to the client's development cloud and built the first proof of concept independently in less than a week. It provided a standalone chat interface and demonstrated how natural-language requests could become real cloud operations.

The proof of concept led to an approximately six-month productization phase. I worked with the engineering lead, a client engineer, and the client's designers. The result was a working internal prototype deployed in the development private cloud, used for demonstrations and alpha testing.

The product hypothesis had two parts: make complex cloud configuration easier, and help customers discover relevant additional services. For example, instead of merely provisioning a virtual machine for an online store, the agent could also propose a managed PostgreSQL database and object storage, connect them, and explain the combined monthly price.

Keeping the complete system inside the client's infrastructure

Security and vendor independence were core requirements. The main deployment could not depend on an externally hosted foundation model. The client already used Kubernetes, so we separated the solution into two deployments: an LLM serving layer based on vLLM, and the agent application with its chat interface, retrieval-augmented documentation search, authorization module, and API tools.

I led the agent architecture and the evaluation of on-premise models. We tested several Qwen, Llama, and GPT-OSS configurations against the project's real scenarios. At that time, gpt-oss-120b produced the most stable tool-calling results while fitting into a compact deployment on one large GPU.

Why exposing the OpenAPI specification was not enough

The proof of concept began with standard LangChain wrappers that exposed an OpenAPI specification to the agent. This worked for simple methods, but the production API contained hundreds of operations. Some resource-creation requests required deeply nested JSON bodies several kilobytes long, and later calls had to reuse UUIDs returned by earlier ones.

Smaller on-premise models struggled with the resulting context. They produced malformed request bodies, confused example identifiers with real ones, or changed a UUID while copying it into the next call. Simply adding more prompt instructions did not make the system reliable.

The breakthrough came from giving the LLM less responsibility, not more.

We replaced the generic wrappers with domain-specific Python tools. Their parameters were small and flat; deterministic code constructed the complex JSON bodies, preserved real UUIDs behind short internal references, handled authorization, and combined multi-step API sequences into operations the agent could call reliably. After establishing the pattern manually, we used a coding agent to generate the remaining wrappers from those examples.

We also split the system into specialized sub-agents. Instead of one agent seeing hundreds of methods, a router selected one of five or six tool domains, each containing no more than about ten tools. The smaller context and clearer responsibilities produced a substantial improvement in reliability.

Confirmation, scoped permissions, and deterministic rollback

The agent operated through a dedicated service account and inherited the cloud's existing role-based access controls. A client could issue a token with only the permissions appropriate for the intended scenarios, excluding destructive operations if necessary.

Before creating paid resources, the interface presented a structured plan, retrieved current tariffs through a dedicated API tool, displayed the expected cost, and requested explicit confirmation. The confirmation was logged. If a multi-step operation failed, transactional application code—not the LLM—could roll back the resources that had already been created.

Individual API failures could be retried after the agent analyzed the returned error, but retry limits prevented endless loops. Once the limit was reached, the agent stopped and asked the user for help rather than continuing to make changes.

Testing a nondeterministic product

A single successful demonstration could not validate an LLM-based interface. We defined six key end-to-end scenarios and ran each repeatedly—typically five to ten times or more. The tests verified that the correct APIs were called with the required parameters and that each scenario reached its intended outcome.

The final prototype completed more than 90% of repetitions across every key scenario. We did not benchmark elapsed time against the web interface or Terraform, so the result should not be presented as a proven speed improvement. The demonstrated benefit was a different interaction model: users could agree on the desired outcome, authorize the plan, and let the system complete the detailed configuration with less active involvement.

From cost reduction to a new product interface

The prototype remained an internal alpha rather than a public-cloud launch. It was positively received in internal demonstrations, used inside the development cloud, and positioned for continued development as part of private-cloud deployments. Because the client's engineer had built the solution alongside me, the code, infrastructure, and operational knowledge all remained inside the company.

Cross-selling additional cloud services was a product hypothesis, not a result measured with public users. The broader lesson was still valuable: AI does not have to be limited to automating employees' work and reducing cost. Embedded in a product, it can create a new user experience, expose capabilities customers might otherwise miss, and open a path toward revenue growth.

The next generation of customers may be agents acting on behalf of people—not only people operating another graphical interface.

If I designed the platform today, I would retain the domain-specific tools, deterministic safeguards, and specialized sub-agents. I would also evaluate the latest Qwen models and introduce an MCP layer from the beginning, making it easier for other assistants to consume the same safe cloud capabilities. That agent-to-agent interface is a logical next step, not something delivered in the original prototype.