Backstage AI Agent

Backstage is a popular developer portal that helps to streamline software development, manage infrastructure, tools, and services in a unified interface.

In this article we will demonstrate how to build an AI Agent that helps you to navigate inside Backstage documentation as well as perform a set of predefined actions on its behalf.

AI agents are now a new standard on how to build and manage enterprise infrastructure. They assist you in information discovery, configuration and maintenance of a product.

AWS Bedrock provides an option to configure such an agent.

The following diagram shows the solution on how to implement a Bedrock AI Agent for the Backstage.
The solution assumes that Backstage Techdocs are deployed following Recommended deployment guide and AWS S3 used as a storage.

1. For each document entry in Backstage s3 bucket there is a correspondent search_index.json that contains all document data (see an example in the attachment).
   The data in search_index.json is split in chunks (one per subheading). Content Creator function reads the
   file and assembles those chunks together. It also adds backstage links that point to the correspondent subheadings (see: generated_document.md in the attachment).
   The resulted document is stored in Bedrock S3 bucket.
2. Bedrock has an option to build a Knowledge Base from the documents (which is essentially RAG). The build process works as follows:
   • The Knowledge Base agent takes a document form the s3 bucket and pipes it thorough amazon.titan-embed-text-v1 model to generate a vector representation of the document text (aka embeddings).
   • The Knowledge Base agent then stores this vector together with the document text in OpenSearch as inverted index for the later use.
3. The Bedrock Agent is a middle man that processes user requests. The flow looks as follows:
   • The user sends a query to the Bedrock Agent.
   • The Bedrock Agent pipes the query through amazon.titan-embed-text-v1 model to generate a vector representation of the query.
   • The Bedrock Agent then sends this vector to OpenSearch to find the most similar documents.
   • The found documents and the query are piped through anthropic.claude-3-5-sonnet-20240620-v1:0 to get an answer.
   • The Bedrock Agent has a set of preconfigured actions that it can perform by invoking Action Performer lambda function (described next).

Agent Configuration

As mentioned above, an Agent is an add-on on top of RAG. It can perform some actions on your behalf.

The Bedrock Agent has an instruction setting that is used to tell it what role it has to impersonate and what behaviour you expect it to have.

Here is how we configured the setting:

The Bedrock Agent has an option to configure so-called action groups. An action group in a combination of:

• instruction for the agent on how and when to perform the action
• one or more lambda functions that agent can invoke to perform the action

here is an example of such configuration:

As you can see here we are configuring an action group to check validity of Backstage links.
Remember that we generated our markdown documents with those links embedded. Unfortunately, the model hallucinates quite often when it comes to those links and may give you a broken one. That's why we instructing the agent to check the link validity before returning it to the user.

Here is how conversation might look like:

You can imagine to configure a fleet of those action groups, each one responsible for a particular domain in your organisation.
Here is an example of one:

With this action group the agent can recognise from the question that an action can be immediately performed in order to assist a user in the discussed topic.

Here is how it looks:

Conclusion

Agents can significantly facilitate user experience and help them to quickly find organisation binding answers from its knowledge base. They can assist users in their daily work and help them to streamline their workflows.

Source Code

The source code can be found here: backstage-agent.