Vertex AI Agent Builder

                                                      Vertex AI Agent Builder

Vertex AI Agent Engine

The Agent Builder menu has seven categories for implementing functionality. 

 Agent designer

We can start with Agent designer that allows the developer to leverage an easy to use graphical interface to build agents. You can design, build and test agents via the Google cloud console. It helps the developer to visualize the workflow process.

Agent Garden

Agent Garden is a library of pre-defined agents for developers to select for use. It contains agent based solutions in the form of end-to-end templates for functions like customer support, data analysis, financial advisory, etc. 

The user can find agents based on functionality.

For example if I wanted to find a pre-built agent with Multimodal capabilities I could click it and the agent would appear on the right ahdn side.

Or you can search for agents by keywords

The Customer Service Agent

Agent Engine

Vertex AI Agent Engine is the environment that allows the user to build, host and run Agents. It leverages Google's managed service infrastructure to provide the resources needed to run agent based solutions.

Some of the core functionalities are a management runtime that automatically scales depending on processing needs, reasoning engines to decide on tool usage, tool integration like search or API access and  memory management to keep track of the context of conversations across steps.

Tools

Tools are the methods that allow AI Agents to access functionality both inside the Google cloud infrastructure and outside resources like APIs. 

Tools can be classified into three categories, extension tools, function calling and Data stores.

Extension Tools can be pre-built or custom created and can allow the agent to connect to external API’s. Google provides pre-built tools for searching or running python code. Custom allows you to develop API calls to external sites by providing specifications for the call.

Function Calling allows the developer to use existing tested and reliable python code as a tool. Other languages like Java, Node.js or C# can also be used.

The Data Store tool leverages RAG (Retrieval-Augmented Generation) to provide structured, unstructured, and web data to the agent. Unstructured PDFs, HTML or word files can be accessed as well as structured tabular data like csv’s and tables as well as web domains.

RAG (Retrieval-Augmented Generation) allows agents to access the data that the developer wishes the agent to focus on as a priority over its training data. 

The Retrieve part searches the Data Store to provide relevant information. The Augment takes that information and adds it to user input. Generate delivers the response.

Example of SQL tool

RAG Engine

Retrieval-Augmented Generation provides developers with managed suite of APIs and infrastructure designed to ground AI responses in your specific data.

While a standard LLM is like a person with a massive library of general knowledge in their head, a RAG Engine is that same person with a high-speed search engine and a filing cabinet full of your private, real-time company documents.

As previously stated RAG allows specific data access for the AI Agents.

Some of the key functionality that it uses are ingestion, indexings, document parsing, chunking or splitting data and embedding for enhanced searching.

The benefits include reduced hallucinations, Up-to-Date Information and Data Security.

A Corpus is a searchable container for the data created with RAG. 

Vertex AI Search

Vertex AI Search allows the developer to create specific search apps for information retrieval for agents.

Developers can build Search Apps for internal documents or PDFs, websites by providing a domain for google to index or media searches that can discover video and audio content.

Create a Site search AI mode app hit Create 

Leave defaults for check for

Enterprise edition features and Generative Responses

Your app name - SEC Web Search App

Company Name - CT Innovation Foundry

Multi region - global

Click Continue

Click Create Data Store

Choose Website Content click Select

Leave Advanced website indexing un-checked

Enter - www.sec.gov

Click Continue

Enter SEC Filings in data store name then

Click Continue

Select General pricing

Click Create

Check on the new SEC Filings data store 

Click Create

App now ready click Preview

Enter - last 10k for NVDA

Returns Documents

Vector Search

Using Vector Search, allows the developers to create searches by meaning as opposed to pattern matching.

Vertex AI Agent Builder concepts

The Vertex AI Agent Builder process allows developers to build systems that can reason, use tools and access specific data sources.

The platform can be broken down into four stages: Build, Connect, Scale and Govern.

The Build layer is where the logic, persona and reasoning of your agent is defined. Developers can use a no-code graphical interface or the Agent Development LIT (ADK), a python framework to help create goals, instructions and tools.

Connect layer provides access to the Tools needed that bring in external data or programs.

Scale provides the cloud infrastructure needed to host and run the AI Agents along with the proper security needed.

Govern makes sure that the agent is safe and accurate.Uses LLM-as-a-judge to validate responses and is integrated with Google monitoring to check if processing is working correctly. Also, can block harmful content from and control data leaks.