Skip to content

New agent skill for building a multi-agent system to automate data science workflows #189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
copybara-service merged 1 commit into from
Jul 14, 2026
Merged

New agent skill for building a multi-agent system to automate data science workflows #189

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
196 changes: 196 additions & 0 deletions skills/cloud/google-cloud-solution-agentic-ai-data-science-workflow/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
---
name: google-cloud-solution-agentic-ai-data-science-workflow
description: >-
Designs a tailored multi-product agentic data science
architecture on Google Cloud that incorporates opinionated best practices. Use
when architecting multi-product solutions for agent-based data analytics or ML
workloads. Don't use for simple queries, non-agentic pipelines, general cloud
reviews, or writing agent code.
---

# Data science workflow with AI agents solution

This skill guides agents through the workflow to design and implement a
tailored multi-product solution in the cloud for a given workload, use case, or
requirement.

## Workflow

The solution design and implementation workflow consists of the following
phases:

- **Phase 1: Requirements discovery and analysis**: Analyze the workload's
requirements, constraints, dependencies, and current state.
- **Phase 2: Solution design**: Build a technology stack, architecture, and
deployment configuration for the workload based on Google Cloud design best
practices and recommendations.
- **Phase 3: Implementation plan**: Generate automation and instructions to
deploy the solution.
- **Phase 4: Solution validation**: Validate that the deployment meets the
requirements of the workload.

## Product Renaming & Terminology

When generating solution designs, architecture diagrams, and documentation,
check the latest Google Cloud documentation for the most up-to-date product
names. The table below provides examples of name mappings to be aware of. Note
that underlying APIs, Terraform resources, and IAM roles may retain their legacy
identifiers.

| Legacy Name | Updated Name |
| :--- | :--- |
| Vertex AI | Gemini Enterprise Agent Platform |
| Vertex AI Agent Engine | Gemini Enterprise Agent Runtime |

### Phase 1: Requirements discovery and analysis

- [ ] **Step 1: Discover requirements**: Understand the functional and
non-functional requirements, business goals, and current state (if any) of the
workload by asking clarifying questions. You must halt and wait for the user
to answer these questions before proceeding to the **Identify components**
step. Use the following questions to guide this requirements discovery
process:
- What data sources and data types do you need to access and analyze?
- Who are the target end users, and what network access model do you require?
- What types of user queries or analytical requests do you expect end users
to submit to the system?
- What performance, security, or governance constraints apply?

- [ ] **Step 2: Identify components**: Only after the user has responded to the
clarifying questions in the **Discover requirements** step, analyze their
responses to identify the components of the workload and their relationships.
Also identify any cross-cloud, hybrid, or on-premises components that the
solution needs to integrate with.

- [ ] **Step 3: Generate component decomposition**: Generate a technical
decomposition outlining the technical components of the workload and their
relationships.

- [ ] **Step 4: Ask for confirmation**: Present the technical decomposition and
ask the user to confirm if it matches their workload requirements. Do not
proceed to Phase 2 until this is confirmed.

- [ ] **Step 5: Iterate**: If the user requests changes, generate an updated
technical decomposition and ask for confirmation again. Continue iterating
until the user explicitly confirms the decomposition.

### Phase 2: Solution design

- [ ] **Step 1: Retrieve relevant Google Cloud documentation**: Use available
search or fetch tools to read the content of the following Google Cloud
documentation to ground the guidance that you generate in the remaining steps
of this phase before proceeding.
- [Data science workflow with AI agents](https://docs.cloud.google.com/architecture/agentic-ai-data-science.md.txt)
- [Multi-agent AI system in Google Cloud](https://docs.cloud.google.com/architecture/multiagent-ai-system.md.txt)
- [Choose your agentic AI architecture components](https://docs.cloud.google.com/architecture/choose-agentic-ai-architecture-components.md.txt)
- [Choose a design pattern for your agentic AI system](https://docs.cloud.google.com/architecture/choose-design-pattern-agentic-ai-system.md.txt)

- [ ] **Step 2: Define agentic AI design pattern**: Select the appropriate agent
design pattern and agent breakdown based on the workload requirements:
- **Recommended primary pattern**: Coordinator pattern.
- **Alternative patterns**:
- *Single-agent pattern*: For simpler workloads scoped to a single data
source and direct tool use without multi-agent orchestration overhead.
- *Sequential or parallel pattern*: For deterministic data processing
pipelines with predefined, non-adaptive execution steps or concurrent
data gathering.
- *Review and critique pattern*: For complex or high-stakes data science
tasks that require dedicated critic loops.

- [ ] **Step 3: Map components to Google Cloud products**: For each component in
the confirmed technical decomposition and agentic design pattern, identify the
appropriate Google Cloud products and features, based on the guidelines in
/references/product-mapping.md.

- [ ] **Step 4: Create architecture diagram**: Create an architecture diagram
that shows the components, their relationships, and data/control flows.
- The diagram must be in the Mermaid format:
https://github.com/mermaid-js/mermaid.
- The diagram must use component labels and groupings consistent with the
official Google Cloud architecture icons.

- [ ] **Step 5: Generate design recommendations**: Generate design guidance
based on the guidelines in /references/design-recommendations.md.

- [ ] **Step 6: Draft solution architecture**: Compile the requirements,
technical decomposition, product mapping, architecture diagram, and design
recommendations into a single Markdown file named
`solution-architecture-guide.md`, based on the template in
`/assets/output-template.md`.

- [ ] **Step 7: Request review**: Present the generated solution architecture to
the user and request their feedback or approval. You must halt and wait for
the user's explicit approval before proceeding to Phase 3.

- [ ] **Step 8: Iterate**: If the user requests changes, then generate an
updated solution architecture and repeat steps 2-7 in this phase until the
user explicitly approves the solution architecture.

### Phase 3: Implementation plan

- [ ] **Step 1: Retrieve relevant implementation resources**:
- [ADK Data Science Sample Code](https://github.com/google/adk-samples/tree/main/python/agents/data-science)
- [Stateful Data Science Agent on Agent Engine](https://codelabs.developers.google.com/next26/adk-deploy-scale)
- [Build and deploy an AI agent to Cloud Run using ADK](https://docs.cloud.google.com/run/docs/ai/build-and-deploy-ai-agents/deploy-adk-agent.md.txt)
- [Use AlloyDB with agents](https://docs.cloud.google.com/alloydb/docs/connect-ide-using-mcp-toolbox.md.txt)
- [MCP Toolbox for Databases Configuration](https://mcp-toolbox.dev/documentation/configuration/)

_Important_: Use these resources as the technical foundation for the IaC and
deployment instructions you generate in the remaining steps of this phase.

- [ ] **Step 2: Identify deployment prerequisites**: Document prerequisites for
the deployment, including the following:
- Projects and billing associations
- Required Google Cloud APIs
- Required IAM permissions
- Any other prerequisites

- [ ] **Step 3: Generate Infrastructure as Code (IaC)**: Generate code, such as
Terraform, and deployment scripts to automate the provisioning of the proposed
Google Cloud resources.

- [ ] **Step 4: Write deployment instructions**: Draft sequential, step-by-step
deployment instructions to execute the IaC and initialize the workload
components. Update deployment instructions in
`solution-architecture-guide.md`, based on the template in
`assets/output-template.md`.

- [ ] **Step 5: Request review**: Present the generated deployment instructions
to the user for feedback and confirmation. You must halt and wait for the
user's explicit approval before proceeding to Phase 4.

- [ ] **Step 6: Iterate**: If the user requests changes, then repeat steps 2-5
to generate an updated implementation plan that the user requested.

- [ ] **Step 7: Proceed to the next phase**: After the user approves the
implementation plan, proceed to Phase 4.

### Phase 4: Solution validation

- [ ] **Step 1: Retrieve relevant verification resources (optional)**: If the
resources from Phase 3 are not already in your context, retrieve the same
implementation resources as the starting point for the
validation checks and verification scripts that you generate in this phase.

- [ ] **Step 2: Define validation checks**: Outline validation steps to verify
that the deployed infrastructure meets the workload's requirements:
- **Deployment dry-run**: Commands like `terraform plan` to preview changes.
- **Connectivity and routing**: Verification of network paths, load balancer
routing, and service endpoints.
- **Security policies**: Verification of restricted access, firewall rules,
and IAM enforcement.

- [ ] **Step 3: Generate verification scripts**: Draft lightweight scripts or
command-line instructions (e.g. using `curl` or `gcloud`) that the user can
run to perform these validation checks.

- [ ] **Step 4: Compile validation report**: Document the validation steps,
verification scripts, and expected outcomes in a single Markdown file.

- [ ] **Step 5: Conduct validation and finalize**: Assist the user in executing
the validation checks and troubleshooting any deployment issues. After you
validate the solution successfully, request final approval from the user.

- [ ] **Step 6: Iterate**: If the user requests changes, then generate an
updated validation plan and repeat the validation drafting and script
generation steps in this phase until the user approves the validation plan.
195 changes: 195 additions & 0 deletions ...oogle-cloud-solution-agentic-ai-data-science-workflow/assets/output-template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
<!-- Use this template to compile the content that you generate based on the
instructions in `SKILL.md`. -->

# Google Cloud solution architecture: Data science workflow with AI agents

## 1. Executive summary and workload overview

[A brief description of the workload, its business goals, and the high-level
solution architecture proposed.]

## 2. Requirements and current state

### 2.1. Functional requirements

* **Business processes**: [Details of the business processes supported]
* **Activities and use cases**: [Details of the key activities and use cases]

### 2.2. Non-functional requirements

* **Security**: [Details of the security requirements including compliance,
encryption, access control requirements]
* **Reliability**: [Details of the reliability requirements including SLA,
RTO/RPO, backup, redundancy requirements]
* **Cost**: [Details of the cost constraints and pricing models]
* **Operations**: [Details of the operational requirements including
monitoring, logging, deployment, maintenance requirements]
* **Performance**: [Details of the performance requirements including latency,
throughput, scaling requirements]
* **Sustainability**: [Details of the sustainability requirements including
carbon footprint, resource optimization requirements]

### 2.3. Current state

[If applicable, describe the current on-premises or other-cloud architecture.]

* **Current infrastructure**: [Details of existing setup]
* **Pain points and drivers for migration/redesign**: [Details of the drivers
for migration/redesign]

### 2.4. Dependencies

* **Internal dependencies**: [Details of internal dependencies including other
workloads and internal services]
* **External dependencies**: [Details of external dependencies including
third-party products and on-premises tools]

## 3. Technical decomposition of the workload

[Technical decomposition of the workload components, breaking down the
application into logical services or layers.]

## 4. Proposed solution architecture

### 4.1. Google Cloud products and features mapping

[Identify Google Cloud products and features mapped to the technical
components. For each component, justify the selection, note alternatives
considered, and describe the pros and cons of the recommended product/feature
and alternatives.]

| Component | Recommended Google Cloud product/feature | Justification and
citations | Alternatives considered | Pros and cons of alternatives |
| :--- |:--- | :--- | :--- | :--- |
| **[Component Name]** | **[Product Name]** | [Why
this product is chosen, citing official docs] | [Alternative product] |
**Pros**: ... <br> **Cons**: ... |

### 4.2. Architecture diagram
[Architecture diagram in Mermaid format showing the relationships and flows
between the components of the architecture.]

```mermaid
%% Example structure
flowchart TD
User([User]) --> Frontend["Frontend (Cloud Run)"]
Frontend <--> Coordinator["Root agent (Coordinator Agent)"]

subgraph Agents["Runtime for agent (Cloud Run)"]
Coordinator
Coordinator <--> Analytics["Analytics Agent"]
Coordinator <--> DBAgents["Database & ML Agents"]
end

Agents <--> Model["Gemini Model (Gemini Enterprise Agent Platform)"]
DBAgents <--> Storage["Database (BigQuery, AlloyDB, BQML)"]
```

### 4.3. Architecture description
[Detailed description of the architecture. Describe the task flow and data
flow between the components of the architecture.]

* **Data flow**: [Describe the flow of data.]
* **Tasks/control flow**: [Describe the flow of tasks/control.]

## 5. Design and configuration recommendations
[Best practices and configuration recommendations for each pillar of the
Google Cloud Architecture Framework.]

### 5.1. Security, privacy, and compliance

* **Access control and safety**: [E.g., Least-privilege agent IAM roles, Model
Armor prompt/response sanitization]
* **Data protection**: [E.g., Secret Manager for credentials, Cloud Data Loss
Prevention API for sensitive data]
* **Network security**: [E.g., Cloud Run Direct VPC egress, private VPC IPs,
regional external Application Load Balancer with Cloud Armor]

### 5.2. Reliability

* **Agent and runtime robustness**: [E.g., Coordinator agent fallback logic,
graceful error handling for SQL/code execution failures]
* **Scale and rate-limit management**: [E.g., Exponential backoff with retries
for 429 errors, provisioned model throughput settings]
* **Execution limits and redundancy**: [E.g., Query timeouts, memory/CPU caps on
generated code, multi-zone Cloud Run deployment]

### 5.3. Operational excellence

* **Monitoring and logging**: [E.g., Structured agent logs routed to Cloud
Logging, tracing inter-agent communication via Cloud Trace / OpenTelemetry]
* **Environment isolation**: [E.g., Sandboxed python code execution,
containerized Cloud Run runtimes]

### 5.4. Cost optimization

* **Resource sizing and scaling**: [E.g., Cloud Run autoscaling to zero for idle
environments, context caching for high input tokens]
* **Tiered model strategy**: [E.g., Routing simple SQL/logical tasks to Gemini
Flash and reserving Gemini Pro for complex reasoning]

### 5.5. Performance efficiency

* **Database and connectivity performance**: [E.g., Cloud Run Direct VPC egress
with connection pooling to reduce latency]
* **Processing and visualization**: [E.g., Aggregating data at the database
level, returning summarized results or charts instead of raw data]

### 5.6. Sustainability

* **Resource utilization**: [E.g., Scaling down dev databases during idle hours,
serverless Cloud Run adoption]
* **Carbon-free deployment**: [E.g., Selecting Google Cloud regions with high
Carbon-Free Energy (CFE) metrics or low CO2 indicators]

## 6. Deployment guidance
[Instructions and code for deploying the architecture.]

### 6.1. Deployment prerequisites

* [Prerequisite 1: E.g., Enabling APIs]
* [Prerequisite 2: E.g., Installing SDK/tools]
* ...and so on

### 6.2. Step-by-step deployment instructions

1. [Step 1: E.g., Authenticate with Google Cloud]
2. [Step 2: E.g., Initialize Terraform]
3. [Step 3: E.g., Apply Terraform configuration]

## 7. Validation report
[Instructions and code for verifying that the deployed infrastructure meets the
workload requirements.]

### 7.1. Validation checks

* **Deployment dry-run**: [E.g., Running terraform plan to preview
infrastructure changes]
* **Connectivity and routing**: [E.g., Verifying VPC egress routing, load
balancer service endpoints, and database connection state]
* **Security policies**: [E.g., Checking IAM role constraints on the agent
service accounts and verifying firewall rules]

### 7.2. Verification scripts
[Lightweight command-line checks or script configurations or cURL/gcloud
commands to validate the deployed environment.]

* **Execution script 1**:
```bash
# E.g., Script to test agent connectivity and endpoint response
```
* **Execution script 2**:
```bash
# E.g., Script to verify IAM permissions and service mappings
```

## 8. References

* [Data science workflow with AI agents](https://docs.cloud.google.com/architecture/agentic-ai-data-science.md.txt)
* [ADK Data Science Sample Code](https://github.com/google/adk-samples/tree/main/python/agents/data-science)
* [Multi-agent AI system in Google Cloud](https://docs.cloud.google.com/architecture/multiagent-ai-system.md.txt)
* [Choose your agentic AI architecture components](https://docs.cloud.google.com/architecture/choose-agentic-ai-architecture-components.md.txt)
* [Choose a design pattern for your agentic AI system](https://docs.cloud.google.com/architecture/choose-design-pattern-agentic-ai-system.md.txt)
* [Build and deploy an AI agent to Cloud Run using ADK](https://docs.cloud.google.com/run/docs/ai/build-and-deploy-ai-agents/deploy-adk-agent.md.txt)
* [Use AlloyDB with agents](https://docs.cloud.google.com/alloydb/docs/connect-ide-using-mcp-toolbox.md.txt)
* [MCP Toolbox for Databases Configuration](https://mcp-toolbox.dev/documentation/configuration/)
Loading