Ae Dataops

ae-dataops

CRITICAL - This skill is self-contained. Use the Global AE CLI Rules below; do not require a separate shared skill for DataOps-side tasks.

The AE Data Development and Operations domain provides capabilities for data warehouse management, flow orchestration, IDE SQL queries, data integration, operations and backfill management, including the following subcommands:

Global AE CLI Rules

AE CLI (ae-cli) is the command-line tool for the AE (Agentic Engine) analysis platform. For AE analysis-side requests, prefer ae-cli and this skill's reference docs over model memory.

Global parameters:

Output and errors:

• Successful commands return machine-readable JSON by default. Envelope may include optional _notice.host_compat.
• Failed commands return { "ok": false, "error": { "type": "...", "message": "...", "hint": "..." } } and exit non-zero.
• CRITICAL — Host compat (do this first): After each ae-cli run, check stderr and _notice.host_compat. If either is present, open the user reply with a short ⚠️ version warning and quote the `npm i -g` / `npx skills add` (or update-cluster) lines verbatim, then present the business result. Soft tip; ok: true can still carry the notice.

Safety constraints:

• Read commands can execute directly after required IDs and references are verified.
• Ordinary write commands execute without --yes; use --yes only for a high-risk-write command after explicit user confirmation.
• Never invent command names, flags, JSON payloads, project_id, resource IDs, field names, event names, property names, metric definitions, or date formats. Read the matching command reference and discover real project metadata first.
• NEVER fabricate or guess resource names (reports, dashboards, events, properties, metrics, clusters, tags, alerts). Always use list commands to discover real resources first. If a resource is not found after fuzzy search and full list fallback, explicitly tell the user "resource not found" and stop - do not proceed with fabricated names.

Domains for DataOps: dataops_repo, dataops_datatable, dataops_flow, dataops_operations, dataops_ide, dataops_integration

Core Concepts and Rules

You must understand the following key concepts before use, otherwise errors are highly likely.

ID System

Environment and Defaults

Schema Naming Rules

• DEV environment: ws_${spaceCode}_dev
• PROD environment: ws_${spaceCode}_product

Responsibility Boundaries

Flow Lifecycle

Create DEV Flow → Create/Update DEV SQL, Integration, Workflow Instance Check, or Task Instance Check Tasks → Configure Dependencies/Schedule → Preview Release → Release to PROD → PROD Manual Execution / Operations Troubleshooting

Backfill lifecycle: Discover eligible PROD flow → Create or fully update DRAFT job → Run explicitly → Search / inspect plans → Stop or rerun the complete job; delete only after target inspection

CRON Format (6 fields)

second minute hour day month weekday — Note: one more "second" field than standard 5-field format.

• 0 0 2 * * ? — Daily at 2 AM
• 0 0 */4 * * ? — Every 4 hours
• 0 30 8 * * 1-5 — Weekdays at 8:30

Preset Repository vs Non-Preset Repository

• Preset Repository (te_etl): datasourceId is te_etl@TASK_ENGINE_TRINO, database field is empty, requires gatewayConfig
• Non-Preset Repository: datasourceId is specific datasource ID, database field is required

Scenario Routing

Choose the appropriate scenario skill based on user intent to get complete step-by-step workflow guidance.

1. Space Discovery

dataops_repo exposes only one read command. Use it to discover a valid spaceCode before calling DataOps commands that require one. It returns createTime, spaceCode, and spaceDisplayName.

• If the user already provided a trusted spaceCode, reuse it.
• If spaceCode is unknown, run +list_spaces first.
• If exactly one space is returned, use its spaceCode.