Databricks Generative AI Engineer Associate · CHEAT SHEET

Databricks Certified Generative AI Engineer Associate

Key concepts grouped by official exam weight.

Download as PDF (free, no sign-up)

1. Design Applications

14–14%

Design Applications

To get a specific format, combine clear instructions with few-shot examples of the desired output and, when the output must be JSON, use structured outputs: a json_schema response_format on Foundation Model APIs or responseFormat in ai_query. Simple, flat schemas generate better results, and a post-processing step can validate the format and apply a fallback response.
Map each business requirement to a model task: classification with fixed labels, schema-based field extraction, summarization, translation, sentiment analysis, entity masking or embeddings for semantic similarity. Databricks recommends starting with the task-specific AI Functions and using ai_query when you need control over the prompt or model, or must query a custom or fine-tuned model.
A RAG chain combines query understanding (rewriting, filter extraction), retrieval (semantic, keyword or hybrid, with reranking and metadata filters), prompt augmentation, the LLM, and post-processing with guardrails. Each component is chosen from the desired input and output, and smaller models suit simple steps such as intent classification.
Before building, agree with stakeholders which inputs the system will receive and who submits them, what the ideal response looks like, which criteria define a good response (accuracy, tone, citations, length) and which cost, latency and scale constraints apply. These define the input and output shape, such as source citations, custom fields or a batch job over tables.
Agents use tools to gather knowledge (table queries, vector retrievers, Genie) and to take actions (external APIs, Unity Catalog functions). The logical order is gather data, reason, then act, with human approval for risky actions. Clear tool descriptions guide tool selection, and iteration limits prevent loops.

2. Data Preparation

14–14%

Data Preparation

Chunking should follow the document structure (Markdown or HTML headers, paragraphs, code functions) to create coherent chunks with useful metadata. Chunk size balances detail and context, overlap avoids losing information at boundaries, and every chunk must fit the embedding model's token limit, measured with its own tokenizer.
Before chunking, clean noise such as page headers, footers, page numbers or HTML navigation menus, remove duplicates and near-duplicates (for example with MinHash) and filter out irrelevant, outdated, toxic or sensitive documents using metadata and classifiers. ai_parse_document labels each element by type, so unhelpful ones can be dropped.
The extraction tool depends on the format: PyPDF2 or unstructured for text PDFs, BeautifulSoup or lxml for HTML and OCR (for example Tesseract) for scans and images. On Databricks, ai_parse_document parses PDF, Office and image files read as binary and returns typed elements (text, HTML tables, headers), which makes it easy to strip noise such as page headers and footers.
The usual order is: read files from a volume with binaryFile, parse them, chunk and explode them into one row per chunk with a unique ID and the document URI, save the Delta table in Unity Catalog, enable change data feed and create the Delta Sync index. For incremental changes, MERGE INTO applies inserts, updates and deletes in one operation.
The right corpus depends on the use case, so involve domain experts early to choose sources and likely queries. Before indexing, remove duplicates (for example with MinHash), filter out outdated, unreliable, toxic or PII-bearing documents, resolve contradictions, and incrementally add any missing sources.

3. Application Development

30–30%

Application Development

Databricks AI Bridge provides per-framework packages: databricks-langchain (ChatDatabricks, DatabricksEmbeddings, DatabricksVectorSearch and VectorSearchRetrieverTool) and databricks-openai for the OpenAI SDK. Indexes with self-managed embeddings need a text_column and an embedding model. With any framework, the MLflow ResponsesAgent interface adds tracing, evaluation and deployment.
Quality is assessed with MLflow judges: RetrievalGroundedness detects hallucinations, Safety flags harmful content and Guidelines checks natural-language tone or compliance rules, all without ground truth; Correctness needs expected facts. The Review App gathers expert feedback. Evaluation and monitoring measure quality, but blocking harmful responses requires runtime guardrails.
Choosing a chunking strategy is iterative and data-dependent: fixed-size chunks are quick but rarely coherent, while paragraph-based, format-specific (Markdown or HTML headers) and semantic chunking follow meaning better. Small chunks localize facts, large ones add context; overlap avoids losing text at splits and parent-child retrieval combines both. Every change should be validated with retrieval evaluation.
Before retrieval, transform the query: rewrite it to reflect intent (for example using chat history), extract filters such as dates, products or countries, and classify intent in separate LLM calls. Filters only work if the pipeline stores those fields as metadata and the retriever applies them. The prompt template is parameterized with request data such as the current date or user context.
Moving an LLM response from a baseline to a desired output is iterative prompt work: few-shot examples fix format and style, chain-of-thought helps multi-step questions, and instructions should be clear and specific. Prompts may not transfer between models. When a fixed JSON shape is required, use structured outputs via response_format, and mlflow.genai.optimize_prompts() improves prompts with data and scorers.

4. Assembling and Deploying Applications

22–22%

Assembling and Deploying Applications

A custom pyfunc model subclasses mlflow.pyfunc.PythonModel: load_context loads artifacts such as tokenizers or weights once, and predict runs preprocessing, the model call and post-processing on every request. Shared local code is packaged with code_paths, type hints on predict add input validation and signature inference, and mlflow.models.predict validates dependencies before deployment.
For external resources, Model Serving injects credentials through environment variables that reference secrets ({{secrets/scope/key}}); the endpoint creator needs READ on the secret. For Databricks resources, automatic authentication passthrough works by declaring resources when logging the agent; on-behalf-of-user authentication uses declared scopes and is initialized in predict.
A simple LangChain chain composes steps with the | operator, for example a ChatPromptTemplate with variables and a ChatDatabricks model that wraps a Model Serving endpoint; invoke takes a dictionary with those variables. Prompts can be loaded from the Prompt Registry by alias, and steps such as query rewriting improve retrieval in multi-turn chats.
A RAG app is logged with a flavor such as langchain or pyfunc using Models from Code; queries use the same embedding model as the documents; the retriever needs a text column and embedding model when embeddings are self-managed; missing dependencies go in extra_pip_requirements; and a valid input example lets MLflow infer the signature that Unity Catalog requires.
To register in Unity Catalog, use a three-level name (catalog.schema.model) with mlflow.register_model or registered_model_name, and set_registry_uri("databricks-uc") when the default catalog is not in Unity Catalog. Creating the model needs CREATE MODEL and USE SCHEMA; adding versions needs ownership or CREATE MODEL VERSION. Aliases decouple deployments from code.

5. Governance

8–8%

Governance

Masking instead of blocking protects PII without rejecting requests: the AI Gateway PII guardrail supports Block or Mask, and deterministic pattern-based detection adds little latency compared with LLM judges, which are kept for names and free text. In batch, ai_mask hides entities before indexing; Unity Catalog column masks perform best with simple, reused SQL functions.
Defences against malicious inputs are layered: built-in AI Gateway guardrails (Jailbreak on requests, Unsafe Content on input or output), custom SQL policies for organization-specific rules, and guard models on Model Serving. An agent is vulnerable when it combines private data, untrusted inputs and external communication; restricting egress or permissions breaks that chain. DetectJailbreak scores inputs without an LLM judge.
Before ingesting a source, read its license: on Hugging Face it sits in the dataset card metadata, and in Databricks Marketplace in the provider's terms of use. Models served on Databricks carry their own terms, and complying with them is the customer's responsibility. Governed tags label each table with its license, and Unity Catalog lineage finds what must be deleted if a permission is withdrawn.
Problematic text is handled in the data pipeline, not in the prompt: keep the raw source for traceability, parse PDFs with ai_parse_document, remove boilerplate while preserving structure, drop near-duplicates with MinHash and filter harmful documents with a toxicity classifier. Content that must be kept, such as superseded versions, is excluded with metadata filters at retrieval time.

6. Evaluation and Monitoring

12–12%

Evaluation and Monitoring

Choose the LLM by evaluating every candidate on the same dataset with the same scorers, discarding those that miss hard requirements, then comparing cost and speed. Total latency is time to first token plus time per output token times the tokens generated; throughput matters most for batch jobs. An MoE uses only part of its parameters per input, and quantizing a model means repeating the evaluation.
Choose metrics by the risk of the use case: retrieval quality (chunk precision and recall), response quality (groundedness, relevance, safety, correctness) and system performance (tokens as cost, latency). For RAG, measure both retrieval and response so you can locate the cause of a failure; endpoint health metrics cover errors, latency, and CPU and memory usage.
mlflow.genai.evaluate() runs the agent (predict_fn) over an evaluation dataset, captures traces and applies scorers; it can also score existing outputs (answer sheet) or traces fetched with mlflow.search_traces(). The keys in inputs must match the predict_fn arguments, and retrieval judges need a RETRIEVER span in the trace.
Inference tables automatically log an endpoint's requests and responses to a Unity Catalog Delta table; for agents they also store MLflow traces, and deploy() enables them by default. Use them to debug, audit and build evaluation sets by joining them with labels. You cannot point them at an existing table or change their schema or name, and some error responses may not be logged.
To control costs: pay-per-token for pilots with irregular traffic and provisioned throughput when guarantees are needed; scale to zero for rarely used endpoints (with a cold start); ai_query with hosted models for batch inference; endpoint tags, which reach system.billing.usage, to attribute spend; and lower sample rates for expensive production scorers.

Practice this certification