Professional Machine Learning Engineer · CHEAT SHEET
Google Cloud Certified Professional Machine Learning Engineer
Key concepts grouped by official exam weight.
Download as PDF (free, no sign-up)1. Architecting low-code AI solutions
13–13%Developing ML models using BigQuery ML or AutoML on Gemini Enterprise Agent Platform
- Pick the model type from the problem: LOGISTIC_REG, BOOSTED_TREE_* or AUTOML_CLASSIFIER to classify; LINEAR_REG or the regressors for continuous values; KMEANS to segment without labels; MATRIX_FACTORIZATION for recommendations; ARIMA_PLUS (univariate, many series with TIME_SERIES_ID_COL) or ARIMA_PLUS_XREG (with external variables) to forecast. ARIMA_PLUS is the cheap, fast baseline against AutoML neural forecasting.
- BigQuery ML preprocesses automatically (mean imputation, standardization, one-hot encoding and TIMESTAMP component extraction). For your own rules use the TRANSFORM clause with functions such as ML.IMPUTER, ML.QUANTILE_BUCKETIZE or ML.FEATURE_CROSS: training statistics are reused in ML.PREDICT, avoiding training-serving skew. ML.TRANSFORM, TRANSFORM_ONLY models, ML.FEATURE_IMPORTANCE and ML.GLOBAL_EXPLAIN help inspect and select features.
- ML.PREDICT scores tables or queries in batch inside BigQuery without moving data, accepts THRESHOLD for binary models and applies the TRANSFORM clause automatically. ARIMA_PLUS uses ML.FORECAST, ML.EXPLAIN_PREDICT adds per-row attributions and ML.RECOMMEND serves matrix factorization. For online inference, register the model in Model Registry and deploy it to an endpoint; imported ONNX and remote models also use ML.PREDICT.
- AutoML splits tabular data into training, validation and test sets: random 80/10/10 by default, chronological with a Time column for time-dependent data, or manual to segregate, for example, stores. Pick the objective from the problem (AUC PR for the rare class, log loss required for multi-class), use a weight column for important rows and Tabular Workflows when you need control over each step.
- Supervised tuning of Gemini from BigQuery uses CREATE MODEL ... REMOTE WITH CONNECTION, OPTIONS(ENDPOINT = ...) and an AS SELECT clause with prompt and label columns (or PROMPT_COL and INPUT_LABEL_COLS). MAX_ITERATIONS is converted to epochs, DATA_SPLIT_METHOD holds out evaluation data for early stopping, and ML.EVALUATE compares the baseline with the tuned model. Tune when prompting is not enough and you have labeled examples.
Building AI solutions using Google Cloud AI APIs or foundational models
- Model Garden gathers foundation, fine-tunable and task-specific models from Google, partners and open sources. Use MaaS for prototyping, variable traffic and minimal operations; self-deploy for custom weights, predictable high volume, hardware control or rules that forbid multi-tenant services. Compare candidates with the Gen AI evaluation service on your data, restrict models with an organization policy and review models flagged as suspicious.
- Document AI works through processors: Enterprise Document OCR to digitize, Form Parser, Layout Parser, Custom Extractor and pretrained parsers (invoices, expenses) to extract, and Custom Classifier or Custom Splitter to classify. Vision API offers OCR, labels, logos, objects and SafeSearch. Cloud Translation Advanced adds glossaries, adaptive translation and document translation that preserves formatting.
- Start with prompting and tune only when it is not enough: supervised fine-tuning fits formatting, classification or extraction with hundreds of examples; preference tuning fits chosen/rejected pairs after a supervised step; reinforcement learning fine-tuning fits verifiable reasoning. Model Garden offers open models as a serverless managed API (MaaS) or self-deployed in your project; Veo generates video.
- Context caching cuts cost and latency for repeated content: implicit caching is automatic (put common content first) and explicit caching guarantees the discount and controls expiration. Batch inference suits non-urgent jobs. Provisioned Throughput reserves capacity at a fixed cost; Priority PayGo adds consistency without commitment; the global endpoint improves availability but does not guarantee data residency.
2. Collaborating within and across teams to manage data and models
16–16%Exploring and preprocessing data for ML
- Keep structured and semi-structured data in BigQuery, materialized for training and read with the BigQuery Storage API; images, video and audio go to Cloud Storage in large sharded containers. Object tables bring those files into SQL. AutoML tabular imports from BigQuery or CSV, images from JSON Lines or CSV, and text is now customized with Gemini prompts and tuning.
- If data is in BigQuery and the logic fits SQL, preprocess there: it is often the most cost-effective, with partitioned tables, TRANSFORM in BigQuery ML or BigQuery DataFrames for pandas-style code at scale. Dataflow fits large volumes, streaming and logic beyond SQL, with MLTransform. Managed Service for Apache Spark fits an existing Spark codebase, and pandas is enough for data that fits in memory.
- Feature Store uses BigQuery tables or views as its offline store. Feature groups register sources and columns and are required for time-series data; a feature view can combine features from several groups and syncs on a schedule or continuously to a Bigtable online serving store. Embeddings go to Vector Search, and all resources must be in the BigQuery source region.
- Sensitive Data Protection discovers PII across the organization with data profiles, inspects it in detail and de-identifies it (redaction, masking, tokenization, bucketing, date shifting). Cryptographic hashing keeps joins working without being reversible, risk analysis measures k-anonymity before sharing, and BigQuery data masking gives partial access per group.
Model prototyping using notebooks (e.g., Gemini Enterprise Agent Platform Workbench and Colab Enterprise)
- Colab Enterprise shares notebooks through IAM with users, groups or domains (Code Editor, Code Viewer and Notebook Runtime User) and lets you turn off saving output that others would see. Workbench controls access with Single user or Service account modes, security options (root, file downloading), external IP, CMEK and VPC Service Controls perimeters.
- Workbench ships JupyterLab with TensorFlow and PyTorch, conda kernels for other versions and a Managed Service for Apache Spark integration; Colab Enterprise uses runtime templates with GPUs. Fit preprocessing only on training data (scikit-learn Pipeline), call model.eval() in PyTorch, and use custom containers for JAX or private dependencies.
- Model Garden groups foundation, fine-tunable and task-specific models from Google, partners and Hugging Face; from a model card you open tuning notebooks or deploy with the SDK (OpenModel, list_deploy_options, deploy). Self-deployed models run in your project and VPC; MaaS is a serverless managed API. An organization policy limits which models are allowed.
Tracking and running ML experiments
- Agent Platform Pipelines runs Kubeflow Pipelines or TFX pipelines serverlessly (TFX recommended for TensorFlow on terabytes of data). Experiments tracks runs, and autolog() records scikit-learn or XGBoost parameters and metrics; Vertex AI TensorBoard shows metrics over time. Ray on Agent Platform reuses Ray code, and the Workbench executor schedules notebooks.
- For predictive models, Agent Platform evaluates precision, recall, AuPRC, F1, confusion matrix, RMSE or MAE from batch inference output and ground truth, also as a pipeline component; ML.EVALUATE accepts a custom threshold. For gen AI, the Gen AI evaluation service uses adaptive rubrics (GENERAL_QUALITY), static rubrics (GROUNDING, SAFETY), computation-based metrics and a configurable judge model.
- Agent Platform Experiments records parameters, summary metrics and time series metrics (stored in Vertex AI TensorBoard) and compares runs, including pipeline runs. Agent Platform ML Metadata keeps the lineage of artifacts and executions per project and region; Knowledge Catalog gives organization-wide lineage. Model Registry manages versions and mutable aliases.
3. Scaling prototypes into ML models
21–21%Building models given the task considering cost, complexity, latency, and scalability
- The task decides the model type: LOGISTIC_REG or boosted trees to classify, KMEANS to segment without labels, MATRIX_FACTORIZATION (IMPLICIT without ratings) to recommend, ARIMA_PLUS or ARIMA_PLUS_XREG (with covariates) for time series, wide-and-deep or an autoencoder for sparse data, and an LLM such as Gemini (AI.CLASSIFY) when no labels exist.
- AutoML is codeless and suits quick prototypes with predefined objectives; BigQuery ML trains with SQL without moving data out of the warehouse; serverless training accepts any framework, loss or metric; training clusters reserve guaranteed capacity. Agent Platform Pipelines orchestrates reproducible serverless workflows, and Tabular Workflows give step-by-step control over AutoML.
- Online inference requires deploying the model to an endpoint; batch inference is asynchronous on an undeployed model and the only option for AutoML tabular forecasting. A traffic split replaces a model gradually. Dedicated public endpoints isolate traffic and support streaming; dedicated private endpoints with Private Service Connect support VPC Service Controls and traffic splits.
- BigQuery Explainable AI gives local explanations with ML.EXPLAIN_PREDICT (linear and logistic regression, boosted trees, random forest) and global ones with ML.GLOBAL_EXPLAIN (needs ENABLE_GLOBAL_EXPLAIN), plus ML.FEATURE_IMPORTANCE, ML.ADVANCED_WEIGHTS with p-values and ML.EXPLAIN_FORECAST for ARIMA_PLUS. Vertex Explainable AI is deprecated: Google suggests SHAP or LIME.
Training models
- Unstructured data (images, audio, video) and formats such as TFRecord belong in Cloud Storage, read through Cloud Storage FUSE in larger and fewer files; BigQuery object tables expose them to SQL. Tabular data fits BigQuery best when large or nested. AutoML imports JSON Lines or CSV files with ml_use, and Gemini tuning uses JSON Lines in Cloud Storage.
- Unstructured data (images, text, TFRecord) usually lives in Cloud Storage, which serverless training mounts under /gcs/ with Cloud Storage FUSE; an NFS share gives low latency and POSIX semantics, and BigQuery suits structured data. Managed datasets split the data and pass URIs through AIP_TRAINING_DATA_URI. When data does not fit in memory, stream it.
- Serverless training runs code from any framework in prebuilt containers (a Python script or source distribution, low effort) or custom containers (other languages or private dependencies). A training pipeline can register the model in Model Registry. Tabular Workflows gives step-level control over AutoML; persistent resources cut startup time for short jobs; Kubeflow Trainer on GKE is portable across clusters.
- Agent Platform runs your code in Google-managed projects: set the project explicitly (CLOUD_ML_PROJECT_ID) to avoid 403 errors. For out-of-memory errors, optimize memory use or pick a larger machine; for restarts, save checkpoints at least every four hours. The interactive shell and Cloud Logging help debugging, and excellent metrics that fail in production usually point to data leakage.
- A hyperparameter tuning job runs several trials of your code, which must read hyperparameters as command-line arguments and report the metric with cloudml-hypertune. The default algorithm is Bayesian optimization; grid search needs integer, categorical or discrete parameters. Parallel trials cut wall-clock time but reduce effectiveness, and Vizier only suggests values for systems you evaluate yourself.
Choosing appropriate hardware for training
- CPUs suit prototyping and simple models; GPUs suit medium-to-large models with custom PyTorch or JAX operations; TPUs suit matrix-dominated models with no custom operations in the loop, large batches and long training, but not dynamic shapes or high-precision arithmetic. Spot VMs cut the cost of fault-tolerant jobs, and Dynamic Workload Scheduler waits for scarce GPUs.
- A distributed serverless training job uses worker pools (a primary replica in workerPoolSpecs[0], workers, parameter servers, evaluators) described in CLUSTER_SPEC or TF_CONFIG. Data parallelism replicates the model with DistributedDataParallel; if it does not fit, FSDP shards it or model parallelism splits it. TPUs suit large matrix-heavy models; a TPU VM job uses one pool with one replica.
4. Serving and scaling models
20–20%Serving models
- Online inference is synchronous against a model deployed to an endpoint; batch inference is an asynchronous BatchPredictionJob on an undeployed Model, reading and writing BigQuery or Cloud Storage, and it does not autoscale. Model Garden offers open models as serverless MaaS or self-deployed in your project and VPC. Cloud Run with GPUs is fully managed and scales to zero; GKE serves model servers such as vLLM with Kubernetes.
- Prebuilt inference containers (TensorFlow, PyTorch, scikit-learn and XGBoost) serve with minimal configuration when the artifact follows their format: SavedModel, model.mar with a handler, model.joblib or model.bst. For other frameworks, your own dependencies or extra logic, a custom container in Artifact Registry runs an HTTP server on 0.0.0.0 on the containerSpec.ports port and reads artifacts from AIP_STORAGE_URI.
- Model Registry is the central repository for the model lifecycle: custom, AutoML and BigQuery ML models (registered without export through MODEL_REGISTRY and VERTEX_AI_MODEL_ID). Each retraining is imported as a new version; an alias is a mutable, unique reference to one version, the default alias marks the version used when none is given, and labels organize models and versions. Copy model moves a model to another region or project.
- In a canary or an A/B test you deploy several models to the same endpoint and split traffic with percentages that add up to 100. The app keeps its URL, rollback is a split change, and the deployedModelId field in each response shows which model answered. A rolling deployment replaces the model and moves all traffic; private services access endpoints do not support traffic splits.
- Custom inference routines build containers around a Predictor (load, preprocess, predict, postprocess) without writing a server or Dockerfile, plus a Handler for non-JSON requests. In batch jobs, instanceConfig excludes or includes fields without code. The BigQuery ML TRANSFORM clause applies preprocessing in ML.PREDICT, Dataflow RunInference chains steps in streaming, and a response_schema fixes Gemini output.
Scaling online model serving
- Feature Store uses BigQuery tables or views as the offline store and serves the latest values from feature views in an online store; Bigtable online serving is the current type, supports CMEK and scheduled or continuous sync (continuous requires feature groups). Optimized online serving is deprecated and embeddings go to Vector Search.
- Online inference offers dedicated public endpoints (recommended: isolation, larger payloads), shared public endpoints, dedicated private endpoints with Private Service Connect (recommended for private production, many consumer projects without peering) and private services access endpoints, which allow one model per endpoint and no traffic split. On public and PSC endpoints you can add a new model with little traffic and raise it gradually.
- CPUs suit prototypes and small models; GPUs suit models with custom operations that partly run on CPU, and work only with TensorFlow SavedModels or GPU-ready containers (not scikit-learn or XGBoost); TPUs suit matrix-dominated models with static shapes and large batches. Benchmark several machine types, and Spot VMs cut the cost of interruption-tolerant jobs.
- A deployed model autoscales when maxReplicaCount exceeds minReplicaCount; autoscalingMetricSpecs can scale on CPU, GPU or request count, and quota is reserved from maxReplicaCount. Batch inference does not autoscale. Many low-traffic models can share VMs in a DeploymentResourcePool, and Gemini offers Standard PayGo, Priority PayGo, Provisioned Throughput and batch, best through the global endpoint.
- The optimized TensorFlow runtime makes inference faster and cheaper without code changes, with XLA precompilation, compression (quantization, pruning) and GPU execution for parts of tabular models. For Gemini, supervised fine-tuning cuts latency and cost by shortening prompts, and parameter-efficient tuning costs less than full fine-tuning. Hyperparameter tuning passes values as arguments and gets metrics via cloudml-hypertune.
5. Automating and orchestrating ML pipelines
18–18%Developing end-to-end ML pipelines
- In a continuous training pipeline, data validation runs before training: a schema skew stops the pipeline and a statistical change justifies retraining. Model validation compares test metrics with the current model, checks segments and tests deployment; it can be automated with evaluation components and dsl.If. In BigQuery ML, ML.VALIDATE_DATA_SKEW uses the saved training statistics.
- Agent Platform Pipelines runs Kubeflow Pipelines or TFX pipelines compiled to YAML in a serverless manner, uses prebuilt Google Cloud Pipeline Components and versioned templates in Artifact Registry, and stores lineage in ML Metadata. Managed Airflow is the recommended orchestrator when a workflow spans heterogeneous systems, and Ray on Agent Platform runs existing Ray code with minimal changes.
- Training-serving skew appears when preprocessing is duplicated outside the model. The BigQuery ML TRANSFORM clause is applied automatically by ML.PREDICT (logic in SELECT is not exported), MLTransform reuses artifacts through write_artifact_location and read_artifact_location, and Feature Store, custom inference routines or a shared component keep a single implementation.
Automating model retraining
- The retraining policy depends on how labels arrive and how expensive training is: a cron schedule when data arrives systematically, an event trigger when it arrives ad hoc, continuous evaluation when labels come quickly, and Model Monitoring drift alerts when they do not. Data validation stops the pipeline on schema skews.
- New data runs the already deployed CT pipeline; new code goes through CI/CD, which deploys a new pipeline. Cloud Build triggers run cloudbuild.yaml to test, build images tagged with $COMMIT_SHA, push them to Artifact Registry and compile the pipeline, with ignored files and required approval; Model Registry aliases promote versions.
6. Monitoring AI solutions
13–13%Identifying risks to AI solutions
- Model Armor screens prompts and responses with prompt injection and jailbreak detection, malicious URL detection, responsible AI filters and Sensitive Data Protection, in Inspect only or Inspect and block mode, through templates or floor settings. Gemini filters cover harm categories, a regex misses paraphrases, and VPC Service Controls keeps data and models inside the perimeter.
- Agent Platform measures fairness by slice: data bias metrics (Difference in Population Size, Difference in Positive Proportions in True Labels) are computed before training and model bias metrics (Accuracy, Recall or Specificity Difference) after it, with DetectDataBiasOp and DetectModelBiasOp in pipelines. For generative AI, curate tuning data, test with representative users and keep human supervision.
- Local explainability attributes each prediction to its features relative to a baseline prediction; global explainability aggregates those attributions for the whole model. BigQuery ML offers ML.EXPLAIN_PREDICT (local), ML.GLOBAL_EXPLAIN (needs ENABLE_GLOBAL_EXPLAIN), ML.FEATURE_IMPORTANCE (trees), ML.ADVANCED_WEIGHTS (p-values) and ML.EXPLAIN_FORECAST (ARIMA_PLUS). Vertex Explainable AI is deprecated; Google suggests SHAP or LIME.
Monitoring, testing, and troubleshooting AI solutions
- Model Monitoring v1 is configured on an endpoint and shares detection type, frequency and sampling across its models; v2 ties a model monitor to a Model Registry version, also works outside Agent Platform and supports scheduled runs with windows and offsets. Alerts go to email, Cloud Monitoring channels or Cloud Logging. Continuous evaluation compares inferences with newly labeled ground truth.
- Training-serving skew compares production with the training data and requires it; drift compares production with earlier production. Feature attribution drift tracks each feature's SHAP contribution and helps reveal concept drift. Distances are L-infinity for categorical and Jensen-Shannon for numerical features; for batch inference, only skew is supported, set when the job is created.
- The Gen AI evaluation service evaluates with adaptive rubrics (recommended), static rubrics, computation-based metrics such as ROUGE when ground truth exists, and custom functions; the judge model is validated against human ratings and tuned with response flipping or multi-sampling. Agents are evaluated by trajectory and final response, and MaaS models are watched with the observability dashboard and BigQuery request logs.