CertifyDeckCargando contenido…
The official Google Cloud Certified Associate Data Practitioner syllabus, organized by domain. Check off each point as you review it.
Inicia sesión para guardar tu progreso entre sesiones.
Prepare and process data
In ETL, data is transformed before it reaches BigQuery (Dataflow, Cloud Data Fusion), useful with an existing process or to save BigQuery resources; in ELT, raw data is loaded and transformed inside with SQL and Dataform, the approach Google generally recommends. ETLT combines a light pre-load step, such as removing sensitive data, with SQL transformations after loading; exporting results to other systems is reverse ETL.
Introduction to loading, transforming, and exporting data ↗Migrate data pipelines ↗
Storage Transfer Service is the managed service for online copies from Amazon S3, Azure, public URLs, HDFS, on-premises file systems with agents or other buckets, on a schedule or event-driven. Transfer Appliance is a physical device for large volumes when bandwidth is lacking; gcloud storage suits small, ad hoc copies. BigQuery Data Transfer Service loads SaaS or Cloud Storage data into BigQuery, and Database Migration Service moves databases.
Data transfer options ↗Migrate to Google Cloud: Transfer your large datasets ↗
Knowledge Catalog (formerly Dataplex) offers profile scans, with column statistics such as nulls, distinct and top values, and auto data quality scans, which validate BigQuery tables with built-in or SQL rules, thresholds, incremental scope, scores and email reports. In Dataform, assertions fail if their query returns rows; SQL can also count nulls and duplicates.
BigQuery data preparation cleans BigQuery tables with Gemini suggestions, filter, deduplicate and validation steps with an error table; SQL offers SAFE_CAST, TRIM, UPPER, UPDATE and ROW_NUMBER. Cloud Data Fusion Wrangler cleans visually before loading, batch only; Dataflow cleans in streaming or batch, with templates and UDFs or Apache Beam, and sends failed rows to a dead-letter table.
Extract and load data into appropriate Google Cloud storage systems
BigQuery loads Avro, CSV, JSON, ORC and Parquet data. Avro, Parquet and ORC are self-describing: BigQuery reads the schema from the data and can read them in parallel even with internal compression codecs. CSV doesn't support nested or repeated data, JSON must be newline-delimited, and CSV and JSON need schema auto-detection or an explicit schema; gzip is their only supported compression and slows the load.
BigQuery Data Transfer Service schedules managed loads into BigQuery from SaaS applications, Cloud Storage, other clouds and data warehouses, with no code and with backfills. Database Migration Service migrates databases into Cloud SQL and AlloyDB for PostgreSQL, continuously or one time. Dataflow runs Apache Beam pipelines or Google-provided templates, and Cloud Data Fusion builds visual, no-code pipelines.
Introduction to the BigQuery Data Transfer Service ↗Database Migration Service overview ↗
BigQuery is the store for SQL analysis of large structured datasets, and Cloud Storage holds unstructured objects of any format. Cloud SQL is the managed relational database for MySQL, PostgreSQL and SQL Server; Spanner is relational with transactional consistency at global scale; Bigtable is a low-latency key-value store for time-series and IoT data; Firestore is the serverless document database.
A regional bucket stores data across zones of one region with no replication charges; dual-regions and multi-regions replicate asynchronously across regions and keep serving in a regional outage, and dual-regions let you pick the regions and enable turbo replication. In BigQuery a multi-region gives no regional redundancy; Cloud SQL high availability protects against a zone outage, not a regional one.
Structured data follows a fixed schema and fits relational databases such as Cloud SQL or BigQuery tables; semi-structured data such as JSON carries keys but no rigid schema and fits Firestore or BigQuery JSON columns. Unstructured data (images, audio, video, free text) is stored as objects in Cloud Storage and analyzed from BigQuery through object tables.
What is unstructured data? ↗Work with JSON data in GoogleSQL ↗
bq load (or the LOAD DATA statement) loads local or Cloud Storage files into BigQuery tables, and gcloud storage cp or rsync upload files to buckets. Storage Transfer Service moves data into Cloud Storage from other clouds, URLs or on-premises file systems with agents; BigQuery Data Transfer Service schedules loads into BigQuery from Cloud Storage and SaaS sources, and client libraries do it from code.