Which EC2 pricing model can provide a discount by paying for capacity ahead of time?

BigQuery is a serverless data analytics platform. You don't need to provision individual instances or virtual machines to use BigQuery. Instead, BigQuery automatically allocates computing resources as you need them. You can also reserve compute capacity ahead of time in the form of slots, which represent virtual CPUs. The pricing structure of BigQuery reflects this design.

Overview of BigQuery pricing

BigQuery pricing has two main components:

• Analysis pricing is the cost to process queries, including SQL queries, user-defined functions, scripts, and certain data manipulation language (DML) and data definition language (DDL) statements that scan tables.

• Storage pricing is the cost to store data that you load into BigQuery.

BigQuery charges for other operations, including using BigQuery Omni, BigQuery ML, BI Engine, and streaming reads and writes.

In addition, BigQuery has free operations and a free usage tier.

Each project that you create has a billing account attached to it. Any charges incurred by BigQuery jobs run in the project are billed to the attached billing account. BigQuery storage charges are also billed to the attached billing account. You can view BigQuery costs and trends by using the Cloud Billing reports page in the Google Cloud console.

Analysis pricing models

BigQuery offers a choice of two pricing models for running queries:

• On-demand pricing. With this pricing model, you are charged for the number of bytes processed by each query. The first 1 TB of query data processed per month is free.

• Flat-rate pricing. With this pricing model, you purchase slots, which are virtual CPUs. When you buy slots, you are buying dedicated processing capacity that you can use to run queries. Slots are available in the following commitment plans:

  • Flex slots: You commit to an initial 60 seconds.
  • Monthly: You commit to an initial 30 days.
  • Annual: You commit to 365 days.

  With monthly and annual plans, you receive a lower price in exchange for a longer-term capacity commitment.

You can combine both models to fit your needs. With on-demand pricing, you pay for what you use. However, your queries run using a shared pool of slots, so performance can vary. With flat-rate pricing, you purchase guaranteed capacity, with a discounted price for a longer-term commitment. For more information about which pricing to choose for your workloads, see Workload management using Reservations.

On-demand analysis pricing

By default, queries are billed using the on-demand pricing model, where you pay for the data scanned by your queries.

With on-demand pricing, you will generally have access to up to 2,000 concurrent slots, shared among all queries in a single project. Periodically, BigQuery will temporarily burst beyond this limit to accelerate smaller queries. In addition, you might occasionally have fewer slots available if there is a high amount of contention for on-demand capacity in a specific location.

On-demand query pricing is as follows:

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Flat-rate pricing is also available for high-volume customers that prefer a stable, monthly cost.

Pricing details

Note the following regarding on-demand query charges:

• BigQuery uses a columnar data structure. You're charged according to the total data processed in the columns you select, and the total data per column is calculated based on the types of data in the column. For more information about how your data size is calculated, see Data size calculation.
• You aren't charged for queries that return an error or for queries that retrieve results from the cache. For procedural language jobs this consideration is provided at a per-statement level.
• Charges are rounded up to the nearest MB, with a minimum 10 MB data processed per table referenced by the query, and with a minimum 10 MB data processed per query.
• Canceling a running query job might incur charges up to the full cost for the query if you let the query run to completion.
• When you run a query, you're charged according to the data processed in the columns you select, even if you set an explicit LIMIT on the results.
• Partitioning and clustering your tables can help reduce the amount of data processed by queries. As a best practice, use partitioning and clustering whenever possible.
• On-demand query pricing is referred to as analysis pricing on the Google Cloud SKUs page.
• When you run a query against a clustered table, and the query includes a filter on the clustered columns, BigQuery uses the filter expression to prune the blocks scanned by the query. This can reduce the number of scanned bytes.
• When querying an external data source from BigQuery, you are charged for the number of bytes read by the query. If the external data is stored in another Google Cloud product such as Cloud Storage, any storage costs for that product apply as well. For more information, see Google Cloud pricing.

BigQuery provides cost control mechanisms that enable you to cap your query costs. You can set:

• User-level and project-level custom cost controls
• The maximum bytes billed by a query

For detailed examples of how to calculate the number of bytes processed, see Query size calculation.

Flat-rate analysis pricing

BigQuery offers flat-rate pricing for customers who prefer a stable cost for queries rather than paying the on-demand price per TB of data processed.

To enable flat-rate pricing, use BigQuery Reservations.

When you enroll in flat-rate pricing, you purchase dedicated query processing capacity, measured in BigQuery slots. Your queries consume this capacity, and you are not billed for bytes processed. If your capacity demands exceed your committed capacity, BigQuery will queue up slots, and you will not be charged additional fees. For more information about how BigQuery leverages slots for query processing, see Slots.

Flat-rate pricing:

• Applies to query costs, including BigQuery ML, DML, and DDL statements.
• Does not apply to storage costs or BI Engine costs.
• Does not apply to streaming inserts and using the BigQuery Storage API.
• Is purchased as a regional resource. Slot commitments purchased in one region or multi-region cannot be used in another region or multi-region and cannot be moved.
• Allows customers to raise per-project concurrency quotas by contacting Google Cloud Support.
• Is available in per-second, monthly, and annual commitments.
• Can be shared across your entire organization. There is no need to buy slot commitments for every project.
• Has a 100-slot minimum and is purchased in increments of 100 slots.
• Is billed per second until you cancel the commitment, which can be done at any time after the commitment end date.

Monthly flat-rate commitments

The following table shows the cost of your monthly slot commitment. For more information, see Monthly commitments.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Annual flat-rate commitments

The following table shows the cost of your annual slot commitment. For more information, see Annual commitments.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Flex slots: short-term commitments

Flex slots are a special commitment type:

• Commitment duration is only 60 seconds.
• You can cancel Flex slots any time thereafter.
• You are charged only for the seconds your commitment was deployed.

Flex slots are subject to capacity availability. When you attempt to purchase Flex Slots, success of this purchase is not guaranteed. However, once your commitment purchase is successful, your capacity is guaranteed until you cancel it. For more information, see Flex slots.

The following table shows the cost of your Flex slot commitment.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Storage pricing

Storage pricing is the cost to store data that you load into BigQuery. You pay for active storage and long-term storage.

• Active storage includes any table or table partition that has been modified in the last 90 days.

• Long-term storage includes any table or table partition that has not been modified for 90 consecutive days. The price of storage for that table automatically drops by approximately 50%. There is no difference in performance, durability, or availability between active and long-term storage.

The first 10 GB of storage per month is free.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Pricing details

Storage pricing is based on the amount of data stored in your tables. The size of the data is calculated based on the data types of the individual columns. For a detailed explanation of how data size is calculated, see Data size calculation.

Storage pricing is prorated per MB, per second. For example, if you store:

• 100 MB for half a month, you pay $0.001 (a tenth of a cent)
• 500 GB for half a month, you pay $5
• 1 TB for a full month, you pay $20

Storage usage is calculated in gigabytes (GB), where 1 GB is 230 bytes. This unit of measurement is also known as a gibibyte (GiB). Similarly, 1 TB is 240 bytes (1,024 GB).

If a table is not edited for 90 consecutive days, it is billed at the long- term storage rate. There is no degradation of performance, durability, availability, or any other functionality when a table is considered long-term storage.

Each partition of a partitioned table is considered separately for long-term storage pricing. If a partition hasn't been modified in the last 90 days, the data in that partition is considered long term storage and is charged at the discounted price.

If the table is edited, the price reverts back to the regular storage pricing, and the 90-day timer starts counting from zero. Anything that modifies the data in a table resets the timer, including:

All other actions do not reset the timer, including the following:

• Querying a table
• Creating a view that queries a table
• Exporting data from a table
• Copying a table (to another destination table)
• Patching or updating a table resource

For tables that reach the 90-day threshold during a billing cycle, the price is prorated accordingly.

Long-term storage pricing applies only to BigQuery storage, not to data stored in external data sources such as Cloud Bigtable, Cloud Storage, and Drive.

Data size calculation

When you load data into BigQuery or query the data, you're charged according to the data size. Data size is calculated based on the size of each column's data type.

The size of your stored data, and the size of the data processed by your queries is calculated in gigabytes (GB), where 1 GB is 230 bytes. This unit of measurement is also known as a gibibyte (GiB). Similarly, 1 TB is 240 bytes (1,024 GB).

For more information, see Data type sizes.

Data Transfer Service pricing

The BigQuery Data Transfer Service charges monthly on a prorated basis. You are charged as follows:

Notes on transfer pricing

All transfers

1. After data is transferred to BigQuery, standard BigQuery storage and query pricing applies. For additional pricing details, contact Sales.

Migrations from other platforms

2. Extraction, uploading to a Cloud Storage bucket, and loading data into BigQuery is free.

3. Costs can be incurred outside of Google by using the BigQuery Data Transfer Service, such as platform egress charges.

Teradata migrations

4. Data is not automatically deleted from your Cloud Storage bucket after it is uploaded to BigQuery. Consider deleting the data from your Cloud Storage bucket to avoid additional storage costs. See Cloud Storage pricing.

Third-party Connectors

5. Costs apply for connectors provided by third-party partners. The pricing model differs for different partners and connectors. For more pricing details, refer to individual connectors when enrolling in Marketplace.

Calculating unique IDs

Each transfer you create generates 1 or more runs per day. Each run maintains a record of each unique ID encountered and the date the transfer run completes. IDs are only counted on the day the transfer completes. For example, if a transfer run begins on July 14th but completes on July 15th, the unique IDs are counted on July 15th.

If a unique ID is encountered in more than one transfer run on a particular day, it is counted only once. Unique IDs are counted separately for different transfers. If a unique ID is encountered in runs for two separate transfers, the ID is counted twice.

Calculating backfill charges

If you schedule a backfill, a transfer run is scheduled for each day. You are then charged based on the method described in Calculating unique IDs.

Stopping BigQuery charges

To stop incurring charges, disable or delete your transfer.

BigQuery Data Transfer Service pricing examples

Example 1: You have 1 transfer with 3 runs that complete on the same day.

• The first run records the following unique IDs: A, B, and C
• The second run records: A
• The third run records: C and D

Because all runs finish on the same day, you are charged based on 4 unique IDs: A, B, C, D. Because ID A and ID C were recorded in two different runs that completed on the same day, IDs A and C are counted only once. If the 3 transfer runs complete every day for a month, your monthly charge is based on 4 unique IDs. If the transfer runs complete fewer times than the number of days in the month in which they run, the charges are prorated.

Example 2: You have multiple transfers with runs that complete on the same day.

• Transfer 1 runs and records the following unique IDs: A, B, and C
• Transfer 2 runs and records: A
• Transfer 3 runs and records: C and D

Because the unique IDs are counted in runs for different transfers, you are charged based on 6 unique IDs: A, B, and C from transfer 1's run; A from transfer 2's run; and C and D from transfer 3's run. If the transfer runs complete fewer times than the number of days in the month in which they run, the charges are prorated.

BigQuery Omni pricing

BigQuery Omni offers the following pricing models depending on your workloads and needs.

On-Demand pricing (limited time offer)

For a limited trial, BigQuery customers can explore BigQuery Omni at no charge using on-demand byte scans from September 15, 2022 to March 31, 2023 (the "trial period"). BigQuery customers interested in exploring BigQuery Omni by using an on-demand bytes scanned model are eligible for this trial. This trial ends either two months after your first use of on-demand queries during the trial period, or at the end of the trial period, whichever comes first. While there is no charge for the bytes scanned by your BigQuery Omni on-demand queries during the trial period*, the BigQuery free tier data processing limit of 1 TB per project per month does apply. This trial is available in existing BigQuery Omni regions and may be limited based on the availability of BigQuery Omni resources. No additional sign-up is required to access the trial once you set up your BigQuery Omni region.

* This limited trial period is intended to let customers experiment with BigQuery Omni. It is not intended for large-scale workloads or queries. At the end of the trial period, customers must move to a paid pricing model to continue running queries in BigQuery Omni. Google reserves the right to end this trial at any time.

BigQuery Omni also offers flat-rate pricing which provides a predictable cost for queries. To enable flat-rate pricing, use BigQuery Reservations.

When you enroll in flat-rate pricing for BigQuery Omni, you purchase dedicated query processing capacity, measured in slots, on Amazon Web Services or Microsoft Azure. Your queries consume this capacity, and you are not billed for bytes processed.

Flat-rate pricing:

• Applies to query costs. Does not apply to storage costs.
• Slot commitments are purchased for a single multi-cloud region. Slots purchased in one region cannot be used in another region.
• Is available in monthly, and annual commitments. Is billed per second until you cancel the commitment, which can be done at any time after the commitment end date.
• Can be shared across your entire organization. There is no need to buy slot commitments for every project.
• Has a 100-slot minimum and is purchased in increments of 100 slots.

Monthly flat-rate commitments

The following table shows the cost of your monthly slot commitment. For more information, see Monthly commitments.

Annual flat-rate commitments

The following table shows the cost of your annual slot commitment. For more information, see Annual commitments.

Flex slots: short-term commitments

Flex slots are a special commitment type:

• Commitment duration is only 60 seconds.
• You can cancel Flex slots any time thereafter.
• You are charged only for the seconds your commitment was deployed.

Flex slots on BigQuery Omni are subject to capacity availability on AWS or Azure. When you attempt to purchase Flex Slots, success of this purchase is not guaranteed. However, once your commitment purchase is successful, your capacity is guaranteed until you cancel it. For more information, see Flex slots.

The following table shows the cost of your Flex slot commitment.

Omni Internet Data Transfer

When using the Cross Cloud Transfer in preview, there will be no additional charges for data transfer.

Starting 1st November 2022, this service will be generally available and you will be charged for data transfer. You will be charged for data transfer only when loading files or copying tables from an AWS or Azure region to a GCP BigQuery region. You will be charged on a per GB rate based on the amount of data transferred from AWS or Azure to GCP

Data ingestion pricing

BigQuery offers two modes of data ingestion:

• Batch loading. Load the source data into one or more BigQuery tables in a single batch operation.

• Streaming. Stream data one record at a time or in small batches.

For more information about which mode to choose, see Introduction to loading data.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Pricing details

By default, you are not charged for batch loading data from Cloud Storage or from local files into BigQuery. Load jobs by default use a shared pool of slots. BigQuery does not make guarantees about the available capacity of this shared pool or the throughput you will see. Alternatively, you can purchase dedicated slots to run load jobs. You are charged flat-rate pricing for dedicated slots. When load jobs are assigned to a reservation, they lose access to the free pool. For more information, see Assignments.

Once your data is loaded into BigQuery, it is subject to BigQuery storage pricing. If you load data from Cloud Storage, you are charged for storing the data in Cloud Storage. For details, see Data storage on the Cloud Storage pricing page.

If the target dataset is located in the US multi-region, you are not charged for network egress when loading from a Cloud Storage bucket in any other region. For more information, see Location considerations.

BigQuery offers two modes of data extraction:

• Batch export. Export table data to Cloud Storage.

• Streaming reads. Use the Storage Read API to perform streaming reads of table data.

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

BigQuery Storage Read API Network Egress Within Google Cloud

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

BigQuery Storage Read API General Network Usage

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Exporting data

By default, you are not charged for exporting data from BigQuery. Export jobs by default use a shared pool of slots. BigQuery does not make guarantees about the available capacity of this shared pool or the throughput you will see. Alternatively, you can purchase dedicated slots to run export jobs. You are charged flat-rate pricing for dedicated slots. When export jobs are assigned to a reservation, they lose access to the free pool. For more information, see Assignments.

Storage Read API pricing

The Storage Read API has an on-demand price model. With on-demand pricing, BigQuery charges for the number of bytes processed (also referred to as bytes read). On-demand pricing is solely based on usage, with a bytes read free tier of 300 TB per month for each billing account. Bytes scanned as part of reads from temporary tables are free and do not count against the 300TB free tier. This free bytes read 300 TB is on the bytes-read component, and does not apply to associated egress.

Note the following regarding Storage Read API charges:

• You are charged according to the total amount of data read. The total data read per column is calculated based on the type of data in the column, and the size of the data is calculated based on the column's data type. For a detailed explanation of how data size is calculated, see Data size calculation.
• You are charged for any data read in a read session even if a ReadRows call fails.
• If you cancel a ReadRows call before the end of the stream is reached, you are charged for any data read before the cancellation. Your charges can include data that was read but not returned to you before the cancellation of the ReadRows call.
• As a best practice, use partitioned and clustered tables whenever possible. You can reduce the amount of data read by using a WHERE clause to prune partitions. For more information, see Querying partitioned tables.

BigQuery ML pricing

BigQuery ML models can be classified into two different categories: built-in models and external models. BigQuery ML built-in models are trained within BigQuery, such as linear regression, logistic regression, kmeans, matrix factorization, and time series models. BigQuery ML external models are trained utilizing other Google Cloud services, DNN, boosted tree and random forest (which are trained on Vertex AI) and AutoML models (which are trained on the AutoML Tables backend). BigQuery ML model training pricing is based on the model type as well as your usage pattern: flat-rate or on-demand. BigQuery ML prediction and evaluation functions are executed within BigQuery ML for all model types, priced as explained below.

BigQuery ML flat-rate pricing

BigQuery offers flat-rate pricing for high-volume or enterprise customers who prefer a stable monthly cost rather than paying the on-demand price for model creation, evaluation, inspection, and prediction.

Customers can use reservations to train in BigQuery ML models. And the BigQuery ML costs are included in the monthly BigQuery flat-rate price.

Reservations to create built-in models

BigQuery has three job types for reservation assignment: QUERY, PIPELINE, and ML_EXTERNAL. QUERY assignments, which are used for analytical queries, are also used to run CREATE MODEL queries for BigQuery ML built-in models. Built-in model training and analytical queries share the same pool of resources in their assigned reservations, and have the same behavior regarding being preemptible, and using idle slots from other reservations.

Reservations to create external models

Because external models are trained outside of BigQuery, these workloads are not preemptible. As a result, to ensure other workloads are not impacted, only reservations with ML_EXTERNAL job type assignment can be used for these external jobs. Reservations workload management describes how to create reservations for external model training jobs. The slots usage per job is calculated to maintain the price parity between BigQuery slot and external Google Cloud service costs.

BigQuery ML on-demand pricing

BigQuery ML pricing for on-demand queries depends on the type of operation: model type, model creation, model evaluation, model inspection, or model prediction.

BigQuery ML on-demand pricing is as follows:

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

1 The CREATE MODEL statement stops at 50 iterations for iterative models. This applies to both on-demand and flat-rate pricing.

2 For time series models, when auto-arima is enabled for automatic hyper-parameter tuning, multiple candidate models are fitted and evaluated during the training phase. In this case, the number of bytes processed by the input SELECT statement is multiplied by the number of candidate models, which can be controlled by the AUTO_ARIMA_MAX_ORDER training option. This applies to both on-demand and flat-rate pricing. The following notes apply to time series model creation:

• For single time series forecasting with auto-arima enabled, when AUTO_ARIMA_MAX_ORDER is (1, 2, 3, 4, 5), the number of candidate models is (6, 12, 20, 30, 42) respectively if non-seasonal d equals one; otherwise, the number of candidate models is (3, 6, 10, 15, 21).

• For multiple time series forecasting using TIME_SERIES_ID_COL, the charge is for (6, 12, 20, 30, 42) candidate models when AUTO_ARIMA_MAX_ORDER is (1, 2, 3, 4, 5) respectively.

• Note that this model selection only applies to model creation. For model evaluation, inspection, and prediction, only the selected model is used, with regular query pricing.

BigQuery ML dry run

Due to the nature of the underlying algorithms of some model types and differences in billing, the bytes processed will not be calculated for some model types until after training is completed due to the complexity of calculating the initial estimate.

BigQuery ML pricing example

BigQuery ML charges are not itemized separately on your billing statement. For current models, if you have a BigQuery flat-rate plan, BigQuery ML charges are included.

If you are using on-demand pricing, BigQuery ML charges are included in the BigQuery analysis (query) charges.

BigQuery ML jobs that perform inspection, evaluation, and prediction operations incur the same charges as on-demand query jobs. Because CREATE MODEL queries incur different charges, you must calculate CREATE MODEL job costs independently by using the Cloud logging audit logs. Using the audit logs, you can determine the bytes billed by the BigQuery ML service for each BigQuery ML CREATE MODEL job. Then, multiply the bytes billed by the appropriate cost for CREATE MODEL queries in your regional or multi-regional location.

For example, to determine the cost of a query job in the US that includes a BigQuery ML CREATE MODEL statement:

1. Open the Cloud Logging page in the Google Cloud console.

2. Verify that the product is set to BigQuery.

3. Click the drop-down arrow in the "Filter by label or text search" box and choose Convert to advanced filter. This adds the following text to the filter:

   resource.type="bigquery_resource"
   

4. Add the following text on line two below the resource.type line:

   protoPayload.serviceData.jobCompletedEvent.job.jobConfiguration.query.statementType="CREATE_MODEL"
   

5. To the right of the Submit Filter button, choose the appropriate time frame from the drop-down list. For example, choosing Last 24 hours would display BigQuery ML CREATE MODEL jobs completed in the past 24 hours.

6. Click Submit Filter to display the jobs completed in the given time window.

7. After the data is populated, click View Options and choose Modify custom fields.

8. In the Add custom fields dialog, enter:

   protoPayload.serviceData.jobCompletedEvent.job.jobStatistics.totalBilledBytes
   

9. Click Save to update the results.

10. After the page is updated, the bytes billed by each BigQuery ML job appear to the right of the job's timestamp. If the bytes billed are included in the free tier, no value appears. For example:

    

11. To calculate the charges for the BigQuery ML CREATE MODEL job, multiply the bytes billed by the BigQuery ML on-demand price. In this example, the CREATE MODEL job processed 100873011200 bytes. To calculate the cost of this job in the US multi-regional location, divide the billed bytes by the number of bytes per TB, and multiply it by the model creation cost:

    100873011200/1099511627776 x $250.00 = $22.94

BI Engine pricing

BI Engine accelerates SQL queries by caching BigQuery data in memory. The amount of data stored is constrained by the amount of capacity you purchase. To purchase BI Engine capacity, create a BI Engine reservation in the project where queries will be run.

When BI Engine accelerates a query, the query stage that reads table data is free. Subsequent stages depend on the type of BigQuery pricing you’re using:

• For on-demand pricing, stages that use BI Engine are charged for 0 scanned bytes. Subsequent stages will not incur additional on-demand charges.

• For flat-rate pricing, the first stage consumes no BigQuery reservation slots. Subsequent stages use slots from the BigQuery reservation.

BI Engine pricing is as follows:

If you pay in a currency other than USD, the prices listed in your currency on Cloud Platform SKUs apply.

Flat-rate analysis bundle

When you are using BigQuery flat-rate analysis pricing, you are eligible to receive a limited amount of BI Engine capacity as part of your flat-rate price, at no extra cost, as shown in the following chart. To receive free BI Engine capacity, follow the instructions to reserve capacity in a project within the same organization as your flat-rate reservation. Free capacity is shown in your Billing Reports as a normal cost, but it is discounted as a "Spending-Based Discount".

Free operations

The following BigQuery operations are free of charge in every location. Quotas and limits apply to these operations.

Free usage tier

As part of the Google Cloud Free Tier, BigQuery offers some resources free of charge up to a specific limit. These free usage limits are available during and after the free trial period. If you go over these usage limits and are no longer in the free trial period, you will be charged according to the pricing on this page.

What's next

• For information on analyzing billing data using reports, see View your billing reports and cost trends.

• For information on analyzing your billing data in BigQuery, see Export Cloud Billing data to BigQuery.

• For information about estimating costs, see Estimating storage and query costs.

• Read the BigQuery documentation.

• Get started with BigQuery.

• Try the Pricing calculator.

• Learn about BigQuery solutions and use cases.