> For the complete documentation index, see [llms.txt](https://docs.controltheory.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.controltheory.com/controltheory-documentation/dstl8-docs/sources/google-cloud.md).

# Google Cloud

Google Cloud Logging collects the logs of every service in a GCP project: GKE containers, Cloud Run revisions, Cloud Functions, Compute Engine instances, Cloud SQL, App Engine, load balancers, and audit logs. Dstl8 polls the Cloud Logging API as a single source, covering one project, and infers a stream from the GCP resource that produced each log.

### Add a Google Cloud source

The Project ID and the service account key both come from Google Cloud, so collect them before you open Dstl8.

#### 1. Create a service account and key

In the Google Cloud console, with the project you want to collect from selected:

1. Go to **IAM & Admin → Service Accounts** and click **Create service account**. Give it a name you'll recognize, such as `dstl8-log-reader`.
2. Grant it the **Logs Viewer** role (`roles/logging.viewer`) on the project, then finish creating the account.
3. Open the service account, go to the **Keys** tab, and click **Add key → Create new key → JSON**. The key file downloads once, and its contents are what you paste into Dstl8.

The same thing from the command line:

```bash
# 1) create the service account
gcloud iam service-accounts create dstl8-log-reader \
  --display-name="Dstl8 log reader"

# 2) grant read-only access to the project's logs
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:dstl8-log-reader@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/logging.viewer"

# 3) create the JSON key; paste this file's contents into Dstl8
gcloud iam service-accounts keys create dstl8-key.json \
  --iam-account="dstl8-log-reader@PROJECT_ID.iam.gserviceaccount.com"
```

Your **Project ID** is the string in the console's project picker, e.g. `my-gcp-project`, not the display name and not the numeric project number. `gcloud config get-value project` prints it.

> **Logs Viewer** covers everything except Data Access audit logs, which Cloud Logging treats as private. Grant **Private Logs Viewer** (`roles/logging.privateLogViewer`) instead if you need those.

#### 2. Open the Add Source dialog

In Dstl8, from **Sources**, click **Add Source** and select **Google Cloud**.

#### 3. Configure the source

In the **Configure Google Cloud Source** dialog:

| Field                          | Notes                                                                      |
| ------------------------------ | -------------------------------------------------------------------------- |
| **Name**                       | A unique name to identify this source.                                     |
| **Service Account Key (JSON)** | The full contents of the JSON key file you downloaded.                     |
| **Project ID**                 | The GCP project whose logs you want to collect.                            |
| **What to collect**            | `All logs` *(default)*, `Categories`, or `Custom filter (LQL)`. See below. |
| **Poll interval**              | How often Dstl8 polls Cloud Logging: `1m` *(default)* or `5m`.             |

**What to collect** decides how much of the project Dstl8 ingests:

* **All logs**: everything the service account is allowed to read.
* **Categories**: check any of GKE (Kubernetes), Cloud Run, Cloud Functions, Compute Engine, Cloud SQL, App Engine, Load Balancing, and Audit Logs. Dstl8 turns the selection into a Cloud Logging filter for you.
* **Custom filter (LQL)**: write the filter yourself in Cloud Logging query language, e.g. `resource.type="gce_instance" AND severity>=WARNING`. See [Google's query language reference](https://cloud.google.com/logging/docs/view/logging-query-language) for the full syntax.

Whichever you pick, Cloud Logging applies the filter server-side, so anything it excludes is never sent to Dstl8.

#### 4. Create

Click **Create** to add the source.

### After creating

The source appears in the Sources list as **Waiting for logs**. Streams usually appear within a couple of minutes, and the source transitions to **Healthy**. Logs trail real time by about two minutes, the lag Cloud Logging needs before entries are settled enough to read reliably.

Each stream's **Stream Type** comes from the GCP resource that produced the log:

| GCP resource    | Stream Type       | Stream name                     |
| --------------- | ----------------- | ------------------------------- |
| GKE (`k8s_*`)   | `gke`             | Kubernetes namespace            |
| Cloud Run       | `cloud-run`       | Service or job name             |
| Cloud Functions | `cloud-functions` | Function name                   |
| Compute Engine  | `gce`             | Log ID, e.g. `syslog`           |
| Cloud SQL       | `cloud-sql`       | Database ID                     |
| App Engine      | `app-engine`      | Module ID                       |
| Load Balancing  | `load-balancer`   | Log ID, e.g. `requests`         |
| Audit logs      | `audit`           | Audit log type, e.g. `activity` |

Anything else keeps its raw GCP resource type as the Stream Type, e.g. `pubsub_topic`, falling back to `unknown` when GCP reports no resource type. Each log keeps its GCP resource labels, severity, trace IDs, and HTTP request details as attributes you can filter on in Dstl8.

### Assign streams to workspaces

Streams need to be assigned to a workspace to be accessible. Open the source's actions menu (gear icon) and select **Assign Streams**, then check the streams you want the **Default** workspace (or any other workspace) to receive and click **Save Assignments**.

See [Assigning streams to workspaces](/controltheory-documentation/dstl8-docs/sources.md#assigning-streams-to-workspaces) for details.

### Tips

* One source covers one project. To collect from several projects, add a source per project, each with a service account key from that project.
* Cloud Logging's read quota is per project (60 requests/minute), and Dstl8 stays well under it. Sources pointed at the same project share that budget, so raise **Poll interval** to `5m` if you're running more than one.
* Cloud Logging matches log IDs exactly rather than by prefix, so to collect audit logs specifically, name them in full in a custom filter: `log_id("cloudaudit.googleapis.com/activity")`, and likewise `data_access`, `system_event`, or `policy`.
* If no logs arrive, check the service account against the **Project ID** you entered. The two are set independently, so a valid key for a project the service account can't read fails only once Dstl8 polls. Confirm the **Logs Viewer** role is granted on that project and that the Cloud Logging API is enabled for it.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.controltheory.com/controltheory-documentation/dstl8-docs/sources/google-cloud.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.