GCP Utilities

The following are applications and ulities that are used in conjunction with Google Cloud Platform (GCP).

Applications

Documentation

gcp-iam-catalog: A comprehensive catalog of GCP IAM roles and permissions, designed to easily identify which roles include a specific permission.

Containers

firestore-batch-incrementer: Iterates through a Firestore collection in batches and atomically increments a specified root‑level numeric field with configurable rate limiting.

Web Applications

gcpmetadataexplorer: A web-based interface for browsing and inspecting the GCP metadata server.

iapheaders: Displays GCP Identity-Aware Proxy headers and JWT for inspection.

gcpidentitytokenportal: Web portal for vending GCP identity tokens via metadata service with flexible audience selection.

Java Utilities

firestoreproto2map: Java helper library to convert Firestore Protocol Buffer from event to map that can be used by Firestore

firestoreproto2json: Java helper library to convert Firestore Protocol Buffer to JSON Object

simplegoogleidtoken: simplegoogleidtoken is a lightweight Java library for effortlessly exchanging Google Cloud Service Account credentials for Google ID tokens

kubetogoogleidtoken: A Java library for obtaining Google ID tokens by leveraging Kubernetes Service Accounts with GCP Workload Identity Federation.

Command Line Utilities

pubsubmsgrestforwarder: A Go command-line application for local testing, simulating the Cloud Run Push use case by consuming Pub/Sub messages and forwarding them as RESTful HTTP POST requests.

Firestore - crossfiresync

crossfiresync: A Java library enabling real-time synchronization between GCP Firestore instances across regions using Pub/Sub.

crossfiresyncrun: Provides real-time synchronization between GCP Firestore instances across regions using Pub/Sub, packaged as a Docker image for deployment on Cloud Run.

crossfiresyncrun-tofu: A module for OpenTofu that deploys crossfiresyncrun to GCP Cloud Run, along with configuring essential services including Firestore and Pub/Sub.

crossfiresync-firestore: Reference implementation of a crossfiresync Firestore publisher, featuring Java code and deployment scripts for Cloud Functions.

crossfiresync-pubsub: Reference implementation of a crossfiresync Pub/Sub consumer, featuring Java code and deployment scripts for Cloud Functions.

Pub/Sub

http-response-collector - Retrieves HTTP responses and headers from specified endpoints and publishes the collected data to Google Cloud Pub/Sub for further processing.

KMS - lockboxkms

lockboxkms: A simple web interface for encrypting text using Google Cloud KMS.

lockboxkms-secretmanager-tofu: OpenTofu module for decrypting value using KMS and creating a secret with that value in GCP

Terraform/OpenTofu Modules

gcp-cloud-run-psc-lb-tofu: Demonstrates how to expose a private Cloud Run service using Private Service Connect and an internal HTTPS load balancer.

gcp-cloud-run-lb-nipio-tofu: Deploys a global load-balanced Cloud Run service using nip.io for automatic SSL certificates.

gcp-cloud-run-iap-authui-tofu: Deploys GCP’s IaP authui-container to Cloud Run as an internet facing endpoint.

firestore-to-bigquery-tofu: This module automates the scheduled export of Firestore data by triggering Cloud Run jobs that export to Cloud Storage and load the data into BigQuery.

Data Replication

firepubauditsource: Publishes Firestore data changes to Pub/Sub as JSON audit records for downstream processing.

firepubauditsource-tofu: A module for OpenTofu that deploys firepubauditsource to GCP Cloud Run, along with configuring essential services including Eventarc for Firestore and Pub/Sub.

bqpubauditsink: Ingests Pub/Sub audit JSON events and inserts the records into BigQuery.

bqpubauditsink-tofu: A module for OpenTofu that deploys bqpubauditsink to GCP Cloud Run, along with configuring essential services including the Pub/Sub subscription and BigQuery dataset and table.

valkeypubauditsink: Ingests Pub/Sub audit JSON events and synchronizes the records into Valkey (Redis).