Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -725,6 +725,59 @@ To enable Prometheus, use the `--prometheus` flag. This will start an HTTP
server on localhost with a `/metrics` endpoint. The Prometheus namespace may
optionally be set with `--prometheus-namespace`.

### Built-in Client Metrics (BETA)

Independently of the OpenCensus metrics above, the Proxy exports a small set
of built-in client metrics to Cloud Monitoring under the
`cloudsql.googleapis.com/client/connector/` prefix, using the
`cloudsql.googleapis.com/InstanceClient` monitored resource. These are
**enabled by default** and need no `--telemetry-project`: they are written to
the **Cloud SQL instance's project**, using the same identity the Proxy uses
for the Cloud SQL Admin API.

| Metric | Description |
| --- | --- |
| `open_connection_count` | Dial attempts, labelled with `status` and `connection_part` |
| `connect_latencies` | Dial latency distribution (ms) |
| `open_connections` | Currently open connections |
| `closed_connection_count` | Closed connections |
| `connection_durations` | Connection lifetime distribution (s) |

That identity needs the `roles/monitoring.metricWriter` role on the instance's
project. Export never affects connectivity: if the role is missing, the Proxy
logs one warning per process explaining what to grant and keeps serving
connections. When using `--impersonate-service-account`, the Proxy requests the
additional `monitoring.write` scope for the impersonated token automatically;
when using `--token`, the supplied token must carry that scope (or
`cloud-platform`) for the export to succeed. In non-default Cloud universes
(`--universe-domain` other than `googleapis.com`), built-in metrics are off by
default unless explicitly enabled with `--enable-builtin-metrics=true`.

To turn the built-in metrics off:

```shell
./cloud-sql-proxy --enable-builtin-metrics=false <INSTANCE_CONNECTION_NAME>
```

To identify your application across its instances (sets the `application_name`
label, which is empty by default):

```shell
./cloud-sql-proxy --application-name my-service <INSTANCE_CONNECTION_NAME>
```

The `compute_platform` label is detected from the environment (GCE, GKE, Cloud
Run, App Engine, Cloud Functions). To set it explicitly, for example for a
platform that is not auto-detected:

```shell
./cloud-sql-proxy --compute-platform ai_studio <INSTANCE_CONNECTION_NAME>
```

All three flags can also be set with the `CSQL_PROXY_ENABLE_BUILTIN_METRICS`,
`CSQL_PROXY_APPLICATION_NAME`, and `CSQL_PROXY_COMPUTE_PLATFORM` environment
variables.

## Debug logging

To enable debug logging to report on internal certificate refresh operations,
Expand Down
43 changes: 43 additions & 0 deletions cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import (
"syscall"
"time"

"cloud.google.com/go/cloudsqlconn"
"contrib.go.opencensus.io/exporter/prometheus"
"contrib.go.opencensus.io/exporter/stackdriver"
"github.com/GoogleCloudPlatform/cloud-sql-proxy/v2/cloudsql"
Expand Down Expand Up @@ -94,6 +95,10 @@ type Command struct {
dialer cloudsql.Dialer
cleanup func() error
connRefuseNotify func()
// enableBuiltInMetrics backs --enable-builtin-metrics (default true). It
// is copied onto conf.BuiltInMetrics in parseConfig only when the flag was
// explicitly set, so that an unset flag keeps the connector's own default.
enableBuiltInMetrics bool
}

var longHelp = `
Expand Down Expand Up @@ -599,6 +604,18 @@ CPU may be throttled and a background refresh cannot run reliably
localFlags.StringVar(&c.conf.SQLDataEndpoint, "sqldata-api-endpoint", "",
"Override the SQL Data API endpoint",
)
localFlags.BoolVar(&c.enableBuiltInMetrics, "enable-builtin-metrics", true,
`Export the built-in client metrics (cloudsql.googleapis.com/client/connector/*)
to Cloud Monitoring in the Cloud SQL instance's project. The Proxy's identity
needs roles/monitoring.metricWriter on that project for the export to succeed;
without it, the Proxy logs one warning and keeps serving. Pass
--enable-builtin-metrics=false to turn the export off.`)
localFlags.StringVar(&c.conf.ComputePlatform, "compute-platform", "",
fmt.Sprintf(`Override the auto-detected compute_platform label of the built-in client
metrics. One of: %s.`, strings.Join(computePlatformNames(), ", ")))
localFlags.StringVar(&c.conf.ApplicationName, "application-name", "",
`Set the application_name label of the built-in client metrics to identify the
application across its instances (e.g., "checkout-api").`)

localFlags.BoolVar(&c.conf.RunConnectionTest, "run-connection-test", false, `Runs a connection test
against all specified instances. If an instance is unreachable, the Proxy exits with a failure
Expand Down Expand Up @@ -782,6 +799,17 @@ func instanceFromConfigFile(v *viper.Viper) []string {
return args
}

// computePlatformNames returns the valid --compute-platform values, from the
// connector, for help text.
func computePlatformNames() []string {
ps := cloudsqlconn.ComputePlatforms()
names := make([]string, len(ps))
for i, p := range ps {
names[i] = string(p)
}
return names
}

func userHasSetLocal(cmd *Command, f string) bool {
return cmd.LocalFlags().Lookup(f).Changed
}
Expand Down Expand Up @@ -897,6 +925,21 @@ and re-try with just --auto-iam-authn`)
if userHasSetLocal(cmd, "sqladmin-api-endpoint") && userHasSetLocal(cmd, "universe-domain") {
return newBadCommandError("cannot specify --sqladmin-api-endpoint and --universe-domain at the same time")
}
// Only set BuiltInMetrics when --enable-builtin-metrics was explicitly
// provided (via CLI flag, environment variable, or config file); otherwise
// leave nil so the connector's default (incl. universe-domain handling)
// applies.
if userHasSetLocal(cmd, "enable-builtin-metrics") {
v := cmd.enableBuiltInMetrics
conf.BuiltInMetrics = &v
}
if conf.ComputePlatform != "" {
// Validate early so a typo is reported as a usage error rather than
// failing later when the dialer is created.
if _, err := cloudsqlconn.ParseComputePlatform(conf.ComputePlatform); err != nil {
return newBadCommandError(fmt.Sprintf("invalid value for --compute-platform: %v", err))
}
}
if userHasSetLocal(cmd, "sqladmin-api-endpoint") && conf.APIEndpointURL != "" {
_, err := url.Parse(conf.APIEndpointURL)
if err != nil {
Expand Down
71 changes: 71 additions & 0 deletions cmd/root_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,41 @@ func TestNewCommandArguments(t *testing.T) {
LazyRefresh: true,
}),
},
{
desc: "disabling built-in metrics",
args: []string{"--enable-builtin-metrics=false", "proj:region:inst"},
want: withDefaults(&proxy.Config{
BuiltInMetrics: pointer(false),
}),
},
{
desc: "enabling built-in metrics explicitly",
args: []string{"--enable-builtin-metrics", "proj:region:inst"},
want: withDefaults(&proxy.Config{
BuiltInMetrics: pointer(true),
}),
},
{
desc: "using the compute platform flag",
args: []string{"--compute-platform", "ai_studio", "proj:region:inst"},
want: withDefaults(&proxy.Config{
ComputePlatform: "ai_studio",
}),
},
{
desc: "using the compute platform flag with mixed case",
args: []string{"--compute-platform", "Cloud_Run", "proj:region:inst"},
want: withDefaults(&proxy.Config{
ComputePlatform: "Cloud_Run",
}),
},
{
desc: "using the application name flag",
args: []string{"--application-name", "my-service", "proj:region:inst"},
want: withDefaults(&proxy.Config{
ApplicationName: "my-service",
}),
},
{
desc: "using the admin port flag",
args: []string{"--admin-port", "7777", "proj:region:inst"},
Expand Down Expand Up @@ -875,6 +910,38 @@ func TestNewCommandWithEnvironmentConfig(t *testing.T) {
AutoIP: true,
}),
},
{
desc: "using the enable-builtin-metrics envvar to disable",
envName: "CSQL_PROXY_ENABLE_BUILTIN_METRICS",
envValue: "false",
want: withDefaults(&proxy.Config{
BuiltInMetrics: pointer(false),
}),
},
{
desc: "using the enable-builtin-metrics envvar to enable",
envName: "CSQL_PROXY_ENABLE_BUILTIN_METRICS",
envValue: "true",
want: withDefaults(&proxy.Config{
BuiltInMetrics: pointer(true),
}),
},
{
desc: "using the compute-platform envvar",
envName: "CSQL_PROXY_COMPUTE_PLATFORM",
envValue: "ai_studio",
want: withDefaults(&proxy.Config{
ComputePlatform: "ai_studio",
}),
},
{
desc: "using the application-name envvar",
envName: "CSQL_PROXY_APPLICATION_NAME",
envValue: "my-service",
want: withDefaults(&proxy.Config{
ApplicationName: "my-service",
}),
},
{
desc: "using the sql-data envvar",
envName: "CSQL_PROXY_SQL_DATA",
Expand Down Expand Up @@ -1351,6 +1418,10 @@ func TestNewCommandWithErrors(t *testing.T) {
"--fuse", "myfusedir",
},
},
{
desc: "using an invalid compute platform",
args: []string{"--compute-platform", "my-laptop", "proj:region:inst"},
},
{
desc: "using both --sqladmin-api-endpoint and --universe-domain",
args: []string{
Expand Down
9 changes: 9 additions & 0 deletions docs/cmd/cloud-sql-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,16 +232,25 @@ cloud-sql-proxy INSTANCE_CONNECTION_NAME... [flags]
```
-a, --address string (*) Address to bind Cloud SQL instance listeners. (default "127.0.0.1")
--admin-port string Port for localhost-only admin server (default "9091")
--application-name string Set the application_name label of the built-in client metrics to identify the
application across its instances (e.g., "checkout-api").
-i, --auto-iam-authn (*) Enables Automatic IAM Authentication for all instances
--auto-ip Supports legacy behavior of v1 and will try to connect to first IP
address returned by the SQL Admin API. In most cases, this flag should not be used.
Prefer default of public IP or use --private-ip instead.
--compute-platform string Override the auto-detected compute_platform label of the built-in client
metrics. One of: gce, gke, cloud_run, app_engine, cloud_functions, ai_studio.
--config-file string Path to a TOML file containing configuration options.
-c, --credentials-file string Use service account key file as a source of IAM credentials.
--debug Enable pprof on the localhost admin server
--debug-logs Enable debug logging
--disable-metrics Disable Cloud Monitoring integration (used with --telemetry-project)
--disable-traces Disable Cloud Trace integration (used with --telemetry-project)
--enable-builtin-metrics Export the built-in client metrics (cloudsql.googleapis.com/client/connector/*)
to Cloud Monitoring in the Cloud SQL instance's project. The Proxy's identity
needs roles/monitoring.metricWriter on that project for the export to succeed;
without it, the Proxy logs one warning and keeps serving. Pass
--enable-builtin-metrics=false to turn the export off. (default true)
--exit-zero-on-sigterm Exit with 0 exit code when Sigterm received (default is 143)
--fuse string Mount a directory at the path using FUSE to access Cloud SQL instances.
--fuse-tmp-dir string Temp dir for Unix sockets created with FUSE (default "/tmp/csql-tmp")
Expand Down
6 changes: 5 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module github.com/GoogleCloudPlatform/cloud-sql-proxy/v2
go 1.26.0

require (
cloud.google.com/go/cloudsqlconn v1.25.4
cloud.google.com/go/cloudsqlconn v1.26.0
contrib.go.opencensus.io/exporter/prometheus v0.4.2
contrib.go.opencensus.io/exporter/stackdriver v0.13.15-0.20230702191903-2de6d2748484
github.com/coreos/go-systemd/v22 v22.7.0
Expand Down Expand Up @@ -32,6 +32,8 @@ require (
cloud.google.com/go/sql v0.3.0 // indirect
cloud.google.com/go/trace v1.17.0 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.60.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.62.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.47.4 // indirect
github.com/aws/aws-sdk-go-v2/config v1.33.10 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.20.10 // indirect
Expand Down Expand Up @@ -86,6 +88,8 @@ require (
go.opentelemetry.io/otel v1.47.0 // indirect
go.opentelemetry.io/otel/log v1.47.0 // indirect
go.opentelemetry.io/otel/metric v1.47.0 // indirect
go.opentelemetry.io/otel/sdk v1.47.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.47.0 // indirect
go.opentelemetry.io/otel/trace v1.47.0 // indirect
go.yaml.in/yaml/v2 v2.4.4 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
Expand Down
17 changes: 15 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKV
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go/auth v0.24.1 h1:NTmd2iJb4DsGIav0KiqFMYI88nMshA2wnFMBatRmoCs=
cloud.google.com/go/auth v0.24.1/go.mod h1:1dmBXa9xfBvBr39ZnDUDkaaA91ccq9CcqENDgJA4d7c=
cloud.google.com/go/auth/oauth2adapt v0.3.0 h1:FY8oSZpCYoUNv6QxVODuMjQz4IlSOVeiQtZ08vLPz88=
Expand All @@ -23,12 +24,16 @@ cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvf
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/cloudsqlconn v1.25.4 h1:RfAQJGUgL8YoKftr7n9uOqbuaw5lliA/fh18hZM2vfQ=
cloud.google.com/go/cloudsqlconn v1.25.4/go.mod h1:C9fx2u6VM8jnl1qdubGJ8BdbENVAj3Bu68ojxtx18x4=
cloud.google.com/go/cloudsqlconn v1.26.0 h1:PAhid+sI4qighL3hZreR+VN4zAXQDRgwaKnoZJQQvXk=
cloud.google.com/go/cloudsqlconn v1.26.0/go.mod h1:QB+On8iYX22wh1pfAzHAwGvRn1zbQJoG1C9zPbXl9WU=
cloud.google.com/go/compute/metadata v0.10.0 h1:pyKMUQSwchgkIBBJGdILqQbs/BNJXqwSA7Ej6LAvvtY=
cloud.google.com/go/compute/metadata v0.10.0/go.mod h1:rGFHRrIif570kSibjFTMbt6/4/tzgJWFGI/HVol4GIk=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/logging v1.20.0 h1:l7yzCB8/JSY5PHT9RsaqdT1jaBGbM6KlSJDX210HlXM=
cloud.google.com/go/logging v1.20.0/go.mod h1:ALUaJQZM691Eeujy9kFx0xREq3AMPuGd+pTRZuTYv+w=
cloud.google.com/go/longrunning v1.3.0 h1:lZ6c8SJJBmILp/Wnr3vGnBLa0q+cYfSHRCcUfVI2hpI=
cloud.google.com/go/longrunning v1.3.0/go.mod h1:wpgLxCn+UA3/TZ1bj/fW39mfDpwVlMPu1xKJyoMt7V0=
cloud.google.com/go/monitoring v1.31.0 h1:LCuAoYiq8snyL/eDAG/VC73uWFXeax8+YEfmoNGJUrY=
cloud.google.com/go/monitoring v1.31.0/go.mod h1:7uk2dIJIf7ra64dI1VuQ0ttoFqvtKjdzq46u6zNP1hU=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
Expand Down Expand Up @@ -65,6 +70,12 @@ github.com/AzureAD/microsoft-authentication-library-for-go v1.8.0 h1:Nljr4q1GRA/
github.com/AzureAD/microsoft-authentication-library-for-go v1.8.0/go.mod h1:Y33QHnf0FfdVewFFISOGe20mkZbxX4H839o955/PoeI=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.60.0 h1:HldzheTs05E3ybqSitI/wHaof6+XERRudgZLjYbs3eE=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.60.0/go.mod h1:evkqaSczW9g2BQm1veCtgNhJ4wCCsRrOsSgNIn9LHQk=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.60.0 h1:Fx8NtDCmKH4ML2hUkPz4Dq250903vRDojMjVCDKwQuc=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.60.0/go.mod h1:V9g30lTKzfUsEW+gpWssck6u9IhARajmipodImLLcwI=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.62.0 h1:aPMa5qZU9dp4fU34o+jQgVIPFIOi20oxYg8TLigmZLg=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.62.0/go.mod h1:sg6Wqbl0BKKZ5zKnQj2q0qic/3ziF894SCx1LpbX37Y=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
Expand Down Expand Up @@ -397,6 +408,8 @@ go.opentelemetry.io/otel/log v1.47.0 h1:cOTS1CcLbSQeZKanGJ+0JpF/+t4PELi3O3bbl2lq
go.opentelemetry.io/otel/log v1.47.0/go.mod h1:9byitSQ5pLC6PpqwGXjqdMKya6ZTswHRZh2vvXT33nw=
go.opentelemetry.io/otel/metric v1.47.0 h1:4PptaldXx3Eat1XjMZ68pPJEs5wrhlemctZE9a3UdWY=
go.opentelemetry.io/otel/metric v1.47.0/go.mod h1:ADGSXxRrXM6bjbvLo535EstVFlPpPYZm4LBKixjDHwU=
go.opentelemetry.io/otel/metric/x v0.69.0 h1:DjRLr15H83v+hCW7JA9NoJvOkYTtmq5YoDRbe9deYpM=
go.opentelemetry.io/otel/metric/x v0.69.0/go.mod h1:uVvsMPMFFyj/HUQfrUnH3JjnOQ1dwFDorgFLRBasM0k=
go.opentelemetry.io/otel/sdk v1.47.0 h1:zWXEr4j2lFefG87TU6Yg8a7ngfohIKFZHKp0Hf5hC6I=
go.opentelemetry.io/otel/sdk v1.47.0/go.mod h1:VUc24kiOeoGsxG8G9ULx3fWKvB7jMhnGE8Oi607lgR0=
go.opentelemetry.io/otel/sdk/metric v1.47.0 h1:lfISg2j93VT6yqdk9OfUaZmw/GfcZqCCV3jdXtsPnKw=
Expand Down
Loading
Loading