-
Notifications
You must be signed in to change notification settings - Fork 31
feat: Support configure connections using DNS #843
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 4 commits
4c2b080
8f8138d
a53c120
b6823f8
3f9ba9d
35526b7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -234,6 +234,62 @@ func connect() { | |
// ... etc | ||
} | ||
``` | ||
### Using DNS to identify an instance | ||
|
||
The connector can be configured to use DNS to look up an instance. This would | ||
allow you to configure your application to connect to a database instance, and | ||
centrally configure which instance in your DNS zone. | ||
|
||
#### Configure your DNS Records | ||
|
||
Add a DNS TXT record for the Cloud SQL instance to a **private** DNS server | ||
or a private Google Cloud DNS Zone used by your application. | ||
|
||
**Note:** You are strongly discouraged from adding DNS records for your | ||
Cloud SQL instances to a public DNS server. This would allow anyone on the | ||
internet to discover the Cloud SQL instance name. | ||
|
||
For example: suppose you wanted to use the domain name | ||
`prod-db.mycompany.example.com` to connect to your database instance | ||
`my-project:region:my-instance`. You would create the following DNS record: | ||
|
||
- Record type: `TXT` | ||
- Name: `prod-db.mycompany.example.com` – This is the domain name used by the application | ||
- Value: `my-project:region:my-instance` – This is the instance name | ||
|
||
#### Configure the connector | ||
|
||
Configure the connector as described above, replacing the conenctor ID with | ||
the DNS name. | ||
|
||
Adapting the MySQL + database/sql example above: | ||
|
||
```go | ||
import ( | ||
"database/sql" | ||
|
||
"cloud.google.com/go/cloudsqlconn" | ||
"cloud.google.com/go/cloudsqlconn/mysql/mysql" | ||
) | ||
|
||
func connect() { | ||
cleanup, err := mysql.RegisterDriver("cloudsql-mysql", | ||
cloudsqlconn.WithDnsResolver(), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. WithDNSResolver() Also, let's do the equivalent of gofmt on this sample code. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed |
||
cloudsqlconn.WithCredentialsFile("key.json")) | ||
if err != nil { | ||
// ... handle error | ||
} | ||
// call cleanup when you're done with the database connection | ||
defer cleanup() | ||
|
||
db, err := sql.Open( | ||
"cloudsql-mysql", | ||
"myuser:mypass@cloudsql-mysql(prod-db.mycompany.example.com)/mydb", | ||
) | ||
// ... etc | ||
} | ||
``` | ||
|
||
|
||
### Using Options | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1016,3 +1016,64 @@ func TestDialerInitializesLazyCache(t *testing.T) { | |
t.Fatalf("dialer was initialized with non-lazy type: %T", tt) | ||
} | ||
} | ||
|
||
type fakeResolver struct{} | ||
|
||
func (r *fakeResolver) Resolve(_ context.Context, name string) (instance.ConnName, error) { | ||
// For TestDialerSuccessfullyDialsDnsTxtRecord | ||
if name == "db.example.com" { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Instead of relying on hardcoded values for particular tests, you could set fields in type fakeResolve struct {
instances map[string]struct{
connName instance.ConnName
err error
}
} And then this implementation becomes: func (r *fakeResolver) Resolve(_ context.Context, name string) (instance.ConnName, error) {
icn, ok := r.instances[name]
if !ok {
return instance.ConnName{}, errors.New("not found")
}
return icn, nil
} There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems excessive. After simplifying obsolete test cases, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it helps test readability. Otherwise, a person has to go searching for what's special about If we're not supporting multiple instances anymore this also works: WithResolver(&fakeResolver{validName: "db.example.com"}), type fakeResolve struct {
validName string
}
func (r *fakeResolver) Resolve(_ context.Context, name string) (instance.ConnName, error) {
if name != r.validName {
// return error
}
// return connection name
} The point is to make the fake extensible -- not just for this use, but for future use. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually this is what you do below in the resolver test -- let's do the same thing here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
return instance.ParseConnName("my-project:my-region:my-instance") | ||
} | ||
if name == "db2.example.com" { | ||
return instance.ParseConnName("my-project:my-region:my-instance") | ||
} | ||
// TestDialerFailsDnsTxtRecordMissing | ||
return instance.ConnName{}, fmt.Errorf("no resolution for %q", name) | ||
} | ||
|
||
func TestDialerSuccessfullyDialsDnsTxtRecord(t *testing.T) { | ||
inst := mock.NewFakeCSQLInstance( | ||
"my-project", "my-region", "my-instance", | ||
) | ||
d := setupDialer(t, setupConfig{ | ||
testInstance: inst, | ||
reqs: []*mock.Request{ | ||
mock.InstanceGetSuccess(inst, 1), | ||
mock.CreateEphemeralSuccess(inst, 1), | ||
}, | ||
dialerOptions: []Option{ | ||
WithTokenSource(mock.EmptyTokenSource{}), | ||
WithResolver(&fakeResolver{}), | ||
}, | ||
}) | ||
|
||
// Target has a trailing '.' | ||
enocom marked this conversation as resolved.
Show resolved
Hide resolved
|
||
testSuccessfulDial( | ||
context.Background(), t, d, | ||
"db.example.com", | ||
) | ||
// Target does not have a trailing '.' | ||
testSuccessfulDial( | ||
context.Background(), t, d, | ||
"db2.example.com", | ||
) | ||
} | ||
|
||
func TestDialerFailsDnsTxtRecordMissing(t *testing.T) { | ||
inst := mock.NewFakeCSQLInstance( | ||
"my-project", "my-region", "my-instance", | ||
) | ||
d := setupDialer(t, setupConfig{ | ||
testInstance: inst, | ||
reqs: []*mock.Request{}, | ||
dialerOptions: []Option{ | ||
WithTokenSource(mock.EmptyTokenSource{}), | ||
WithResolver(&fakeResolver{}), | ||
}, | ||
}) | ||
_, err := d.Dial(context.Background(), "doesnt-exist.example.com") | ||
wantMsg := "no resolution for \"doesnt-exist.example.com\"" | ||
if !strings.Contains(err.Error(), wantMsg) { | ||
t.Fatalf("want = %v, got = %v", wantMsg, err) | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,123 @@ | ||
// Copyright 2024 Google LLC | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// https://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package cloudsql | ||
|
||
import ( | ||
"context" | ||
"fmt" | ||
"net" | ||
"sort" | ||
|
||
"cloud.google.com/go/cloudsqlconn/instance" | ||
) | ||
|
||
// DNSResolver uses the default net.Resolver to find | ||
// TXT records containing an instance name for a DNS record. | ||
var DNSResolver = &DNSInstanceConnectionNameResolver{ | ||
enocom marked this conversation as resolved.
Show resolved
Hide resolved
|
||
dnsResolver: net.DefaultResolver, | ||
} | ||
|
||
// DefaultResolver simply parses instance names. | ||
var DefaultResolver = &ConnNameResolver{} | ||
|
||
// ConnNameResolver simply parses instance names. Implements | ||
// InstanceConnectionNameResolver | ||
type ConnNameResolver struct { | ||
} | ||
|
||
// Resolve returns the instance name, possibly using DNS. This will return an | ||
// instance.ConnName or an error if it was unable to resolve an instance name. | ||
func (r *ConnNameResolver) Resolve(_ context.Context, icn string) (instanceName instance.ConnName, err error) { | ||
return instance.ParseConnName(icn) | ||
} | ||
|
||
// netResolver groups the methods on net.Resolver that are used by the DNS | ||
// resolver implementation. This allows an application to replace the default | ||
// net.DefaultResolver with a custom implementation. For example: the | ||
// application may need to connect to a specific DNS server using a specially | ||
// configured instance of net.Resolver. | ||
type netResolver interface { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could we remove the duplicate interface and test this e2e? Or alternatively, what would it take to assemble a mock DNS server in unit tests? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we need a mock for practicality. We need to write test cases covering our code when the DNS records are misconfigured. Also, we need to do this across all our supported connector languages. Mocking the DNS response is the most reliably way to implement these test cases. |
||
LookupTXT(ctx context.Context, name string) ([]string, error) | ||
} | ||
|
||
// DNSInstanceConnectionNameResolver can resolve domain names into instance names using | ||
// TXT records in DNS. Implements InstanceConnectionNameResolver | ||
type DNSInstanceConnectionNameResolver struct { | ||
dnsResolver netResolver | ||
} | ||
|
||
// Resolve returns the instance name, possibly using DNS. This will return an | ||
// instance.ConnName or an error if it was unable to resolve an instance name. | ||
func (r *DNSInstanceConnectionNameResolver) Resolve(ctx context.Context, icn string) (instanceName instance.ConnName, err error) { | ||
cn, err := instance.ParseConnName(icn) | ||
if err != nil { | ||
// The connection name was not project:region:instance | ||
// Attempt to query a TXT record and see if it works instead. | ||
cn, err = r.queryDNS(ctx, icn) | ||
if err != nil { | ||
return instance.ConnName{}, err | ||
} | ||
} | ||
|
||
return cn, nil | ||
} | ||
|
||
// queryDNS attempts to resolve a TXT record for the domain name. | ||
// The DNS TXT record's target field is used as instance name. | ||
// | ||
// This handles several conditions where the DNS records may be missing or | ||
// invalid: | ||
// - The domain name resolves to 0 DNS records - return an error | ||
// - Some DNS records to not contain a well-formed instance name - return the | ||
// first well-formed instance name. If none found return an error. | ||
// - The domain name resolves to 2 or more DNS record - return first valid | ||
// record when sorted by priority: lowest value first, then by target: | ||
// alphabetically. | ||
func (r *DNSInstanceConnectionNameResolver) queryDNS(ctx context.Context, domainName string) (instance.ConnName, error) { | ||
// Attempt to query the TXT records. | ||
// This could return a partial error where both err != nil && len(records) > 0. | ||
records, err := r.dnsResolver.LookupTXT(ctx, domainName) | ||
// If resolve failed and no records were found, return the error. | ||
if err != nil { | ||
return instance.ConnName{}, fmt.Errorf("unable to resolve TXT record for %q: %v", domainName, err) | ||
} | ||
|
||
// Process the records returning the first valid TXT record. | ||
|
||
// Sort the TXT record values alphabetically by instance name | ||
sort.Slice(records, func(i, j int) bool { | ||
return records[i] < records[j] | ||
}) | ||
|
||
var perr error | ||
// Attempt to parse records, returning the first valid record. | ||
for _, record := range records { | ||
// Parse the target as a CN | ||
cn, parseErr := instance.ParseConnName(record) | ||
if parseErr != nil { | ||
perr = fmt.Errorf("unable to parse TXT for %q -> %q : %v", domainName, record, parseErr) | ||
continue | ||
} | ||
return cn, nil | ||
} | ||
|
||
// If all the records failed to parse, return one of the parse errors | ||
if perr != nil { | ||
return instance.ConnName{}, perr | ||
} | ||
|
||
// No records were found, return an error. | ||
return instance.ConnName{}, fmt.Errorf("no valid TXT records found for %q", domainName) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"For example, suppose ... to your database instance
my-project:region:my-instance
, you would create the following DNS record:There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated. Switching to TXT records.