The bigtable package

import "cloud.google.com/go/bigtable"

Introduction

Package bigtable is an API to Google Cloud Bigtable.

See https://cloud.google.com/bigtable/docs/ for general product documentation.

Setup and Credentials

Use NewClient or NewAdminClient to create a client that can be used to access the data or admin APIs respectively. Both require credentials that have permission to access the Cloud Bigtable API.

If your program is run on Google App Engine or Google Compute Engine, using the Application Default Credentials (https://developers.google.com/accounts/docs/application-default-credentials) is the simplest option. Those credentials will be used by default when NewClient or NewAdminClient are called.

To use alternate credentials, pass them to NewClient or NewAdminClient using option.WithTokenSource. For instance, you can use service account credentials by visiting https://cloud.google.com/console/project/MYPROJECT/apiui/credential, creating a new OAuth "Client ID", storing the JSON key somewhere accessible, and writing

jsonKey, err := ioutil.ReadFile(pathToKeyFile)
...
config, err := google.JWTConfigFromJSON(jsonKey, bigtable.Scope) // or bigtable.AdminScope, etc.
...
client, err := bigtable.NewClient(ctx, project, instance, option.WithTokenSource(config.TokenSource(ctx)))
...

Here, `google` means the golang.org/x/oauth2/google package and `option` means the google.golang.org/api/option package.

Reading

The principal way to read from a Bigtable is to use the ReadRows method on *Table. A RowRange specifies a contiguous portion of a table. A Filter may be provided through RowFilter to limit or transform the data that is returned.

tbl := client.Open("mytable")
...
// Read all the rows starting with "com.google.",
// but only fetch the columns in the "links" family.
rr := bigtable.PrefixRange("com.google.")
err := tbl.ReadRows(ctx, rr, func(r Row) bool {
    // do something with r
    return true // keep going
}, bigtable.RowFilter(bigtable.FamilyFilter("links")))
...

To read a single row, use the ReadRow helper method.

r, err := tbl.ReadRow(ctx, "com.google.cloud") // "com.google.cloud" is the entire row key
...

Writing

This API exposes two distinct forms of writing to a Bigtable: a Mutation and a ReadModifyWrite. The former expresses idempotent operations. The latter expresses non-idempotent operations and returns the new values of updated cells. These operations are performed by creating a Mutation or ReadModifyWrite (with NewMutation or NewReadModifyWrite), building up one or more operations on that, and then using the Apply or ApplyReadModifyWrite methods on a Table.

For instance, to set a couple of cells in a table,

tbl := client.Open("mytable")
mut := bigtable.NewMutation()
mut.Set("links", "maps.google.com", bigtable.Now(), []byte("1"))
mut.Set("links", "golang.org", bigtable.Now(), []byte("1"))
err := tbl.Apply(ctx, "com.google.cloud", mut)
...

To increment an encoded value in one cell,

tbl := client.Open("mytable")
rmw := bigtable.NewReadModifyWrite()
rmw.Increment("links", "golang.org", 12) // add 12 to the cell in column "links:golang.org"
r, err := tbl.ApplyReadModifyWrite(ctx, "com.google.cloud", rmw)
...

Retries

If a read or write operation encounters a transient error it will be retried until a successful response, an unretryable error or the context deadline is reached. Non-idempotent writes (where the timestamp is set to ServerTime) will not be retried. In the case of ReadRows, retried calls will not re-scan rows that have already been processed.

Authentication

See examples of authorization and authentication at https://godoc.org/cloud.google.com/go#pkg-examples.

Index

Constants
func GCRuleToString(rule *bttdpb.GcRule) string
type AdminClient
    func NewAdminClient(ctx context.Context, project, instance string, opts ...option.ClientOption) (*AdminClient, error)
    func (ac *AdminClient) Close() error
    func (ac *AdminClient) CreateColumnFamily(ctx context.Context, table, family string) error
    func (ac *AdminClient) CreatePresplitTable(ctx context.Context, table string, split_keys []string) error
    func (ac *AdminClient) CreateTable(ctx context.Context, table string) error
    func (ac *AdminClient) DeleteColumnFamily(ctx context.Context, table, family string) error
    func (ac *AdminClient) DeleteTable(ctx context.Context, table string) error
    func (ac *AdminClient) DropRowRange(ctx context.Context, table, rowKeyPrefix string) error
    func (ac *AdminClient) SetGCPolicy(ctx context.Context, table, family string, policy GCPolicy) error
    func (ac *AdminClient) TableInfo(ctx context.Context, table string) (*TableInfo, error)
    func (ac *AdminClient) Tables(ctx context.Context) ([]string, error)
type ApplyOption
    func GetCondMutationResult(matched *bool) ApplyOption
type Client
    func NewClient(ctx context.Context, project, instance string, opts ...option.ClientOption) (*Client, error)
    func (c *Client) Close() error
    func (c *Client) Open(table string) *Table
type FamilyInfo
type Filter
    func ChainFilters(sub ...Filter) Filter
    func ColumnFilter(pattern string) Filter
    func ColumnRangeFilter(family, start, end string) Filter
    func ConditionFilter(predicateFilter, trueFilter, falseFilter Filter) Filter
    func FamilyFilter(pattern string) Filter
    func InterleaveFilters(sub ...Filter) Filter
    func LatestNFilter(n int) Filter
    func RowKeyFilter(pattern string) Filter
    func StripValueFilter() Filter
    func TimestampRangeFilter(startTime time.Time, endTime time.Time) Filter
    func TimestampRangeFilterMicros(startTime Timestamp, endTime Timestamp) Filter
    func ValueFilter(pattern string) Filter
    func ValueRangeFilter(start, end []byte) Filter
type GCPolicy
    func IntersectionPolicy(sub ...GCPolicy) GCPolicy
    func MaxAgePolicy(d time.Duration) GCPolicy
    func MaxVersionsPolicy(n int) GCPolicy
    func UnionPolicy(sub ...GCPolicy) GCPolicy
type InstanceAdminClient
    func NewInstanceAdminClient(ctx context.Context, project string, opts ...option.ClientOption) (*InstanceAdminClient, error)
    func (iac *InstanceAdminClient) Close() error
    func (iac *InstanceAdminClient) CreateInstance(ctx context.Context, conf *InstanceConf) error
    func (iac *InstanceAdminClient) DeleteInstance(ctx context.Context, instanceId string) error
    func (iac *InstanceAdminClient) Instances(ctx context.Context) ([]*InstanceInfo, error)
type InstanceConf
type InstanceInfo
type Mutation
    func NewCondMutation(cond Filter, mtrue, mfalse *Mutation) *Mutation
    func NewMutation() *Mutation
    func (m *Mutation) DeleteCellsInColumn(family, column string)
    func (m *Mutation) DeleteCellsInFamily(family string)
    func (m *Mutation) DeleteRow()
    func (m *Mutation) DeleteTimestampRange(family, column string, start, end Timestamp)
    func (m *Mutation) Set(family, column string, ts Timestamp, value []byte)
type ReadItem
type ReadModifyWrite
    func NewReadModifyWrite() *ReadModifyWrite
    func (m *ReadModifyWrite) AppendValue(family, column string, v []byte)
    func (m *ReadModifyWrite) Increment(family, column string, delta int64)
type ReadOption
    func LimitRows(limit int64) ReadOption
    func RowFilter(f Filter) ReadOption
type Row
    func (r Row) Key() string
type RowList
type RowRange
    func InfiniteRange(start string) RowRange
    func NewRange(begin, end string) RowRange
    func PrefixRange(prefix string) RowRange
    func (r RowRange) Contains(row string) bool
    func (r RowRange) String() string
    func (r RowRange) Unbounded() bool
type RowRangeList
type RowSet
    func SingleRow(row string) RowSet
type StorageType
type Table
    func (t *Table) Apply(ctx context.Context, row string, m *Mutation, opts ...ApplyOption) error
    func (t *Table) ApplyBulk(ctx context.Context, rowKeys []string, muts []*Mutation, opts ...ApplyOption) ([]error, error)
    func (t *Table) ApplyReadModifyWrite(ctx context.Context, row string, m *ReadModifyWrite) (Row, error)
    func (t *Table) ReadRow(ctx context.Context, row string, opts ...ReadOption) (Row, error)
    func (t *Table) ReadRows(ctx context.Context, arg RowSet, f func(Row) bool, opts ...ReadOption) error
type TableInfo
type Timestamp
    func Now() Timestamp
    func Time(t time.Time) Timestamp
    func (ts Timestamp) Time() time.Time
    func (ts Timestamp) TruncateToMilliseconds() Timestamp

Constants

const (
    // Scope is the OAuth scope for Cloud Bigtable data operations.
    Scope = "https://www.googleapis.com/auth/bigtable.data"
    // ReadonlyScope is the OAuth scope for Cloud Bigtable read-only data operations.
    ReadonlyScope = "https://www.googleapis.com/auth/bigtable.readonly"

    // AdminScope is the OAuth scope for Cloud Bigtable table admin operations.
    AdminScope = "https://www.googleapis.com/auth/bigtable.admin.table"

    // InstanceAdminScope is the OAuth scope for Cloud Bigtable instance (and cluster) admin operations.
    InstanceAdminScope = "https://www.googleapis.com/auth/bigtable.admin.cluster"
)

Scope constants for authentication credentials. These should be used when using credential creation functions such as oauth.NewServiceAccountFromFile.

func GCRuleToString

func GCRuleToString(rule *bttdpb.GcRule) string

GCRuleToString converts the given GcRule proto to a user-visible string.

type AdminClient

type AdminClient struct {
    // contains filtered or unexported fields
}

AdminClient is a client type for performing admin operations within a specific instance.

func NewAdminClient

func NewAdminClient(ctx context.Context, project, instance string, opts ...option.ClientOption) (*AdminClient, error)

NewAdminClient creates a new AdminClient for a given project and instance.

func (*AdminClient) Close

func (ac *AdminClient) Close() error

Close closes the AdminClient.

func (*AdminClient) CreateColumnFamily

func (ac *AdminClient) CreateColumnFamily(ctx context.Context, table, family string) error

CreateColumnFamily creates a new column family in a table.

func (*AdminClient) CreatePresplitTable

func (ac *AdminClient) CreatePresplitTable(ctx context.Context, table string, split_keys []string) error

CreatePresplitTable creates a new table in the instance. The list of row keys will be used to initially split the table into multiple tablets. Given two split keys, "s1" and "s2", three tablets will be created, spanning the key ranges: [, s1), [s1, s2), [s2, ). This method may return before the table's creation is complete.

func (*AdminClient) CreateTable

func (ac *AdminClient) CreateTable(ctx context.Context, table string) error

CreateTable creates a new table in the instance. This method may return before the table's creation is complete.

func (*AdminClient) DeleteColumnFamily

func (ac *AdminClient) DeleteColumnFamily(ctx context.Context, table, family string) error

DeleteColumnFamily deletes a column family in a table and all of its data.

func (*AdminClient) DeleteTable

func (ac *AdminClient) DeleteTable(ctx context.Context, table string) error

DeleteTable deletes a table and all of its data.

func (*AdminClient) DropRowRange

func (ac *AdminClient) DropRowRange(ctx context.Context, table, rowKeyPrefix string) error

DropRowRange permanently deletes a row range from the specified table.

func (*AdminClient) SetGCPolicy

func (ac *AdminClient) SetGCPolicy(ctx context.Context, table, family string, policy GCPolicy) error

SetGCPolicy specifies which cells in a column family should be garbage collected. GC executes opportunistically in the background; table reads may return data matching the GC policy.

func (*AdminClient) TableInfo

func (ac *AdminClient) TableInfo(ctx context.Context, table string) (*TableInfo, error)

TableInfo retrieves information about a table.

func (*AdminClient) Tables

func (ac *AdminClient) Tables(ctx context.Context) ([]string, error)

Tables returns a list of the tables in the instance.

type ApplyOption

type ApplyOption interface {
    // contains filtered or unexported methods
}

An ApplyOption is an optional argument to Apply.

func GetCondMutationResult

func GetCondMutationResult(matched *bool) ApplyOption

GetCondMutationResult returns an ApplyOption that reports whether the conditional mutation's condition matched.

type Client

type Client struct {
    // contains filtered or unexported fields
}

Client is a client for reading and writing data to tables in an instance.

A Client is safe to use concurrently, except for its Close method.

func NewClient

func NewClient(ctx context.Context, project, instance string, opts ...option.ClientOption) (*Client, error)

NewClient creates a new Client for a given project and instance.

func (*Client) Close

func (c *Client) Close() error

Close closes the Client.

func (*Client) Open

func (c *Client) Open(table string) *Table

Open opens a table.

type FamilyInfo

type