Viele Echtzeitapps haben Dokumente, die als Zähler fungieren. Sie können z. B. "Gefällt mir"-Angaben für einen Beitrag zählen oder wie oft ein bestimmter Artikel als "Favorit" markiert wurde.
In Firestore kann jedes einzelne Dokument nur einmal pro Sekunde aktualisiert werden, was für einige Anwendungen mit hohen Zugriffszahlen ein zu niedriger Wert sein kann.
Lösung: Verteilte Zähler
Wenn Sie möchten, dass häufigere Zähleraktualisierungen unterstützt werden, können Sie einen verteilten Zähler erstellen. Jeder Zähler ist ein Dokument mit einer untergeordneten Sammlung von "Shards". Der Wert des Zählers ist die Summe der Werte der Shards.
Der Schreibdurchsatz nimmt linear mit der Anzahl der Shards zu, sodass ein verteilter Zähler mit 10 Shards 10-mal so viele Schreibvorgänge verarbeiten kann wie ein herkömmlicher Zähler.
Web
// counters/${ID}
{
"num_shards": NUM_SHARDS,
"shards": [subcollection]
}
// counters/${ID}/shards/${NUM}
{
"count": 123
}
Swift
// counters/${ID}
struct Counter {
let numShards: Int
init(numShards: Int) {
self.numShards = numShards
}
}
// counters/${ID}/shards/${NUM}
struct Shard {
let count: Int
init(count: Int) {
self.count = count
}
}
Objective-C
// counters/${ID}
@interface FIRCounter : NSObject
@property (nonatomic, readonly) NSInteger shardCount;
@end
@implementation FIRCounter
- (instancetype)initWithShardCount:(NSInteger)shardCount {
self = [super init];
if (self != nil) {
_shardCount = shardCount;
}
return self;
}
@end
// counters/${ID}/shards/${NUM}
@interface FIRShard : NSObject
@property (nonatomic, readonly) NSInteger count;
@end
@implementation FIRShard
- (instancetype)initWithCount:(NSInteger)count {
self = [super init];
if (self != nil) {
_count = count;
}
return self;
}
@end
Java
Android
// counters/${ID} public class Counter { int numShards; public Counter(int numShards) { this.numShards = numShards; } } // counters/${ID}/shards/${NUM} public class Shard { int count; public Shard(int count) { this.count = count; } }
Kotlin+KTX
Android
// counters/${ID} data class Counter(var numShards: Int) // counters/${ID}/shards/${NUM} data class Shard(var count: Int)
Python
import random from google.cloud import firestore class Shard(object): """ A shard is a distributed counter. Each shard can support being incremented once per second. Multiple shards are needed within a Counter to allow more frequent incrementing. """ def __init__(self): self._count = 0 def to_dict(self): return {"count": self._count} class Counter(object): """ A counter stores a collection of shards which are summed to return a total count. This allows for more frequent incrementing than a single document. """ def __init__(self, num_shards): self._num_shards = num_shards
Node.js
Nicht zutreffend, siehe Snippet zur Zählererhöhung unten.
Go
import ( "context" "fmt" "math/rand" "strconv" "cloud.google.com/go/firestore" "google.golang.org/api/iterator" ) // Counter is a collection of documents (shards) // to realize counter with high frequency. type Counter struct { numShards int } // Shard is a single counter, which is used in a group // of other shards within Counter. type Shard struct { Count int }
PHP
Nicht zutreffend, siehe Snippet zur Zählerinitialisierung unten.
C#
/// <summary> /// Shard is a document that contains the count. /// </summary> [FirestoreData] public class Shard { [FirestoreProperty(name: "count")] public int Count { get; set; } }
Ruby
import random from google.cloud import firestore class Shard(object): """ A shard is a distributed counter. Each shard can support being incremented once per second. Multiple shards are needed within a Counter to allow more frequent incrementing. """ def __init__(self): self._count = 0 def to_dict(self): return {"count": self._count} class Counter(object): """ A counter stores a collection of shards which are summed to return a total count. This allows for more frequent incrementing than a single document. """ def __init__(self, num_shards): self._num_shards = num_shards
Mit dem folgenden Code wird ein verteilter Zähler initialisiert:
Web
function createCounter(ref, num_shards) {
var batch = db.batch();
// Initialize the counter document
batch.set(ref, { num_shards: num_shards });
// Initialize each shard with count=0
for (let i = 0; i < num_shards; i++) {
const shardRef = ref.collection('shards').doc(i.toString());
batch.set(shardRef, { count: 0 });
}
// Commit the write batch
return batch.commit();
}
Swift
func createCounter(ref: DocumentReference, numShards: Int) {
ref.setData(["numShards": numShards]){ (err) in
for i in 0...numShards {
ref.collection("shards").document(String(i)).setData(["count": 0])
}
}
}
Objective-C
- (void)createCounterAtReference:(FIRDocumentReference *)reference
shardCount:(NSInteger)shardCount {
[reference setData:@{ @"numShards": @(shardCount) } completion:^(NSError * _Nullable error) {
for (NSInteger i = 0; i < shardCount; i++) {
NSString *shardName = [NSString stringWithFormat:@"%ld", (long)shardCount];
[[[reference collectionWithPath:@"shards"] documentWithPath:shardName]
setData:@{ @"count": @(0) }];
}
}];
}
Java
Android
public Task<Void> createCounter(final DocumentReference ref, final int numShards) { // Initialize the counter document, then initialize each shard. return ref.set(new Counter(numShards)) .continueWithTask(new Continuation<Void, Task<Void>>() { @Override public Task<Void> then(@NonNull Task<Void> task) throws Exception { if (!task.isSuccessful()) { throw task.getException(); } List<Task<Void>> tasks = new ArrayList<>(); // Initialize each shard with count=0 for (int i = 0; i < numShards; i++) { Task<Void> makeShard = ref.collection("shards") .document(String.valueOf(i)) .set(new Shard(0)); tasks.add(makeShard); } return Tasks.whenAll(tasks); } }); }
Kotlin+KTX
Android
fun createCounter(ref: DocumentReference, numShards: Int): Task<Void> { // Initialize the counter document, then initialize each shard. return ref.set(Counter(numShards)) .continueWithTask { task -> if (!task.isSuccessful) { throw task.exception!! } val tasks = arrayListOf<Task<Void>>() // Initialize each shard with count=0 for (i in 0 until numShards) { val makeShard = ref.collection("shards") .document(i.toString()) .set(Shard(0)) tasks.add(makeShard) } Tasks.whenAll(tasks) } }
Python
def init_counter(self, doc_ref): """ Create a given number of shards as subcollection of specified document. """ col_ref = doc_ref.collection("shards") # Initialize each shard with count=0 for num in range(self._num_shards): shard = Shard() col_ref.document(str(num)).set(shard.to_dict())
Node.js
Nicht zutreffend, siehe Snippet zur Zählererhöhung unten.
Go
// initCounter creates a given number of shards as // subcollection of specified document. func (c *Counter) initCounter(ctx context.Context, docRef *firestore.DocumentRef) error { colRef := docRef.Collection("shards") // Initialize each shard with count=0 for num := 0; num < c.numShards; num++ { shard := Shard{0} if _, err := colRef.Doc(strconv.Itoa(num)).Set(ctx, shard); err != nil { return fmt.Errorf("Set: %v", err) } } return nil }
PHP
$numShards = 10; $colRef = $ref->collection('SHARDS'); for ($i = 0; $i < $numShards; $i++) { $doc = $colRef->document($i); $doc->set(['Cnt' => 0]); }
C#
/// <summary> /// Create a given number of shards as a /// subcollection of specified document. /// </summary> /// <param name="docRef">The document reference <see cref="DocumentReference"/></param> private static async Task CreateCounterAsync(DocumentReference docRef, int numOfShards) { CollectionReference colRef = docRef.Collection("shards"); var tasks = new List<Task>(); // Initialize each shard with Count=0 for (var i = 0; i < numOfShards; i++) { tasks.Add(colRef.Document(i.ToString()).SetAsync(new Shard() { Count = 0 })); } await Task.WhenAll(tasks); }
Ruby
# project_id = "Your Google Cloud Project ID" # num_shards = "Number of shards for distributed counter" # collection_path = "shards" require "google/cloud/firestore" firestore = Google::Cloud::Firestore.new project_id: project_id shards_ref = firestore.col collection_path # Initialize each shard with count=0 num_shards.times do |i| shards_ref.doc(i).set(count: 0) end puts "Distributed counter shards collection created."
Wenn Sie den Zähler erhöhen möchten, wählen Sie einen zufälligen Shard aus und erhöhen Sie dessen Zählerwert:
Web
function incrementCounter(db, ref, num_shards) {
// Select a shard of the counter at random
const shard_id = Math.floor(Math.random() * num_shards).toString();
const shard_ref = ref.collection('shards').doc(shard_id);
// Update count
return shard_ref.update("count", firebase.firestore.FieldValue.increment(1));
}
Swift
func incrementCounter(ref: DocumentReference, numShards: Int) {
// Select a shard of the counter at random
let shardId = Int(arc4random_uniform(UInt32(numShards)))
let shardRef = ref.collection("shards").document(String(shardId))
shardRef.updateData([
"count": FieldValue.increment(Int64(1))
])
}
Objective-C
- (void)incrementCounterAtReference:(FIRDocumentReference *)reference
shardCount:(NSInteger)shardCount {
// Select a shard of the counter at random
NSInteger shardID = (NSInteger)arc4random_uniform((uint32_t)shardCount);
NSString *shardName = [NSString stringWithFormat:@"%ld", (long)shardID];
FIRDocumentReference *shardReference =
[[reference collectionWithPath:@"shards"] documentWithPath:shardName];
[shardReference updateData:@{
@"count": [FIRFieldValue fieldValueForIntegerIncrement:1]
}];
}
Java
Android
public Task<Void> incrementCounter(final DocumentReference ref, final int numShards) { int shardId = (int) Math.floor(Math.random() * numShards); DocumentReference shardRef = ref.collection("shards").document(String.valueOf(shardId)); return shardRef.update("count", FieldValue.increment(1)); }
Kotlin+KTX
Android
fun incrementCounter(ref: DocumentReference, numShards: Int): Task<Void> { val shardId = Math.floor(Math.random() * numShards).toInt() val shardRef = ref.collection("shards").document(shardId.toString()) return shardRef.update("count", FieldValue.increment(1)) }
Python
def increment_counter(self, doc_ref): """Increment a randomly picked shard.""" doc_id = random.randint(0, self._num_shards - 1) shard_ref = doc_ref.collection("shards").document(str(doc_id)) return shard_ref.update({"count": firestore.Increment(1)})
Node.js
function incrementCounter(docRef, numShards) { const shardId = Math.floor(Math.random() * numShards); const shardRef = docRef.collection('shards').doc(shardId.toString()); return shardRef.set({count: FieldValue.increment(1)}, {merge: true}); }
Go
// incrementCounter increments a randomly picked shard. func (c *Counter) incrementCounter(ctx context.Context, docRef *firestore.DocumentRef) (*firestore.WriteResult, error) { docID := strconv.Itoa(rand.Intn(c.numShards)) shardRef := docRef.Collection("shards").Doc(docID) return shardRef.Update(ctx, []firestore.Update{ {Path: "Count", Value: firestore.Increment(1)}, }) }
PHP
$colRef = $ref->collection('SHARDS'); $numShards = 0; $docCollection = $colRef->documents(); foreach ($docCollection as $doc) { $numShards++; } $shardIdx = random_int(0, $numShards-1); $doc = $colRef->document($shardIdx); $doc->update([ ['path' => 'Cnt', 'value' => FieldValue::increment(1)] ]);
C#
/// <summary> /// Increment a randomly picked shard by 1. /// </summary> /// <param name="docRef">The document reference <see cref="DocumentReference"/></param> /// <returns>The <see cref="Task"/></returns> private static async Task IncrementCounterAsync(DocumentReference docRef, int numOfShards) { int documentId; lock (s_randLock) { documentId = s_rand.Next(numOfShards); } var shardRef = docRef.Collection("shards").Document(documentId.ToString()); await shardRef.UpdateAsync("count", FieldValue.Increment(1)); }
Ruby
# project_id = "Your Google Cloud Project ID" # num_shards = "Number of shards for distributed counter" # collection_path = "shards" require "google/cloud/firestore" firestore = Google::Cloud::Firestore.new project_id: project_id # Select a shard of the counter at random shard_id = rand 0...num_shards shard_ref = firestore.doc "#{collection_path}/#{shard_id}" # increment counter shard_ref.update count: firestore.field_increment(1) puts "Counter incremented."
Wenn Sie den Gesamtzählerwert ermitteln möchten, führen Sie eine Abfrage für alle Shards durch und addieren Sie die count
-Felder:
Web
function getCount(ref) {
// Sum the count of each shard in the subcollection
return ref.collection('shards').get().then((snapshot) => {
let total_count = 0;
snapshot.forEach((doc) => {
total_count += doc.data().count;
});
return total_count;
});
}
Swift
func getCount(ref: DocumentReference) {
ref.collection("shards").getDocuments() { (querySnapshot, err) in
var totalCount = 0
if err != nil {
// Error getting shards
// ...
} else {
for document in querySnapshot!.documents {
let count = document.data()["count"] as! Int
totalCount += count
}
}
print("Total count is \(totalCount)")
}
}
Objective-C
- (void)getCountWithReference:(FIRDocumentReference *)reference {
[[reference collectionWithPath:@"shards"]
getDocumentsWithCompletion:^(FIRQuerySnapshot *snapshot,
NSError *error) {
NSInteger totalCount = 0;
if (error != nil) {
// Error getting shards
// ...
} else {
for (FIRDocumentSnapshot *document in snapshot.documents) {
NSInteger count = [document[@"count"] integerValue];
totalCount += count;
}
NSLog(@"Total count is %ld", (long)totalCount);
}
}];
}
Java
Android
public Task<Integer> getCount(final DocumentReference ref) { // Sum the count of each shard in the subcollection return ref.collection("shards").get() .continueWith(new Continuation<QuerySnapshot, Integer>() { @Override public Integer then(@NonNull Task<QuerySnapshot> task) throws Exception { int count = 0; for (DocumentSnapshot snap : task.getResult()) { Shard shard = snap.toObject(Shard.class); count += shard.count; } return count; } }); }
Kotlin+KTX
Android
fun getCount(ref: DocumentReference): Task<Int> { // Sum the count of each shard in the subcollection return ref.collection("shards").get() .continueWith { task -> var count = 0 for (snap in task.result!!) { val shard = snap.toObject<Shard>() count += shard.count } count } }
Python
def get_count(self, doc_ref): """Return a total count across all shards.""" total = 0 shards = doc_ref.collection("shards").list_documents() for shard in shards: total += shard.get().to_dict().get("count", 0) return total
Node.js
async function getCount(docRef) { const querySnapshot = await docRef.collection('shards').get(); const documents = querySnapshot.docs; let count = 0; for (const doc of documents) { count += doc.get('count'); } return count; }
Go
// getCount returns a total count across all shards. func (c *Counter) getCount(ctx context.Context, docRef *firestore.DocumentRef) (int64, error) { var total int64 shards := docRef.Collection("shards").Documents(ctx) for { doc, err := shards.Next() if err == iterator.Done { break } if err != nil { return 0, fmt.Errorf("Next: %v", err) } vTotal := doc.Data()["Count"] shardCount, ok := vTotal.(int64) if !ok { return 0, fmt.Errorf("firestore: invalid dataType %T, want int64", vTotal) } total += shardCount } return total, nil }
PHP
$result = 0; $docCollection = $ref->collection('SHARDS')->documents(); foreach ($docCollection as $doc) { $result += $doc->data()['Cnt']; }
C#
/// <summary> /// Get total count across all shards. /// </summary> /// <param name="docRef">The document reference <see cref="DocumentReference"/></param> /// <returns>The <see cref="int"/></returns> private static async Task<int> GetCountAsync(DocumentReference docRef) { var snapshotList = await docRef.Collection("shards").GetSnapshotAsync(); return snapshotList.Sum(shard => shard.GetValue<int>("count")); }
Ruby
# project_id = "Your Google Cloud Project ID" # collection_path = "shards" require "google/cloud/firestore" firestore = Google::Cloud::Firestore.new project_id: project_id shards_ref = firestore.col_group collection_path count = 0 shards_ref.get do |doc_ref| count += doc_ref[:count] end puts "Count value is #{count}."
Beschränkungen
Die oben gezeigte Lösung ist eine skalierbare Methode zum Erstellen gemeinsam genutzter Zähler in Firestore. Beachten Sie jedoch die folgenden Beschränkungen:
- Shard-Anzahl: Die Leistung des verteilten Zählers ergibt sich aus der Anzahl der Shards. Wenn zu wenige Shards vorhanden sind, müssen möglicherweise einige Transaktionen wiederholt werden, bevor sie erfolgreich ausgeführt werden. Dadurch werden Schreibvorgänge verlangsamt. Bei zu vielen Shards werden Lesevorgänge langsamer und teurer. Sie können die Lesekosten ausgleichen. Dazu belassen Sie die Zählersumme in einem separaten Rollup-Dokument, das mit einer langsameren Taktung aktualisiert wird (z. B. einmal pro Sekunde), und lassen die Clients aus diesem Dokument lesen, um die Gesamtsumme zu erhalten. Dabei gehen Sie den Kompromiss ein, dass Clients auf die Aktualisierung des Rollup-Dokuments warten müssen, anstatt die Shards unmittelbar nach jeder Aktualisierung auslesen und daraus die Gesamtsumme berechnen zu können.
- Kosten: Die Kosten für das Lesen eines Zählerwerts steigen linear mit der Anzahl der Shards, da die gesamte untergeordnete Sammlung der Shards geladen werden muss.