如果通道已满或经过特定时间,我希望从通道进行处理。我的用例类似于现有的question,我已尝试修改现有的ans
我的代码位于https://go.dev/play/p/HaGZ9HHqj0i,
package main
import (
"fmt"
"sync"
"time"
)
type Audit struct {
ID int
}
const batchSize = 5
var batch = make([]Audit, 0, batchSize)
func upsertBigQueryAudits(audits []Audit) {
fmt.Printf("Processing batch of %d\n", len(audits))
for _, a := range audits {
fmt.Printf("%d ", a.ID)
}
fmt.Println()
batch = []Audit{}
}
func process(full <-chan struct{}) {
ticker := time.NewTicker(1 * time.Nanosecond)
for {
select {
case <-full:
fmt.Println("From full")
upsertBigQueryAudits(batch)
case <-ticker.C:
fmt.Println("From ticker")
if len(batch) > 0 {
fmt.Println("From ticker sending batch")
upsertBigQueryAudits(batch)
}
}
}
}
func processAudits(audits <-chan Audit, full chan<- struct{}, batchSize int) {
for audit := range audits {
batch = append(batch, audit)
if len(batch) == cap(batch) {
// upsertBigQueryAudits(batch)
fmt.Println("Sending full")
full <- struct{}{}
}
}
}
func produceAudits(x int, to chan Audit) {
for i := 0; i < x; i++ {
to <- Audit{
ID: i,
}
}
}
func main() {
var wg sync.WaitGroup
audits := make(chan Audit)
full := make(chan struct{})
wg.Add(1)
go func() {
defer wg.Done()
process(full)
}()
wg.Add(1)
go func() {
defer wg.Done()
processAudits(audits, full, batchSize)
}()
wg.Add(1)
go func() {
defer wg.Done()
produceAudits(25, audits)
close(audits)
}()
wg.Wait()
fmt.Println("Complete")
}
在这里,我想处理的批次时,它是充满batchSize
元素或当一定的时间(如1纳秒)过去了。
编辑:添加了我的关注内联。这是示例输出。
timeout running program
Sending full <--- this line comes twice
Sending full
From full
Processing batch of 10 <--- instead of 5 it is processing 10 items
0 1 2 3 4 5 6 7 8 9
From full
Processing batch of 1
10
Sending full
Sending full
From full
Processing batch of 1 <-- instead of 5 it is processing 1 items. It does not process more than 1 item.
11
From full
Processing batch of 0
...
...
Sending full
Sending full
From full
Processing batch of 1 <-- instead of 5 it is processing 1 items
24
From full
Processing batch of 0
From ticker <-- this line comes only after consuming 25 items.
From ticker
From ticker
From ticker
1条答案
按热度按时间mklgxw1f1#
我在https://elliotchance.medium.com/batch-a-channel-by-size-or-time-in-go-92fa3098f65找到了答案