Skip to content

MongoDB replicaset oplog tailing in realtime by Golang

License

Notifications You must be signed in to change notification settings

gnokoheat/oplog

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Oplog

GitHub GitHub top language GitHub release (latest by date) GitHub last commit

MongoDB replicaset oplog tailing by Golang

Tailing of MongoDB Oplog

It can be send oplog to everywhere with minimal system resource usage.

  • MongoDB replicaset -> Oplog -> MongoDB | Logstash | Fluentd | ...

Install

go get -u github.com/gnokoheat/oplog

Usage example

  • main.go
package main

import (
	"log"
	"github.com/gnokoheat/oplog"
)

func main() {
	var o = &oplog.Options{
		// (e.g. mongodb://username:[email protected]:27017,127.0.0.1:27018/local?replicaSet=rs01&authSource=admin)
		Addrs:      []string{"127.0.0.1:27017", "127.0.0.1:27018"}, // replicaset host and port
		Username:   "username", // admin db username
		Password:   "password", // admin db user password
		ReplicaSet: "rs01", // replicaset name
		DB:         "myDB", // tailing target db
		Collection: "myCollection", // tailing target collection
		Events:     []string{"insert", "update", "delete"}, // tailing target method
	}

	l := make(chan *[]oplog.Log) // Oplog Channel
	e := make(chan error) // Error Channel
	
	// Oplog tailing start ! 
	go o.Tail(l, e)

	for {
		select {
		case err := <-e:
			log.Println("[Error] ", err)
			return
		case op := <-l:
			// input oplog handling code
			log.Println("[Result] ", op)
			break
		}
	}
}
  • Run
go run main.go
  • Result
2019/11/08 16:13:57 [Oplog Tail Start]  2019-11-08 07:13:57.485633 +0000 UTC
2019/11/08 16:14:04 [Result]  &[{2019-11-08 16:14:02.744 +0900 ... ]
2019/11/08 16:14:08 [Result]  &[{2019-11-08 16:14:08.554 +0900 ... ]
2019/11/08 16:16:57 [Result]  &[{2019-11-08 16:16:57.364 +0900 ... ]
...