Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: Badger expired data is still there(using txn.Get(k) ). after a restart or several days running. #2138

Open
winjeg opened this issue Dec 18, 2024 · 0 comments
Labels
kind/bug Something is broken.

Comments

@winjeg
Copy link

winjeg commented Dec 18, 2024

What version of Badger are you using?

github.com/dgraph-io/badger/v4 v4.2.0

What version of Go are you using?

go 1.22.0
toolchain go1.22.4

Have you tried reproducing the issue with the latest release?

No

What is the hardware spec (RAM, CPU, OS)?

16Core CPU. 64GB Ram , Linux

What steps will reproduce the bug?

I am not completely sure about how to reproduce the bug. It will appear after I run several hours or days badger operation. After a restart, some of the expired keys is still there when I scan the related keys or get the key directly.

Can someone give me a bit of advice or guidance? Thanks.

Pseudocode:

func writeData(k1, k2, k3[]byte) {
  	txn := tu.db.NewTransaction(true)
	defer txn.Discard()

        k1DataV1 := txn.Get(k1)
        k1DataV2 := processK1(k1DataV1)
        txn.set(k1, k1DataV2)

        k2DataV1 := txn.Get(k2)
        k2DataV2 := processK1(k2DataV1)
        txn.set(k2, k2DataV2)

        k3DataV1 := txn.Get(k3)
        k3DataV2 := processK1(k3DataV1)
        txn.set(k3, k3DataV2)

        txn.setWithTTL(k2, k2DataV3, time.Second * 300)

}

Expected behavior and actual result.

Expired data should not be accessable via the basic get or scan APIs of badgerdb.
Should return key not found like errors.

Additional information

shutdown is called when the program caught kill signal.

The sync error log didn't appear.
The Server exiting end --- log appeared.

shutdown code

func BadgerClose() {
	for _, v := range defaultMgr.instances {
		if v != nil {
			if syncErr := v.Sync(); syncErr != nil {
				logger.Errorf("sync error:%+v\n", syncErr)
			}
			err := v.Close()
			if err != nil {
				logger.Errorln(err.Error())
			}
		}
	}
}

func shutdown() {
     BadgerClose()
     logger.Info("Server exiting end ---------------- ")
}

badger options

opts := badger.DefaultOptions(getPath(badgerCfg.BasePath, key))
opts = opts.WithLogger(logger)
opts = opts.WithMetricsEnabled(true)
opts = opts.WithNumMemtables(badgerCfg.MemTableNum)
opts = opts.WithMemTableSize(badgerCfg.MemTableSize << 20)
opts = opts.WithNumLevelZeroTables(8)
opts = opts.WithNumCompactors(4)
opts = opts.WithValueLogFileSize(128 << 20)
opts = opts.WithCompactL0OnClose(true)
db, err := badger.Open(opts)
@winjeg winjeg added the kind/bug Something is broken. label Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something is broken.
Development

No branches or pull requests

1 participant