Consumer results location

[CROprogrammer]

Hi,

how does KLoadGen consumer work, where can I see the results of its execution?

In the logs I do not see nothing.
Logs:

2023-07-21 11:13:55,268 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : Kloadgen Thread Group
2023-07-21 11:13:55,268 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Kloadgen Thread Group.
2023-07-21 11:13:55,268 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2023-07-21 11:13:55,268 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 delayedStart=false
2023-07-21 11:13:55,268 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2023-07-21 11:13:55,268 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2023-07-21 11:13:55,268 INFO o.a.j.t.JMeterThread: Thread started: Kloadgen Thread Group 1-1
2023-07-21 11:13:58,306 INFO o.a.k.c.p.ProducerConfig: ProducerConfig values: 
	acks = 1
	batch.size = 16384
	bootstrap.servers = [localhost:9092]
	buffer.memory = 33554432
	client.dns.lookup = use_all_dns_ips
	client.id = 
	compression.type = none
	connections.max.idle.ms = 540000
	delivery.timeout.ms = 120000
	enable.idempotence = true
	interceptor.classes = []
	key.serializer = class org.apache.kafka.common.serialization.StringSerializer
	linger.ms = 0
	max.block.ms = 60000
	max.in.flight.requests.per.connection = 5
	max.request.size = 1048576
	metadata.max.age.ms = 300000
	metadata.max.idle.ms = 300000
	metric.reporters = []
	metrics.num.samples = 2
	metrics.recording.level = INFO
	metrics.sample.window.ms = 30000
	partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
	receive.buffer.bytes = 32768
	reconnect.backoff.max.ms = 1000
	reconnect.backoff.ms = 50
	request.timeout.ms = 30000
	retries = 2147483647
	retry.backoff.ms = 100
	sasl.client.callback.handler.class = null
	sasl.jaas.config = null
	sasl.kerberos.kinit.cmd = /usr/bin/kinit
	sasl.kerberos.min.time.before.relogin = 60000
	sasl.kerberos.service.name = null
	sasl.kerberos.ticket.renew.jitter = 0.05
	sasl.kerberos.ticket.renew.window.factor = 0.8
	sasl.login.callback.handler.class = null
	sasl.login.class = null
	sasl.login.connect.timeout.ms = null
	sasl.login.read.timeout.ms = null
	sasl.login.refresh.buffer.seconds = 300
	sasl.login.refresh.min.period.seconds = 60
	sasl.login.refresh.window.factor = 0.8
	sasl.login.refresh.window.jitter = 0.05
	sasl.login.retry.backoff.max.ms = 10000
	sasl.login.retry.backoff.ms = 100
	sasl.mechanism = GSSAPI
	sasl.oauthbearer.clock.skew.seconds = 30
	sasl.oauthbearer.expected.audience = null
	sasl.oauthbearer.expected.issuer = null
	sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000
	sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000
	sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100
	sasl.oauthbearer.jwks.endpoint.url = null
	sasl.oauthbearer.scope.claim.name = scope
	sasl.oauthbearer.sub.claim.name = sub
	sasl.oauthbearer.token.endpoint.url = null
	security.protocol = PLAINTEXT
	security.providers = null
	send.buffer.bytes = 131072
	socket.connection.setup.timeout.max.ms = 30000
	socket.connection.setup.timeout.ms = 10000
	ssl.cipher.suites = null
	ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
	ssl.endpoint.identification.algorithm = https
	ssl.engine.factory.class = null
	ssl.key.password = null
	ssl.keymanager.algorithm = SunX509
	ssl.keystore.certificate.chain = null
	ssl.keystore.key = null
	ssl.keystore.location = null
	ssl.keystore.password = null
	ssl.keystore.type = JKS
	ssl.protocol = TLSv1.3
	ssl.provider = null
	ssl.secure.random.implementation = null
	ssl.trustmanager.algorithm = PKIX
	ssl.truststore.certificates = null
	ssl.truststore.location = null
	ssl.truststore.password = null
	ssl.truststore.type = JKS
	transaction.timeout.ms = 60000
	transactional.id = null
	value.serializer = class io.apicurio.registry.serde.avro.AvroKafkaSerializer

2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'apicurio.registry.auto-register' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'schema.registry.name' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'ssl.keystore.type' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'schema.registry.url' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'apicurio.registry.artifact-resolver-strategy' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'message.key.type' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'ssl.endpoint.identification.algorithm' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'ssl.protocol' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'ssl.enabled.protocols' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'apicurio.registry.url' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'ssl.keymanager.algorithm' was supplied but isn't a known config.
2023-07-21 11:13:58,308 WARN o.a.k.c.p.ProducerConfig: The configuration 'message.key.value' was supplied but isn't a known config.
2023-07-21 11:13:58,308 INFO o.a.k.c.u.AppInfoParser: Kafka version: 3.1.0
2023-07-21 11:13:58,308 INFO o.a.k.c.u.AppInfoParser: Kafka commitId: 37edeed0777bacb3
2023-07-21 11:13:58,308 INFO o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1689930838308
2023-07-21 11:13:58,318 INFO o.a.k.c.Metadata: [Producer clientId=] Resetting the last seen epoch of partition kloadgen-0 to 0 since the associated topicId changed from null to H9wOl5O5QiqrtteuU8wAsA
2023-07-21 11:13:58,319 INFO o.a.k.c.Metadata: [Producer clientId=] Cluster ID: zDWX7ZkbTaixUVY84rEgGw
2023-07-21 11:14:01,347 INFO o.a.k.c.c.ConsumerConfig: ConsumerConfig values: 
	allow.auto.create.topics = true
	auto.commit.interval.ms = 5000
	auto.offset.reset = earliest
	bootstrap.servers = [localhost:9092]
	check.crcs = true
	client.dns.lookup = use_all_dns_ips
	client.id = consumer-anonymous-38
	client.rack = 
	connections.max.idle.ms = 540000
	default.api.timeout.ms = 60000
	enable.auto.commit = false
	exclude.internal.topics = true
	fetch.max.bytes = 57671680
	fetch.max.wait.ms = 500
	fetch.min.bytes = 1
	group.id = anonymous
	group.instance.id = null
	heartbeat.interval.ms = 3000
	interceptor.classes = []
	internal.leave.group.on.close = true
	internal.throw.on.fetch.stable.offset.unsupported = false
	isolation.level = read_uncommitted
	key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
	max.partition.fetch.bytes = 1048576
	max.poll.interval.ms = 300000
	max.poll.records = 500
	metadata.max.age.ms = 300000
	metric.reporters = []
	metrics.num.samples = 2
	metrics.recording.level = INFO
	metrics.sample.window.ms = 30000
	partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor, class org.apache.kafka.clients.consumer.CooperativeStickyAssignor]
	receive.buffer.bytes = 32768
	reconnect.backoff.max.ms = 1000
	reconnect.backoff.ms = 50
	request.timeout.ms = 30000
	retry.backoff.ms = 100
	sasl.client.callback.handler.class = null
	sasl.jaas.config = null
	sasl.kerberos.kinit.cmd = /usr/bin/kinit
	sasl.kerberos.min.time.before.relogin = 60000
	sasl.kerberos.service.name = null
	sasl.kerberos.ticket.renew.jitter = 0.05
	sasl.kerberos.ticket.renew.window.factor = 0.8
	sasl.login.callback.handler.class = null
	sasl.login.class = null
	sasl.login.connect.timeout.ms = null
	sasl.login.read.timeout.ms = null
	sasl.login.refresh.buffer.seconds = 300
	sasl.login.refresh.min.period.seconds = 60
	sasl.login.refresh.window.factor = 0.8
	sasl.login.refresh.window.jitter = 0.05
	sasl.login.retry.backoff.max.ms = 10000
	sasl.login.retry.backoff.ms = 100
	sasl.mechanism = GSSAPI
	sasl.oauthbearer.clock.skew.seconds = 30
	sasl.oauthbearer.expected.audience = null
	sasl.oauthbearer.expected.issuer = null
	sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000
	sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000
	sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100
	sasl.oauthbearer.jwks.endpoint.url = null
	sasl.oauthbearer.scope.claim.name = scope
	sasl.oauthbearer.sub.claim.name = sub
	sasl.oauthbearer.token.endpoint.url = null
	security.protocol = PLAINTEXT
	security.providers = null
	send.buffer.bytes = 131072
	session.timeout.ms = 45000
	socket.connection.setup.timeout.max.ms = 30000
	socket.connection.setup.timeout.ms = 10000
	ssl.cipher.suites = null
	ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
	ssl.endpoint.identification.algorithm = https
	ssl.engine.factory.class = null
	ssl.key.password = null
	ssl.keymanager.algorithm = SunX509
	ssl.keystore.certificate.chain = null
	ssl.keystore.key = null
	ssl.keystore.location = null
	ssl.keystore.password = null
	ssl.keystore.type = JKS
	ssl.protocol = TLSv1.3
	ssl.provider = null
	ssl.secure.random.implementation = null
	ssl.trustmanager.algorithm = PKIX
	ssl.truststore.certificates = null
	ssl.truststore.location = null
	ssl.truststore.password = null
	ssl.truststore.type = JKS
	value.deserializer = class io.apicurio.registry.serde.avro.AvroKafkaDeserializer

2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'schema.registry.name' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'kafka.topic.name' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'value.subject.name.strategy' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'ssl.keystore.type' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'schema.registry.url' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'ssl.endpoint.identification.algorithm' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'ssl.protocol' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'ssl.enabled.protocols' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'timeout.millis' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'apicurio.registry.url' was supplied but isn't a known config.
2023-07-21 11:14:01,348 WARN o.a.k.c.c.ConsumerConfig: The configuration 'ssl.keymanager.algorithm' was supplied but isn't a known config.
2023-07-21 11:14:01,348 INFO o.a.k.c.u.AppInfoParser: Kafka version: 3.1.0
2023-07-21 11:14:01,348 INFO o.a.k.c.u.AppInfoParser: Kafka commitId: 37edeed0777bacb3
2023-07-21 11:14:01,348 INFO o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1689930841348
2023-07-21 11:14:01,348 INFO o.a.k.c.c.KafkaConsumer: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Subscribed to topic(s): kloadgen
2023-07-21 11:14:01,355 INFO o.a.k.c.Metadata: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Resetting the last seen epoch of partition kloadgen-0 to 0 since the associated topicId changed from null to H9wOl5O5QiqrtteuU8wAsA
2023-07-21 11:14:01,356 INFO o.a.k.c.Metadata: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Cluster ID: zDWX7ZkbTaixUVY84rEgGw
2023-07-21 11:14:01,356 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Discovered group coordinator localhost:9092 (id: 2147483647 rack: null)
2023-07-21 11:14:01,356 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] (Re-)joining group
2023-07-21 11:14:01,360 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Request joining group due to: need to re-join with the given member-id
2023-07-21 11:14:01,360 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] (Re-)joining group
2023-07-21 11:14:01,364 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Successfully joined group with generation Generation{generationId=50, memberId='consumer-anonymous-38-8fc3febc-b63b-4fcb-9bef-92ae407f468b', protocol='range'}
2023-07-21 11:14:01,364 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Finished assignment for group at generation 50: {consumer-anonymous-38-8fc3febc-b63b-4fcb-9bef-92ae407f468b=Assignment(partitions=[kloadgen-0])}
2023-07-21 11:14:01,367 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Successfully synced group in generation Generation{generationId=50, memberId='consumer-anonymous-38-8fc3febc-b63b-4fcb-9bef-92ae407f468b', protocol='range'}
2023-07-21 11:14:01,367 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Notifying assignor about the new Assignment(partitions=[kloadgen-0])
2023-07-21 11:14:01,367 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Adding newly assigned partitions: kloadgen-0
2023-07-21 11:14:01,369 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Setting offset for partition kloadgen-0 to the committed offset FetchPosition{offset=29, offsetEpoch=Optional[0], currentLeader=LeaderAndEpoch{leader=Optional[localhost:9092 (id: 0 rack: null)], epoch=0}}
2023-07-21 11:14:01,402 INFO o.a.j.t.JMeterThread: Thread is done: Kloadgen Thread Group 1-1
2023-07-21 11:14:01,402 INFO o.a.j.t.JMeterThread: Thread finished: Kloadgen Thread Group 1-1
2023-07-21 11:14:01,402 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2023-07-21 11:14:01,403 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Revoke previously assigned partitions kloadgen-0
2023-07-21 11:14:01,403 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Member consumer-anonymous-38-8fc3febc-b63b-4fcb-9bef-92ae407f468b sending LeaveGroup request to coordinator localhost:9092 (id: 2147483647 rack: null) due to the consumer is being closed
2023-07-21 11:14:01,403 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Resetting generation due to: consumer pro-actively leaving the group
2023-07-21 11:14:01,403 INFO o.a.k.c.c.i.ConsumerCoordinator: [Consumer clientId=consumer-anonymous-38, groupId=anonymous] Request joining group due to: consumer pro-actively leaving the group
2023-07-21 11:14:01,406 INFO o.a.k.c.m.Metrics: Metrics scheduler closed
2023-07-21 11:14:01,406 INFO o.a.k.c.m.Metrics: Closing reporter org.apache.kafka.common.metrics.JmxReporter
2023-07-21 11:14:01,406 INFO o.a.k.c.m.Metrics: Metrics reporters closed
2023-07-21 11:14:01,407 INFO o.a.k.c.u.AppInfoParser: App info kafka.consumer for consumer-anonymous-38 unregistered
2023-07-21 11:14:01,407 INFO o.a.k.c.p.KafkaProducer: [Producer clientId=] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
2023-07-21 11:14:01,410 INFO o.a.k.c.m.Metrics: Metrics scheduler closed
2023-07-21 11:14:01,410 INFO o.a.k.c.m.Metrics: Closing reporter org.apache.kafka.common.metrics.JmxReporter
2023-07-21 11:14:01,410 INFO o.a.k.c.m.Metrics: Metrics reporters closed
2023-07-21 11:14:01,410 INFO o.a.k.c.u.AppInfoParser: App info kafka.producer for  unregistered
2023-07-21 11:14:01,411 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, *local*)

Used shema:

{
  "type": "record",
  "name": "FirstSchemaStreamingPOCTest",
  "namespace": "test",
  "fields": [
    {
      "name": "randomNumber",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "randomCode",
      "type": [
        "null",
        "string"
      ],
      "default": null
    }
  ]
}

Used version: KLoadGen 5.6.5 => from branch https://github.com/sngular/kloadgen/tree/395-issue-schema-registry-url-in-consumer

JMX file:
Kafka KLoadGen Test.zip

[AdrianLagartera]

If you want check the results of KLoad from the consumer and the producer it's the same way, using one of the config elements called "View Results In Table" or "View Results Tree" I usually use the second one but both should work fine. They show the headers, and the message content, and a few more variables.

And basically works after you run the process.