CCDAK Confluent Certified Developer for Apache Kafka Certification Examination Questions and Answers

Using assign() gives manual control over partition assignment. In this mode:

B is true: The application chooses partitions directly without involving the group coordinator.

C is true: Since no group coordination occurs, no rebalancing happens if a consumer dies.

From Kafka Consumer API Docs:

“assign() manually assigns a list of partitions to this consumer. No coordination takes place and no rebalancing is triggered.”

A is false: subscribe() and assign() are mutually exclusive.

D is incorrect: Partitions don't need to be equal across topics.

According to the official Apache Kafka consumer semantics documentation, at-most-once delivery means that records may be lost but must never be processed more than once. To achieve this guarantee, a consumer must commit offsets before processing records. If the application crashes after committing but before processing, messages are skipped rather than reprocessed, which satisfies at-most-once semantics.

Option B correctly implements this pattern. It disables automatic offset commits (enable.auto.commit=false) and explicitly calls commitAsync() immediately after polling and before processing records. This ensures that offsets are advanced regardless of whether processing succeeds, eliminating the possibility of duplicate processing.

Option C represents at-least-once semantics, because offsets are committed after processing, which can lead to duplicates on failure. Options A and D rely on automatic commits or incorrect ordering and do not provide strict at-most-once guarantees.

The Kafka documentation explicitly states that manual offset management with commits before processing is required for at-most-once delivery.

Therefore, Option B is the best and officially documented configuration and code skeleton to avoid duplicate message processing.

Increasing tasks.max allows the connector to create multiple tasks, each responsible for processing different tables or table partitions. If only one task is used, adding more workers has no effect.

From Kafka Connect JDBC Connector Docs:

“Use tasks.max to configure how many tasks are created for the connector. Each task can process a subset of the data.”

A and C help only if the bottleneck is in Kafka, not Connect.

D is a large architectural change, not the first step.

Increasing partitions increases parallelism, but also means:

Consumers in a group may have to handle more partitions, especially if the number of consumers is lower than the number of partitions.

This can result in increased lag, especially under high load.

From Kafka Topic Management Docs:

“Increasing the number of partitions increases consumer work, and if consumers can’t keep up, lag can accumulate.”

A is false: existing data is not redistributed.

B is false: records with the same key always map to the same partition based on hash.

D is not directly impacted by the partition count.

A GlobalKTable is a replicated, read-only table available in full on all instances. It's best for reference or lookup datasets such as:

Product catalog

Customer profiles

Warehouse inventory

From Kafka Streams Developer Guide:

“Use GlobalKTable when you need to perform joins using non-partition-aligned reference data that's small enough to replicate.”

Purchases and transactions are high-throughput, time-sensitive streams, not static reference data.

If the application is CPU-bound and lagging, adding more consumers to the group will allow better parallel processing, especially since the topic has 4 partitions, allowing up to 4 active consumers.

From Kafka Consumer Group Docs:

“Kafka achieves parallelism by distributing partitions across consumers in a group. Adding consumers helps reduce lag if partitions are underutilized.”

B may help but requires repartitioning and coordination.

C or D affects how much data is polled, not how fast it’s processed.

With acks=all, the leader waits for min.insync.replicas to acknowledge the message. Since min.insync.replicas=3, Kafka will only commit the message once 3 brokers (leader + 2 followers) confirm they have the message.

From Kafka Documentation > Acks and Durability:

“If acks=all is specified, the producer will wait until the full set of in-sync replicas has acknowledged the record. The minimum number of in-sync replicas is controlled by min.insync.replicas.”

Even though the replication factor is 4, only 3 acknowledgments are needed, as defined by min.insync.replicas.

unclean.leader.election.enable=false → Prevents data loss by only considering in-sync replicas when rebalancing.

replication.factor → Specifies how many redundant copies of partitions are distributed across brokers.

min.insync.replicas=2 → Sets the standard for the number of partition instances that must keep up with the latest committed message.

unclean.leader.election.enable=false ensures that only in-sync replicas can be elected as leaders. If disabled, an out-of-sync replica may become leader, potentially leading to data loss.

replication.factor defines how many brokers will maintain copies of each partition, directly impacting durability and availability.

min.insync.replicas determines how many replicas must acknowledge a write when acks=all is used, enforcing write durability.

Kafka Connect sink connectors use the standard Kafka consumer mechanism to track offsets, which means offsets are stored in the __consumer_offsets internal topic.

From Kafka Connect Documentation:

“Sink connectors are regular Kafka consumers and store their offsets in the __consumer_offsets topic like any other consumer.”

The other options refer to source connectors or worker configuration.

offsets.storage.topic is relevant for source connectors.

Kafka clients use SLF4J as a logging facade. To redirect Kafka logs to log4j2, you must include the SLF4J binding for log4j2 in your classpath:

“org.apache.logging.log4j:log4j-slf4j-impl bridges SLF4J to Log4j2 so that Kafka's logs integrate into your application's Log4j2 configuration.”

Option A is for Log4j 1.x, which is outdated.

Option C is incorrect; Kafka clients don't include logging bindings.

Option D is insufficient without the SLF4J bridge.

The default maximum message size that a Kafka broker accepts is 1MB (1,048,576 bytes), controlled by the config property message.max.bytes.

From Kafka Broker Configuration Docs:

“The default maximum message size is 1MB. To accept larger messages, configure message.max.bytes and the producer’s max.request.size.”

Producers also have a matching limit via max.request.size, and consumers via fetch.message.max.bytes.

This scenario describes vertical scaling, as defined in the Apache Kafka Streams documentation and general distributed systems terminology. Vertical scaling refers to adding more resources (CPU, memory, or disk) to a single machine to increase processing capacity.

In this example, the Kafka Streams application is moved from a server with four CPU threads to one with ten CPU threads, increasing the available compute resources without increasing the number of application instances. Kafka Streams can take advantage of additional CPU threads through its num.stream.threads configuration, allowing more partitions to be processed in parallel within the same application instance.

Horizontal scaling (also known as scaling out) would involve adding more application instances across multiple servers, not upgrading a single server. Option D is therefore incorrect, and Option C is not a recognized scaling term.

Thus, this example clearly illustrates vertical scaling, where performance is improved by increasing the capacity of a single node.

A: Kafka batches store compression metadata in their headers, indicating how the batch was compressed.

B: Compression is controlled via the producer config compression.type, which defaults to none. Producers explicitly set Gzip or other formats.

From Kafka Producer Docs:

“Compression is done by the producer, and the type is stored in the message batch metadata. The broker does not recompress.”

C is false: messages are serialized before compression.

D is false: messages without keys are distributed across multiple partitions.

According to the official Apache Kafka Connect documentation, RegexRouter is the SMT specifically designed to dynamically change the destination topic name of records produced by a connector. It works by applying a regular expression to the original topic name and rewriting it to a new topic name.

This makes RegexRouter the correct choice when some records must be routed to a different topic, typically in combination with connector-level logic, predicates, or multiple connectors. It is commonly used for topic renaming, topic versioning, or routing records to alternate topics.

InsertField (Option B) only adds metadata fields (such as topic, partition, or timestamp) to the record payload and does not affect routing. TombstoneHandler (Option C) is used to manage null-value records, especially with compacted topics. HeaderFrom (Option D) copies fields into headers but does not change the target topic.

Therefore, RegexRouter is the only SMT that directly supports changing the output topic, as documented in the Kafka Connect SMT reference.

=========================

To isolate logs for a specific connector, you can configure a separate logger and appender in the Connect worker’s log4j.properties file, using the connector’s name as the logging context.

From Kafka Connect Logging Docs:

“Kafka Connect loggers use hierarchical logger names. You can configure per-connector log levels and output files by extending log4j.properties.”

A and C change verbosity but don’t separate logs.

D is false; targeted logging is possible.

When a message key is specified in Kafka, the producer uses a partitioner (typically the default hash-based partitioner) to deterministically map all records with the same key to the same partition. Kafka guarantees order within a partition, so this enables per-key ordering.

From Kafka Producer Concepts:

“If a key is present, the producer will always route records with the same key to the same partition. Kafka preserves the order of records within a partition.”

This is essential for ordered processing and join semantics.

To ensure exactly-once delivery and avoid duplicates even during retries:

enable.idempotence=true ensures deduplication on the broker

retries=2147483647 allows unlimited retries on retriable errors

max.in.flight.requests.per.connection=5 is the maximum value that preserves message order with idempotence

From Kafka Producer Config Docs:

“To achieve exactly-once semantics, set enable.idempotence=true, and max.in.flight.requests.per.connection ≤ 5.”

A is unrelated (consumer-side)

C disables retries

D disables idempotence, leading to duplicates

Apache Kafka does not impose any schema or message format; it treats records as byte arrays on the wire. Serialization is therefore the responsibility of the producer. For lightweight XML messages, the most common and recommended approach is to represent the XML as a UTF-8 encoded string and use Kafka’s built-in StringSerializer.

The official Kafka producer documentation provides StringSerializer as the standard serializer for textual data formats, including XML and JSON. This serializer converts Java String objects into byte arrays using UTF-8 encoding, making it simple, efficient, and interoperable with a wide range of consumers.

Option A is incorrect because Kafka does not provide a built-in XmlSerializer. Option C (ByteSerializer) could technically work, but it would require manually converting the XML string to a byte array, which adds unnecessary complexity for lightweight XML payloads. Option D (VoidSerializer) is used only when no value is sent and is not applicable here.

Therefore, using StringSerializer for the value field is the correct and officially supported approach for sending lightweight XML messages in Kafka.

=========================

To maintain message ordering per bank account, the message key must be the account number. Kafka guarantees ordering per partition, and the partition is selected based on the hash of the key.

From Kafka Producer Docs:

“All records with the same key will be sent to the same partition, and Kafka preserves the order of records in a partition.”

Using account number ensures:

All messages for the same account go to the same partition

Order is maintained

Workload is evenly distributed if account keys are varied

Other options break ordering or introduce unnecessary randomness.

According to the official Apache Kafka Streams documentation, Kafka Streams relies on internal topics (such as changelog topics and repartition topics) to store state, task assignments, and application metadata. These topics are essential for fault tolerance, rebalancing, and state recovery.

When auto-topic creation is disabled, Kafka Streams cannot automatically create the required internal topics unless they already exist. In this situation, the Streams application will fail during startup with a fatal, non-retriable exception, indicating that the necessary internal topics could not be created.

Kafka Streams does not pause or wait for manual topic creation, nor does it operate without storing metadata. Instead, it explicitly requires these topics to exist or be creatable. While administrators can manually pre-create the required internal topics with the correct configuration, failure to do so causes the application to terminate.

Therefore, the correct and documented behavior is that the Kafka Streams application terminates with a non-retriable exception when auto-topic creation is disabled and required internal topics are missing.

According to the official Apache Kafka Streams documentation, session windows are specifically designed to model periods of activity separated by gaps of inactivity. A session window groups events that occur close together in time and closes when no new events arrive within a defined inactivity gap.

In this use case, tracking user shopping cart behavior requires detecting when a user becomes inactive (for example, stops adding or removing items). Session windows naturally represent this pattern by creating a new window when activity resumes after a period of silence. This makes them ideal for modeling user sessions, cart abandonment detection, and inactivity analysis.

Hopping and tumbling windows (Options B and C) are fixed-time windows and are not well-suited for detecting inactivity, as they segment data based on clock time rather than behavior. Sliding windows are primarily used for continuous joins and do not explicitly model inactivity gaps.

Therefore, session windows are the correct and officially recommended choice for identifying periods of user inactivity in Kafka Streams applications.

=========================

To skip a corrupted record and avoid failing the application, you must seek past the failed offset manually using consumer.seek(). This allows the application to resume consumption from the next offset.

From Kafka Consumer Error Handling Docs:

“On deserialization failure, you can catch RecordDeserializationException, log the error, and call seek() to the next offset to skip the bad record.”

A does not prevent re-processing the bad record.

C is invalid; there's no skip() method in the Kafka consumer API.

D results in service interruption — not ideal for resiliency.

1st: Place the connector's JAR file in the directory specified by plugin.path

2nd: Restart the Kafka Connect cluster

3rd: Verify using REST API (/connector-plugins)

4th: Configure the connector

5th: (Repeat of 4th, duplicate step)

1st → Place the connector’s JAR file in the directory specified by the plugin.path configuration.

2nd → Restart the Kafka Connect cluster.

3rd → Verify that the connector is installed by listing all available connectors using the Kafka Connect REST API (/connector-plugins).

4th → Configure the connector using a JSON or properties file with the necessary settings.

5th → Configure the connector using a JSON or properties file with the necessary settings.

Kafka Connect requires that custom connectors be placed in the directory defined by plugin.path. After restarting the cluster, you can use the REST API to confirm availability and then deploy the connector configuration.

From Kafka Connect Documentation:

“After placing the JAR in the plugin.path, you must restart the Connect cluster to pick it up. Use the /connector-plugins REST endpoint to verify.”

The duplication of configuration is an error in the question options and should occur only once.

Unit Testing → MockProducer

Integration Testing → Testcontainers

Performance Testing → Trogdor

Mock Data Generation → Connect Datagen

MockProducer: Simulates a Kafka producer in unit tests (no real broker interaction).

Testcontainers: Spawns Kafka in Docker for real environment testing.

Trogdor: Kafka’s built-in performance load testing framework.

Connect Datagen: Creates sample source records for test and demo purposes.

From Kafka Developer Tools Guide:

“Kafka developers commonly use MockProducer for unit tests, Testcontainers for integration, and Trogdor for performance tests.”