Add Kafka event sourcing for Go functions by aliok · Pull Request #3950 · knative/func

aliok · 2026-07-16T11:59:03Z

Depends on knative-extensions/func-go#185 which is merged

Summary

Add run.kafka config to func.yaml (brokers, topic, consumerGroup)
instead of a separate invoke: kafka type
Deployer and runner automatically set FUNC_TRANSPORT=kafka plus
KAFKA_BROKERS, KAFKA_TOPIC, KAFKA_CONSUMER_GROUP env vars from the
config — user never sets these manually
Scaffolding main.go branches at runtime between kafka.Start() and
ce.Start() based on FUNC_TRANSPORT
Add Kafka consumption docs to CloudEvents template READMEs
Add fmt.Println(e) to instanced CloudEvents template to match static
template behavior

How it works

Users add a run.kafka section to their existing CloudEvents function:

invoke: cloudevent
run:
  kafka:
    brokers: "my-cluster-kafka-bootstrap.kafka:9092"
    topic: "my-topic"
    consumerGroup: "my-function-group"

No code changes — the same Handle(event.Event) method receives Kafka
messages automatically converted to CloudEvents by the runtime.

Dependencies

Requires func-go #185

Limitations

Go only

Supersedes #3923

Closes #3940

Copilot

Pull request overview

This PR introduces Kafka-backed event consumption for Go CloudEvents functions by adding a run.kafka configuration block (brokers/topic/consumerGroup) and wiring deploy/run paths to select Kafka transport at runtime via environment variables.

Changes:

Adds run.kafka to the function model, validation, and func.yaml JSON schema.
Injects FUNC_TRANSPORT=kafka plus KAFKA_* env vars from run.kafka in the runner and Kubernetes/Knative deployers.
Updates Go CloudEvents templates/scaffolding and READMEs to support/describe Kafka consumption, and aligns instanced template logging with static template behavior.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
templates/go/scaffolding/static-cloudevents/main.go	Switches runtime start between CloudEvents HTTP and Kafka based on `FUNC_TRANSPORT`.
templates/go/scaffolding/instanced-cloudevents/main.go	Same transport switch for instanced CloudEvents functions.
templates/go/cloudevents/README.md	Adds Kafka consumption documentation for the Go CloudEvents template.
templates/go/cloudevents/function.go	Adds event echo output to match static template behavior.
templates/go/.static-cloudevents/README.md	Adds Kafka consumption documentation for the static Go CloudEvents template.
schema/func_yaml-schema.json	Extends schema with `run.kafka` configuration (`KafkaConfig`).
pkg/scaffolding/signatures.go	Minor formatting/comment tweak to signature map.
pkg/scaffolding/signatures_test.go	Asserts `kafka` invoke/template remains unknown in scaffolding signature selection.
pkg/knative/deployer.go	Appends Kafka env vars when generating/updating Knative services.
pkg/k8s/deployer.go	Appends Kafka env vars during Kubernetes deployment generation.
pkg/k8s/deployer_test.go	Adds unit tests for Kafka env var injection helper.
pkg/functions/runner.go	Adds Kafka env var injection for local/host runner jobs.
pkg/functions/runner_test.go	Adds unit tests validating runner env injection behavior.
pkg/functions/function.go	Adds `KafkaConfig`, validation rules, and `RunSpec.Kafka` field.
pkg/functions/function_test.go	Adds YAML round-trip and omit-empty tests for `Run.Kafka`.
docs/reference/func_create.md	Updates generated CLI reference output to include a Go `kafka` template.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 1 comment.

lkingland

This is great!
/hold for anyone else's eyes, but it looks good to me.

knative-prow · 2026-07-20T07:38:31Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aliok, lkingland

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [lkingland]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

lkingland · 2026-07-21T04:28:02Z

func-go knative-extensions/func-go/pull/185 is on main and tagged as v0.22.0:

To update:

make update-runtimes   # update-runtime-go + generate/zz_filesystem_generated.go

To verify:

grep 'knative.dev/func-go' templates/go/scaffolding/*/go.mod
# expect: knative.dev/func-go v0.22.0

cc @aliok

knative-prow · 2026-07-21T10:40:17Z

New changes are detected. LGTM label has been removed.

Copilot

Pull request overview

Copilot reviewed 20 out of 25 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated no new comments.

Copilot

Pull request overview

Copilot reviewed 17 out of 22 changed files in this pull request and generated no new comments.

codecov · 2026-07-21T14:17:32Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.08%. Comparing base (d93ceec) to head (4d952d8).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3950      +/-   ##
==========================================
- Coverage   54.19%   52.08%   -2.12%     
==========================================
  Files         203      203              
  Lines       23858    23900      +42     
==========================================
- Hits        12931    12448     -483     
- Misses       9688    10300     +612     
+ Partials     1239     1152      -87

Flag	Coverage Δ
e2e	`34.11% <29.26%> (-0.02%)`	⬇️
e2e go	`29.24% <23.52%> (-0.02%)`	⬇️
e2e node	`25.53% <23.52%> (-0.03%)`	⬇️
e2e python	`29.56% <23.52%> (-0.02%)`	⬇️
e2e quarkus	`25.67% <23.52%> (-0.01%)`	⬇️
e2e rust	`25.10% <23.52%> (-0.01%)`	⬇️
e2e springboot	`23.81% <23.52%> (-0.01%)`	⬇️
e2e typescript	`25.66% <23.52%> (-0.01%)`	⬇️
e2e-config-ci	`26.80% <23.52%> (-0.03%)`	⬇️
integration	`?`
unit macos-14	`43.66% <97.05%> (+0.09%)`	⬆️
unit macos-latest	`43.66% <97.05%> (+0.09%)`	⬆️
unit ubuntu-24.04-arm	`43.93% <97.56%> (+0.09%)`	⬆️
unit ubuntu-latest	`44.51% <97.05%> (+0.09%)`	⬆️
unit windows-latest	`43.71% <97.05%> (+0.09%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

knative-prow Bot requested review from dsimansk and jrangelramos July 16, 2026 11:59

knative-prow Bot added the size/L 🤖 PR changes 100-499 lines, ignoring generated files. label Jul 16, 2026

aliok mentioned this pull request Jul 16, 2026

[WIP] Add Kafka invoke type for Go functions #3923

Closed

aliok changed the title ~~Add Kafka event sourcing for Go functions~~ [WIP] Add Kafka event sourcing for Go functions Jul 16, 2026

knative-prow Bot added the do-not-merge/work-in-progress 🤖 PR should not merge because it is a work in progress. label Jul 16, 2026

aliok requested a review from Copilot July 16, 2026 13:56

Copilot started reviewing on behalf of aliok July 16, 2026 13:57 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread templates/go/cloudevents/README.md

Comment thread docs/reference/func_create.md

Comment thread pkg/k8s/deployer.go

Comment thread templates/go/scaffolding/static-cloudevents/main.go

Comment thread templates/go/scaffolding/instanced-cloudevents/main.go

aliok requested a review from Copilot July 16, 2026 20:20

Copilot started reviewing on behalf of aliok July 16, 2026 20:21 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread pkg/functions/runner_test.go

Comment thread pkg/functions/runner_test.go

aliok requested a review from Copilot July 16, 2026 20:48

Copilot started reviewing on behalf of aliok July 16, 2026 20:48 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread templates/go/cloudevents/README.md Outdated

Comment thread templates/go/.static-cloudevents/README.md Outdated

aliok requested a review from Copilot July 16, 2026 21:10

Copilot started reviewing on behalf of aliok July 16, 2026 21:11 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread pkg/functions/function.go

Comment thread schema/func_yaml-schema.json

aliok requested a review from Copilot July 17, 2026 08:12

Copilot started reviewing on behalf of aliok July 17, 2026 08:12 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Comment thread pkg/functions/runner.go

Comment thread pkg/functions/function.go

aliok requested a review from Copilot July 17, 2026 08:56

Copilot started reviewing on behalf of aliok July 17, 2026 08:56 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Comment thread pkg/functions/function_test.go

lkingland approved these changes Jul 20, 2026

View reviewed changes

knative-prow Bot added the do-not-merge/hold 🤖 PR should not merge because someone has issued a /hold command. label Jul 20, 2026

knative-prow Bot assigned lkingland Jul 20, 2026

knative-prow Bot added the lgtm 🤖 PR is ready to be merged. label Jul 20, 2026

knative-prow Bot added the approved 🤖 PR has been approved by an approver from all required OWNERS files. label Jul 20, 2026

aliok force-pushed the kafka-cloudevents-source branch from 2cdeeb6 to d682952 Compare July 21, 2026 10:40

knative-prow Bot removed the lgtm 🤖 PR is ready to be merged. label Jul 21, 2026

knative-prow Bot added size/XL 🤖 PR changes 500-999 lines, ignoring generated files. and removed size/L 🤖 PR changes 100-499 lines, ignoring generated files. labels Jul 21, 2026

aliok requested a review from Copilot July 21, 2026 11:15

Copilot started reviewing on behalf of aliok July 21, 2026 11:15 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Comment thread pkg/scaffolding/scaffold.go Outdated

Add Kafka event sourcing for Go functions

b7eeeae

aliok force-pushed the kafka-cloudevents-source branch from d682952 to b7eeeae Compare July 21, 2026 12:28

aliok changed the title ~~[WIP] Add Kafka event sourcing for Go functions~~ Add Kafka event sourcing for Go functions Jul 21, 2026

knative-prow Bot added size/L 🤖 PR changes 100-499 lines, ignoring generated files. and removed size/XL 🤖 PR changes 500-999 lines, ignoring generated files. do-not-merge/work-in-progress 🤖 PR should not merge because it is a work in progress. labels Jul 21, 2026

aliok requested a review from Copilot July 21, 2026 12:28

Copilot started reviewing on behalf of aliok July 21, 2026 12:29 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

fix: tidy scaffolding go.mod/go.sum for func-go v0.22.0 kafka deps

4d952d8

knative-prow Bot added size/XL 🤖 PR changes 500-999 lines, ignoring generated files. and removed size/L 🤖 PR changes 100-499 lines, ignoring generated files. labels Jul 21, 2026

aliok requested a review from Copilot July 21, 2026 14:06

Copilot started reviewing on behalf of aliok July 21, 2026 14:07 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Uh oh!

Conversation

aliok commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How it works

Dependencies

Limitations

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

lkingland left a comment

Choose a reason for hiding this comment

Uh oh!

knative-prow Bot commented Jul 20, 2026

Uh oh!

lkingland commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

knative-prow Bot commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

codecov Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

aliok commented Jul 16, 2026 •

edited

Loading

lkingland commented Jul 21, 2026 •

edited

Loading

codecov Bot commented Jul 21, 2026 •

edited

Loading