feat(recipes): b2b_saas_ltv_v1 recipe assets + e2e [LTV-Po.2b]#132
Open
shaypal5 wants to merge 2 commits into
Open
feat(recipes): b2b_saas_ltv_v1 recipe assets + e2e [LTV-Po.2b]#132shaypal5 wants to merge 2 commits into
shaypal5 wants to merge 2 commits into
Conversation
Add the three recipe assets for the lifecycle / pLTV scheme and resolve
difficulty per tier, completing LTV-M6 (the first end-to-end
b2b_saas_ltv_v1 bundle from the public Generator API).
Recipe assets (leadforge/recipes/b2b_saas_ltv_v1/):
- recipe.yaml: scheme: lifecycle, primary_task pltv_revenue_365d,
default_population {n_customers: 1500}, both exposure modes, all tiers.
- narrative.yaml: subscription-LTV story; market declares 4 icp_industries
+ 3 geographies so the public industry/region columns keep variance
(student_public invariant #6).
- difficulty_profiles.yaml: intro/intermediate/advanced distortion knobs.
The registry auto-discovers the dir; no manual registration needed.
Difficulty resolution:
- LifecycleScheme._resolve_difficulty mirrors lead-scoring (minus the
lead-scoring-only category_latent_correlations): reads the active profile
and attaches DifficultyParams to config. build_world calls it so the
resolved params ride on spec.config and the snapshot builders apply
per-tier distortions. Simulation-level scaling stays deferred (issue #129)
— the world is still tier-independent.
Tests:
- tests/recipes/test_b2b_saas_ltv_v1.py: discovery, asset shape, config
resolution (n_customers), build_world round-trip, narrative-driven
firmographics, determinism, both-mode bundle round-trip + byte-identity,
public industry/region variance.
- Flipped the two tracked-gap guards: test_difficulty_not_yet_differentiating
-> test_difficulty_resolves_params_but_world_unchanged; the explicit-param
test_difficulty_params_thread_into_snapshots -> tier-based
test_difficulty_tiers_produce_different_task_features.
Roadmap + .agent-plan.md mark Po.2b done and LTV-M6 complete.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
Adds the first end-to-end, public-API round-trip for the new lifecycle scheme via the b2b_saas_ltv_v1 recipe, including recipe assets, difficulty-profile resolution into snapshot distortions, and comprehensive e2e tests across exposure modes and tiers.
Changes:
- Added
leadforge/recipes/b2b_saas_ltv_v1/assets (recipe.yaml,narrative.yaml,difficulty_profiles.yaml) for lifecycle/pLTV generation. - Implemented lifecycle difficulty resolution in
LifecycleScheme.build_world()to attachDifficultyParamsfrom the recipe profile for downstream snapshot distortion. - Added/updated tests to validate recipe discovery, config resolution, determinism, bundle round-trips, and difficulty-tier distortion behavior.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/schemes/lifecycle/test_write_bundle.py | Updates difficulty-threading test to assert tier-based distortions (intro vs advanced). |
| tests/schemes/lifecycle/test_build_world.py | Updates tracked-gap test: difficulty params now resolve; simulated world remains tier-independent. |
| tests/recipes/test_b2b_saas_ltv_v1.py | New e2e recipe tests: discovery, config resolution, determinism, both exposure modes, and snapshot invariants. |
| leadforge/schemes/lifecycle/init.py | Adds difficulty-profile resolution into DifficultyParams for lifecycle bundle writing/snapshots. |
| leadforge/recipes/b2b_saas_ltv_v1/recipe.yaml | New lifecycle recipe definition and defaults. |
| leadforge/recipes/b2b_saas_ltv_v1/narrative.yaml | New narrative with multi-valued industry/geo vocab for public variance invariant. |
| leadforge/recipes/b2b_saas_ltv_v1/difficulty_profiles.yaml | New per-tier distortion knobs (noise/missing/outliers) plus carried forward params. |
| leadforge/recipes/b2b_saas_ltv_v1/init.py | Adds package marker for recipe directory. |
| docs/ltv/roadmap.md | Marks LTV-M6 complete and records Po.2b deliverables. |
| .agent-plan.md | Updates agent plan to reflect Po.2b completion and next milestone. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+143
to
+150
| try: | ||
| raw = load_recipe(config.recipe_id) | ||
| recipe = Recipe.from_dict(raw) | ||
| profiles = recipe.load_difficulty_profiles() | ||
| except (FileNotFoundError, KeyError): | ||
| return config | ||
| if not profiles: | ||
| return config |
Comment on lines
+172
to
+181
| cr_range = profile["conversion_rate_range"] | ||
| difficulty_params = DifficultyParams( | ||
| signal_strength=profile["signal_strength"], | ||
| noise_scale=profile["noise_scale"], | ||
| missing_rate=profile["missing_rate"], | ||
| outlier_rate=profile["outlier_rate"], | ||
| conversion_rate_lo=cr_range[0], | ||
| conversion_rate_hi=cr_range[1], | ||
| committee_friction=profile["committee_friction"], | ||
| ) |
Comment on lines
+184
to
+187
| for col in ("industry", "region"): | ||
| if col in train.columns: | ||
| assert train[col].nunique(dropna=True) >= 2, f"{col} is zero-variance" | ||
|
|
….2b] Self-review follow-ups on the b2b_saas_ltv_v1 recipe PR. - Dedupe difficulty resolution: extract leadforge/core/difficulty.py resolve_difficulty_params(), the single home for recipe-load -> profile lookup -> validation -> DifficultyParams. Both schemes delegate: the lead-scoring _resolve_difficulty is now a thin wrapper that also reads its category_latent_correlations extra; lifecycle build_world calls the helper inline (its copied ~45-line method is deleted). This removes the fork where the two "mirror" resolvers had already diverged on the no-profiles policy (lead-scoring raised, lifecycle returned) — now unified: a loadable recipe with no difficulty_profiles.yaml -> (None, None); an unknown recipe_id or a malformed profile propagates InvalidRecipeError. Lead-scoring bundles remain byte-identical vs main (both modes, verified via worktree SHA-256). - Stop the recipe.yaml comments from lying: horizon_days and primary_task are Recipe-schema-required but NOT consumed by the lifecycle scheme (windows come from the locked forward_windows_days; task set is fixed). Comments now say so. - difficulty_profiles.yaml header flags which knobs are ACTIVE (noise/missing/ outlier) vs INERT-until-#129 (signal_strength/committee_friction/ conversion_rate_range) so the placeholder values aren't read as observed rates. - Test hygiene: assert industry/region are PRESENT before asserting variance (was `if col in columns` -> vacuous pass on drop); drop the redundant build-level determinism test (byte-determinism + lifecycle suites cover it); add tests/core/test_difficulty.py pinning the shared helper's contract. - Roadmap: the "leadforge generate --recipe b2b_saas_ltv_v1" claim is now marked verified, with the deferred caveats (no --n-customers flag yet; validate/inspect still lead-scoring-coupled). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
pr-agent-context report: This run includes unresolved review comments on PR #132 in repository https://github.com/leadforge-dev/leadforge
For each unresolved review comment, recommend one of: resolve as irrelevant, accept and implement
the recommended solution, open a separate issue and resolve as out-of-scope for this PR, accept and
implement a different solution, or resolve as already treated by the code.
After I reply with my decision per item, implement the accepted actions, resolve the corresponding
PR comments, and push all of these changes in a single commit.
# Copilot Comments
## COPILOT-1
Location: leadforge/schemes/lifecycle/__init__.py
URL: https://github.com/leadforge-dev/leadforge/pull/132#discussion_r3445500919
Status: outdated
Root author: copilot-pull-request-reviewer
Comment:
LifecycleScheme._resolve_difficulty will apply difficulty profiles for whatever recipe_id is on the config, even if that recipe is for a different scheme (GenerationConfig defaults to the lead_scoring recipe). This can unintentionally attach lead-scoring difficulty_params to lifecycle runs when build_world is called with an ad-hoc/default config, affecting downstream snapshot distortions. Guard by ensuring the loaded recipe's scheme matches lifecycle before applying profiles.
## COPILOT-2
Location: leadforge/schemes/lifecycle/__init__.py
URL: https://github.com/leadforge-dev/leadforge/pull/132#discussion_r3445500922
Status: outdated
Root author: copilot-pull-request-reviewer
Comment:
conversion_rate_range is assumed to be a 2-item sequence; if the profile YAML is malformed (wrong type/length), this will raise an unhelpful IndexError. Validate the shape/type and raise InvalidRecipeError with a clear message instead.
## COPILOT-3
Location: tests/recipes/test_b2b_saas_ltv_v1.py:184
URL: https://github.com/leadforge-dev/leadforge/pull/132#discussion_r3445500927
Status: outdated
Root author: copilot-pull-request-reviewer
Comment:
This test currently skips the assertion if the expected firmographic columns are missing, which would let a regression (dropping industry/region from the public snapshot) pass silently. Since the invariant is about these columns in the public snapshot, assert that the columns exist and then assert variance.Run metadata: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LTV-Po.2b —
b2b_saas_ltv_v1recipe assets + end-to-end round-tripThe finish line for LTV-M6: the first point where
Generator.from_recipe("b2b_saas_ltv_v1").generate()produces a completelifecycle / pLTV bundle from the public API, in both exposure modes.
What this adds
Recipe assets —
leadforge/recipes/b2b_saas_ltv_v1/:recipe.yaml—scheme: lifecycle,primary_task: pltv_revenue_365d,default_population: {n_customers: 1500},horizon_days: 730, both exposuremodes, all three difficulty tiers.
narrative.yaml— a subscription revenue-lifecycle story.marketdeclares4
icp_industries+ 3geographiesso the publicindustry/regionsnapshot columns keep ≥2 distinct values (student_public invariant feat: carry primary_task and label_window_days into WorldSpec for dataset card rendering #6 — the
constraint flagged in the Po.1 review).
difficulty_profiles.yaml—intro/intermediate/advancedwith thedistortion knobs (
noise_scale/missing_rate/outlier_rate) the snapshotbuilders consume;
signal_strength/committee_friction/conversion_rate_rangecarried for forward-compatible sim-level scaling (issue lifecycle: simulation-level difficulty scaling (advanced = genuinely harder world) #129).
The recipe registry auto-discovers the directory — no manual registration.
Difficulty resolution —
LifecycleScheme._resolve_difficultymirrorsLeadScoringScheme._resolve_difficulty(minus the lead-scoring-onlycategory_latent_correlations): it reads the active profile and attachesDifficultyParamsto the config.build_worldnow calls it, so the resolvedparams ride on the returned
spec.configand the snapshot builders applyper-tier distortions. The simulation itself stays tier-independent — every
tier yields the same underlying world; simulation-level scaling remains deferred
(issue #129).
Tests
tests/recipes/test_b2b_saas_ltv_v1.py(new) — discovery, asset shape, configresolution (
n_customersfromdefault_population),build_worldround-trip,narrative-driven firmographics, determinism, both-mode bundle round-trip +
full-bundle byte-identity, and public
industry/regionvariance.test_difficulty_not_yet_differentiating→test_difficulty_resolves_params_but_world_unchanged(params differ per tier; the world stays identical — lifecycle: simulation-level difficulty scaling (advanced = genuinely harder world) #129 still open).
test_difficulty_params_thread_into_snapshots→tier-based
test_difficulty_tiers_produce_different_task_features(since_resolve_difficultyalways overwritesdifficulty_paramsfrom the profile,an explicitly-passed one would be clobbered).
Carried limitations / constraints
early_tenure_weeks/observation_datestay override-only (carried fromPo.2a): the
Recipeschema has no field for them, sorecipe.yamldoes notdeclare them; the bundle uses the
GenerationConfigdefaults (4 weeks;observation_datederived by the population builder).is omitted from
student_public(4 task dirs public / 8 instructor).Gates
pytest: 1899 passed, 51 skipped (full suite).ruff check .clean ·ruff format --checkclean ·mypy leadforge/clean(114 source files).
its determinism/byte-identity tests pass unchanged.
Closes LTV-M6. Next: LTV-M7 (
LTV-Pp— scheme-awarevalidate_bundle).🤖 Generated with Claude Code