Repository navigation
explain: open with the decision, describe the optimizer model and evcc, 24h prompt cache - #250
Merged
Merged
Conversation
…filler Six of nine thumbs down since 2026-10-16 complain that the explanation states the obvious (fixed price, no solar at night) and tells what happens instead of why. Prompt hash cfa3bedab0b5
Prompt hash 1ebddfdbbd10
Prompt hash 02143c20719b
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six of the nine thumbs down since 2026-10-16 complain that the explanation states the obvious and tells what happens instead of why: "stating the static prices/fees is useless", "Explains what happens but not why", "telling me the obvious", "Anfang unnötiger Füllinhalt", and two mocking the "no solar at night" opener. Three of them are on the prompt from #246, so the softened wording there was not enough.
Three commits:
prompt_cache_retention: "24h". October billing showed only cache write meters and no cache reads: at a handful of explanations a day the in-memory cache expires between calls. With 24h retention the second of two test calls came back with 1106 cached tokens, same pricing for both policies per the Azure docs.optimizer.pyand evcc'ssite_optimizer.go: the objective (import cost minus export revenue minus end value), the slot balance, losses and the 25% rule, what stored energy is worth, the end of plan valuation, export, limits and penalties, goals, must charge sources, minimum charge power, grid charging and discharging gates, ties and preferences.Prompt hash
02143c20719b, about 2.5k tokens for the system prompt against 7 to 14k for a digest.Does the knowledge improve the "why"? A/B on the 24 unique plans behind the stored votes. Prompt variants built from this branch by swapping sections (
base= the old short "How the plan is made" without "How evcc works",evcc,model,both), answers by the production model and two larger ones, graded twice on one criterion: how well the answer explains why the plan looks the way it does, checked decision by decision against the plan data and the model rules (1 to 5). First bygpt-6.1-solas a judge, then by me, blind: the eight answers per plan shuffled, the key read only after all 192 scores were written.Head to head per plan (blind): astra both beats astra base on 12 plans, loses 1, ties 11. The model section carries most of it (model vs base 8 better, 2 worse), the evcc section alone little (5 better, 3 worse), and on top of the model section it still adds something (both vs model 5 better, 1 worse). Sol is on the level of astra with the model section and below astra with both (1 better, 6 worse). Astra with both beats luna with both on 22 of 24 plans, never loses.
On
gpt-6-lunathe knowledge changes nothing. It keeps inventing plausible defaults ("the battery has priority", "little solar", "the night price is not low enough") instead of checking whether the stored energy already covers the expensive hours, whether the charge power limit forces export before the battery is full, or that a constant price makes grid charging lose through the losses. What the larger models get right with the knowledge and luna never does: the losses argument against grid charging, export equals import price as the reason for selling at the evening peak, the car charging while the battery still has room because the later weaker sun cannot meet its minimum power, the peak preference as the reason for a flat export plateau, a battery listed as "never discharges" as the reason for a held battery. The remaining gaps even for astra: why the car charges before the battery when both end with leftover (a tie), and why a battery keeps charge instead of selling into a high export price (leftover valued near the cheapest import price).Astra answered in 4.8 s median with 171 completion tokens and no reasoning tokens, sol in 6.5 s with 189, both within the 150 words and without dashes. Cost (Global Standard, USD per 1M tokens, input / cache write / cached read / output): luna 0.12 / 0.15 / 0.012 / 0.60, sol 2 / 2.5 / 0.1 / 10, astra 10 / 12.5 / 1 / 50. With the 24h cached prefix an explanation is about 0.0015 USD on luna, 0.021 on sol and 0.105 on astra; at October's 131 explanations 0.19, 2.8 and 14 USD a month. Sol gets about 90 % of astra's gain for a fifth of the price, so the default deployment moves to
gpt-6.1-solin #251. One sol answer in the first run hit a 180 s read timeout, the service timeout stays at 45 s so such a request fails with 502 instead of hanging.Replay of the six "obvious" thumbs down on luna: every answer now opens with the decision and a reason. The two "why is the battery not grid-charged" cases no longer blame missing solar.
🤖 Generated with Claude Code