Prefer early charging if economics are similar

[andig]

One potential enhancement I see is that the optimizer seems to favor later charging over earlier charging when price differences are very small (for example, below €0.5 ct/kWh). Especially for home batteries, I would often prefer earlier charging in such situations.The reason is that a PV forecast is inherently uncertain. By postponing charging, there is a risk that the battery ends up not fully charged, even though there was an earlier opportunity to do so at nearly the same cost. The consequence can be additional grid imports later, which are typically far more expensive than the small price advantage gained by delaying charging.I realize this is not a simple optimization problem. It involves forecast uncertainty, risk tolerance, and the trade-off between expected savings and the cost of a missed charging opportunity.

The optimal strategy will likely differ between use cases, which may make sense as a user-configurable "risk appetite" setting. Some users may prefer the lowest expected cost, while others may prefer a more conservative strategy that prioritizes charge certainty when cost differences are negligible.

Originally posted by @krispek in evcc-io/evcc#32881 (comment)

[andig]

@ekkea any idea how we would do this? It seems early charging is better than late and would have economic value in this case?

[andig]

Not sure if it fits here but there's also the issue of charging the car sooner than later while home battery could still be filled later. This is preferred since the car might leave and the home battery not be able to absorb remaining energy. Maybe this is like a "risk of avoidable surplus" that might have a value here?

[jeremiahpslewis]

I'd separate two things that the "risk appetite" framing merges. The optimizer currently minimizes cost along a single forecast path, so it evaluates cost at the mean instead of taking the mean of the cost. The optimizer cost is convex in PV output, short of forecast you import, over it you export at a fraction of the price, so a point forecast handed to the optimizer understates the cost of waiting. Even a purely risk-neutral agent, given a distribution rather than a path, would prefer earlier charging at small spreads. Risk aversion is the separate question of paying more on average to avoid very expensive days; a purely rational user should not want this as it means less money in their pocket at the end of the year.

That suggests making optimizer aware of forecast probabilities before one took the further step of factoring in a user’s risk preferences: let the request carry weighted scenarios (forecast 80%, 120% of forecast; passing along historical probability of vehicle availability), solve them jointly with the first interval shared, execute it and replan next cycle. A single scenario at p=1 for forecast accuracy and vehicle ability is today's behaviour.

Happy to sketch the formulation if there's appetite for going that route.

tldr: once you factor in forecast uncertainty, economics will generally favor earlier charging without any hard coding, we should do this...

Aside: backup-capable house batteries have a dual payoff from maintaining a higher SOC: backup service. Separate issue / mechanism, but may motivate the OPs gut sense that earlier higher SOCs are better.

[andig]

The optimizer cost is convex in PV output

It is what?

tldr: once you factor in forecast uncertainty, economics will generally favor earlier charging without any hard coding, we should do this...

This is what's stated above.

Happy to sketch the formulation if there's appetite for going that route.

This is what's asked above:

any idea how we would do this

[krispek]

A relatively simple solution would be to work with a pessimistic scenario for PV. This could be a setting, or (more impactful) derived from historic predictions compared to the actual energy generated. That still doesn’t answer the question on how certain you want to be that the battery is fully charged, and at what price that may be. I think this can be mathematically determined, but not sure how.

[andig]

A relatively simple solution would be to work with a pessimistic scenario for PV

I don’t think so. Optimizer will work with variable tariff just as well, no PV required. This also wouldn’t help with early charging. Maybe we need to separate the two topics.

[krispek]

Let's see if I can separate the several use cases you see:

Given dynamic prices and given that feeding into the grid is most often way less beneficial than taking energy from the grid:

• I want the battery to charge early to prevent late charging because of curtailing;
• I want the car to charge early so that unforeseen absence doesn't have a major influence;
• I want both the car and the battery to charge early to overcome negative differences in the actual solar energy production compared to the forecasted production (timing and/of total production).

Am I right on this?

[jeremiahpslewis]

I think the common thread in all use cases is that assumptions the optimizer makes are not 'right enough' for a series of edge cases. E.g. optimizer assumes that solar forecast is a guarantee, rather than a probabilistic estimate; optimizer has no concept that a car may be disconnected after it's plugged in; the third being the juxtaposition of the prior two.

Proposed solution: use probability weighted scenarios which specify a range of plausible PV yields. Let's say the scenario as-is gets a weight of 80% and the max and min PV yield scenarios are each weighted 10%.

Internally the time-indexed variables are replicated per scenario, all scenarios are constrained to share the decision for the current step, and the objective becomes the probability-weighted sum of the per-scenario objectives.

A first step in this direction would be entirely on the optimizer side, take the solar forecast and generate a 70% and 130% yield scenarios. Then, as a second step, this range could be calculated from the observed error between forecast and actual PV yield. A more elaborate version later on would use the currently discarded forecast bands (max / min PV yield) services like pvnode provide.

The same mechanism extends to charging availability, separately.

I'll post a branch with the concrete changes necessary (plus a technical tweak vis a vis a penalty term) later tonight.

[jeremiahpslewis]

Ok, tried out a few ideas (see linked branch), one thing is clear...adding more probabilistic information (via a crude approach) increases running time without relevant performance gains.

More sophisticated solutions are definitely out there, but my gut sense is that we start eking out additional <2-5% efficiency for a lot of work and complexity. If forecast was so off such that a forecast adjustment changes optimizer behavior, then optimizer would be giving poor results already. But it mostly works, incredibly well at that.

Where I do see space for meaningful gains is car charging availability…at the moment it assumes once car is plugged in, it will stay plugged in, if car unplugged it won’t be plugged back in. But we have charging data that should let us, analogous to the house electrical load heuristics, calculate whether car is likely to be plugged in or unplugged during a given window.

[ScumbagSteve]

Hi, I think this discussion is generally important, as it addresses a main weakness of the optimizer: An imprecise forecast.
By nature there will always be differences to the forecast, the question is how we can control the system in a way, that these differences do not lead to stupid decisions.
I see two things that already cover this (mostly).

#131 which addresses the information if a car will be there later. With this information we can fix that optimizer is planning to charge the car later in a time, when it is not connected anymore.

#162 which let's us define priorities (fill home battery first, car A second, car B third)

And generally we already have the optimizer strategy "fill battery first" which prioritizes battery filling first, before we export to grid.
This however only comes into play if it doesn't matter economically, if I charge battery first or I export to grid first. However it should fix the issue where the forecast is too optimistic, because if I fill the batteries first, the only thing I "lose" is that less energy is exported to grid - which is what we want in this scenario, right?

[jeremiahpslewis]

Three thoughts:

1. We are going to a lot of effort to create an 'automatic' mode, and yet we are now introducing a somewhat extensive manual configuration, car presence / absence, which may a) may need to be manually adjusted over time; b) for which we already have decent historical data for inferring the same thing.
2. Forecasts are by their nature imprecise and many forecasts output a range of likely outcomes. The optimizer should on average deliver the best outcome across this range (assuming the forecast is imprecise but on average calibrated). At the moment we 'throw away' information we could be using to improve average performance.
3. It does matter economically exactly when we charge the battery. Depending on your feed-in rules, you may be constrained in max feed-in power (kW), the battery timing ensures electricity isn't thrown away.

[andig]

Hi, I think this discussion is generally important, as it addresses a main weakness of the optimizer: An imprecise forecast.

And: uncertainty about the vehicle remaining connected. The home battery always is.

I see two things that already cover this (mostly).

#131 which addresses the information if a car will be there later. With this information we can fix that optimizer is planning to charge the car later in a time, when it is not connected anymore.

It doesn't imho. It only addresses the "will for sure not be there" case.

#162 which let's us define priorities (fill home battery first, car A second, car B third)

Also think that's not the right thing to do. The whole point of the optimizer is automatic economics. Putting vehicle before home battery on a fixed priority is imho contradictory.

None of these fix this issue in a satisfying way.

[krispek]

This issue seems more complicated than I thought. :)

Today I had an issue where the battery could have been charged earlier in a more economic way. But I agree with @jeremiahpslewis the performance wouldn't be much better. However, this will change when the spread between importing and exporting energy increases (currently €2,5ct).

I would argue that an active charging plan is a pretty good indication that a vehicle will be charged. However, I'm not sure how the optimizer should deal with this information. For example: I 'plan' to have a sufficient SoC just before I leave, but I expect that the car charges way earlier. Should the optimizer just calculate with a late charge, or with the most economical charge (which is way earlier)? Presence schedules would help, but I would like to have an option to (semi)automate this, f.e. via a calendar or being able to set a time when the car is home via an API.