Skip to content

Add smaller fixes to make running from files more robust - #15

Merged
KristianHMoller merged 14 commits into
dmidk:mainfrom
JoachimKoenigslieb:main
Aug 20, 2026
Merged

Add smaller fixes to make running from files more robust#15
KristianHMoller merged 14 commits into
dmidk:mainfrom
JoachimKoenigslieb:main

Conversation

@JoachimKoenigslieb

Copy link
Copy Markdown
Contributor

I wanted to use sunflow to do nowcasting over the US with data from GOES.

I ultimately get the data from this public AWS source here https://registry.opendata.aws/noaa-goes/.

Unfortunately my files are stored converted to grib2, so I'm getting into some quite off-label usage here. Maybe this is not something you guys are interested in supporting up stream here? The specific changes needed to support it are quite general and feels like reasonable changes tough.

Turns out that xarray is happy enough to read my files out of the box if I install cfgrib as a peer-dependency of sunflow. One big issue remains: I'm not storing the clearsky in these grib2 files! Instead I made a small config switch that allows for fetching the clearksy via the already-installed pvlib.

A list of fixes implemented are:

  • Make minutes available for formatting file names
  • use .expand_dims instead of .assign_coords to make sure we have both time dimension AND time coordinates when loading
  • Added a helper make_pvlib_clearsky_dataset which makes clearsky by calling pvlib. Also added a new config variable clearsky_source which defaults to file and switches behavior when set to pvlib.
  • check_solar_elevation now does not take Copenhagen as a default as that can easily lead to silent buggy behavior.

With these changes I'm able to do nowcasts over CONUS:
image

@KristianHMoller
KristianHMoller self-requested a review June 26, 2026 09:01
@KristianHMoller

Copy link
Copy Markdown
Collaborator

Thanks a lot @JoachimKoenigslieb, this is awesome!
For my first quick test of the pvlib clear sky values, it worked perfectly.
One question: Is there a particular reason for your choice of the pvlib.clearsky.simplified_solis values for the clear-sky? I just noticed that the library had multiple options: https://pvlib-python.readthedocs.io/en/stable/reference/clearsky.html . But I can see that some options require additional input.

@KristianHMoller

Copy link
Copy Markdown
Collaborator

In general, I think that support for reading in data from GRIB files would also be nice. Is it correctly understood, that this would require only cfgrib included as a dependency in the pyproject.toml file?

Comment thread sunflow/data_io.py
Comment thread sunflow/main.py Outdated
Comment thread sunflow/forecast.py Outdated
@KristianHMoller

Copy link
Copy Markdown
Collaborator

The command: pre-commit run --all should fix most linting issues and highlight the ones it cannot automatically fix.

Comment thread sunflow/forecast.py Outdated
Comment thread sunflow/forecast.py
Comment thread CHANGELOG.md
@KristianHMoller

Copy link
Copy Markdown
Collaborator

All implemented functionality seems to work as intended! This is great!
I have a few suggestions of which the one regarding maximum solar elevation is scientific, whereas the others are mainly format

@AdamRJensen AdamRJensen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few comments, but my main concern that I want to raise is that climatological clear sky models as used in this PR are not ideal for forecasting applications given that water vapor varies significantly from day to day. But if this is for testing purposes, then it's fine to add.

Using CAMS clear sky data based on forecasting of aerosols is the way forward in my opinion.

Comment thread sunflow/forecast.py
Comment thread CHANGELOG.md Outdated
JoachimKoenigslieb and others added 3 commits July 7, 2026 13:46
Co-authored-by: KristianHMoller <150110122+KristianHMoller@users.noreply.github.com>
Co-authored-by: KristianHMoller <150110122+KristianHMoller@users.noreply.github.com>
Co-authored-by: Adam R. Jensen <39184289+AdamRJensen@users.noreply.github.com>
@JoachimKoenigslieb

Copy link
Copy Markdown
Contributor Author

Have been a bit swamped with other things but have not forgotten about this MR!

Will apply all suggestions and get it ready.

Comment thread sunflow/data_io.py Outdated
Comment thread sunflow/data_io.py Outdated
Comment thread config.yaml Outdated
@KristianHMoller

Copy link
Copy Markdown
Collaborator

Thanks for the update @JoachimKoenigslieb
This will be a great addition to the code.
I have a single suggestion to change seconds to second for consistency.
But apart from that, it should be ready for merge from a content point of view.

However the main sunflow code has changed a bit since this was created, so there were some conflicts. I tried to fix them, but possibly created more issues in that attmpt, at least according to the linter. I am not sure if the easiest is if you revert df286f4 and pull in the updates from the main repo into this PR? Alternatively, we can merge this in and fix the issues after?

Comment thread sunflow/main.py
Comment thread sunflow/main.py Outdated
Comment thread sunflow/main.py Outdated
Comment thread sunflow/main.py Outdated

@KristianHMoller KristianHMoller left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@KristianHMoller
KristianHMoller merged commit 68c3d84 into dmidk:main Aug 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants