Repository navigation

Fixes: schema validation errors, bad int records, clean_dataframe args, column renames - #40

Open

emmleroy wants to merge 18 commits into

masterfrom

emmleroy-dev

Open

Fixes: schema validation errors, bad int records, clean_dataframe args, column renames#40
emmleroy wants to merge 18 commits into
masterfrom
emmleroy-dev

Commits on Aug 28, 2026
- 3) clean_dataframe's args should pass through to validate_schema
  emmleroy
  committed
  3a914cf
- 1) print schema validation errors per line
  emmleroy
  committed
  eac9767
- 6) undo adding of opc_ prefix to bin column names to match what's used in the pm models
  emmleroy
  committed
  07649dc
- 2) bad records in int-column should be coerced to NaN
  emmleroy
  committed
  2d92f85
- 5) rename old wind column names
  emmleroy
  committed
  c50c390
- add old sdcard u/v
  emmleroy
  committed
  d134274
Commits on Sep 16, 2026
- recast flag values to in64
  emmleroy
  committed
  f537f94
- drop empty bins
  emmleroy
  committed
  04ea403
Commits on Sep 17, 2026
- remove opc_ prefix from _OPC_COLUMNS
  emmleroy
  committed
  d73e9fb
Commits on Oct 2, 2026
- Set flags to zero before re-flagging
  emmleroy
  committed
  d371403
Commits on Oct 3, 2026
- Make resetting flags optional
  emmleroy
  committed
  0e5987e
Commits on Oct 8, 2026
- Add FLAG_OPC and FLAG_NEPH when both are reading 0.0 (this happens when these flags are set in the firmware but the raw data is kept at 0.0 with the flags carried through --> we can't reflag this i…
  emmleroy
  committed
  5207d84
Commits on Oct 20, 2026
- Relax pandas constraint for compatibility with quantaq-gases
  emmleroy
  committed
  c0761b5
- Relax numpy constraint for compatibility with quantaq-gases
  emmleroy
  committed
  e777d5e
Commits on Oct 23, 2026
- Add schema utils for undoing column name standardization
  emmleroy
  committed
  0c3eab3
Commits on Oct 26, 2026

Navigation Menu

Repository navigation

Fixes: schema validation errors, bad int records, clean_dataframe args, column renames - #40

Fixes: schema validation errors, bad int records, clean_dataframe args, column renames#40
emmleroy wants to merge 18 commits into
masterfrom
emmleroy-dev

Commits on Aug 28, 2026

3) clean_dataframe's args should pass through to validate_schema

1) print schema validation errors per line

6) undo adding of opc_ prefix to bin column names to match what's used in the pm models

2) bad records in int-column should be coerced to NaN

5) rename old wind column names

add old sdcard u/v

Commits on Sep 16, 2026

recast flag values to in64

drop empty bins

Commits on Sep 17, 2026

remove opc_ prefix from _OPC_COLUMNS

Commits on Oct 2, 2026

Set flags to zero before re-flagging

Commits on Oct 3, 2026

Make resetting flags optional

Commits on Oct 8, 2026

Add FLAG_OPC and FLAG_NEPH when both are reading 0.0 (this happens when these flags are set in the firmware but the raw data is kept at 0.0 with the flags carried through --> we can't reflag this i…

Commits on Oct 20, 2026

Relax pandas constraint for compatibility with quantaq-gases

Relax numpy constraint for compatibility with quantaq-gases

Commits on Oct 23, 2026

Add schema utils for undoing column name standardization

Commits on Oct 26, 2026

Count clean flags with flag ==0 or flag ==4096 (overheat)

Fix to make flag-aware resampling work with the completeness_checks (but note on usage: flag-aware resampling of 5s data to 1min should not use completeness checks to match the firmware logic (if a…

Simplify resample.py: only allow completeness_threshold check to work if flag_aware=False (if flag_aware=True, mirror the logic in the firmware)