# Simulation Inputs Guide

This guide covers the data-preparation stages that bridge the GRASS `r.sun` outputs and the simulation models: rooftop parquet artifact generation, horizon prerequisites, and weather data preparation.

## Contents

- [Pipeline map](#pipeline-map)
- [Parquet pipeline endpoints](#parquet-pipeline-endpoints)
- [Weather data](#weather-data)
- [Output layout](#output-layout)
- [Common problems](#common-problems)

---

## Pipeline Map

The following diagram summarizes how the simulation-input artifacts are derived from the `r.sun` outputs and then passed forward to the photovoltaic simulation stages.

```mermaid
flowchart LR
    A[r.sun output] --> B[Points static parquet]
    B --> C[Suitable areas parquet]
    C --> D[Area means parquet]
    H[Horizon rasters] --> F[Fast horizon PV]
    C --> F
    E[Weather fetch / upload] --> F
    D --> G[Real-sky batch / co-sim]
    C --> G
    E --> G
```

*Figure 1: Simulation-input paths used by the implemented simulation branches. Real-sky batch and co-simulation depend on area means, while the fast horizon-based branch depends on suitable areas, existing horizon outputs, and weather input.*

---

## Parquet Pipeline Endpoints

All endpoints are available under `/solar`.

- `POST /solar/points-static-parquet`
- `POST /solar/suitable-areas-parquet`
- `POST /solar/area-clearsky-means-parquet`

### Required Order

1. `points-static-parquet`
2. `suitable-areas-parquet`
3. `area-clearsky-means-parquet`

### 1. Points-Static Parquet

Use:

```text
POST /solar/points-static-parquet
```

Purpose:

- generate rooftop point-level parquet artifacts
- create static geometry-derived attributes used later in aggregation

This endpoint expects the terrain and radiation prerequisites from earlier stages to already exist.

### 2. Suitable Areas Parquet

Use:

```text
POST /solar/suitable-areas-parquet
```

Purpose:

- create the `suitable_areas.parquet` dataset

This stage supports two main paths.

#### Path 1. The user already has a suitable-areas polygon file

Provide:

- `suitable_areas_file`

This uploaded path itself has two common cases.

##### 1.1. The uploaded file already contains the needed attributes

If the uploaded polygons already contain fields that can serve as:

- suitable-area identifiers
- effective area values

then the backend can reuse them by being told which columns to use:

- `suitable_area_id_column`
- `effective_area_column`

The backend still joins each suitable polygon back to the building-footprint layer so that the final artifact also contains `building_id`.

##### 1.2. The uploaded file does not contain all needed attributes

If the uploaded suitable-area polygons do not already contain all required output attributes, the backend fills in the missing pieces where possible.

#### Path 2. The user does not have a suitable-areas file

Leave `suitable_areas_file` empty and let the backend derive suitable polygons from earlier workflow outputs.

In this mode, the system computes suitable areas from the existing terrain-derived rasters and applies the requested suitability constraints.

Relevant inputs for this derived mode are:

- `building_id_column_in_footprints`
- `slope_min`
- `slope_max`
- `aspect_min`
- `aspect_max`

#### Identifier and Area Normalization

- `building_id` is assigned by spatially joining suitable polygons back to the building-footprint layer.
- `building_id_column_in_footprints` tells the backend which footprint column should be used as the stable building identifier during that join. If it is omitted, the workflow falls back to the GRASS `cat` value.
- `suitable_area_id` can be reused from an uploaded attribute column through `suitable_area_id_column`; otherwise, the backend generates it automatically.
- `effective_area_m2` can be reused from an uploaded attribute column through `effective_area_column`. If no such column is provided, the backend tries to derive it from the available polygon attributes.

#### Derived-Mode Filters

- `slope_min` and `slope_max` define the allowed slope range for suitable-area selection in the derived mode.
- `aspect_min` and `aspect_max` define the allowed aspect range for suitable-area selection in the derived mode. These thresholds use the GRASS-derived aspect convention stored in the terrain raster. When `aspect_mean` is later used by the fast horizon PV branch as pvlib `surface_azimuth`, it is converted with `pvlib_azimuth = (90 - grass_angle) % 360`.
- These slope/aspect thresholds are relevant when the backend computes suitable areas itself. They are not the main control path when the user uploads an already prepared suitable-areas polygon dataset.

In the derived mode, the filtering logic can be summarized as:

$$
\mathrm{slopeMask} =
\begin{cases}
1, & \mathrm{slope}_{\min} \leq \mathrm{slope} \leq \mathrm{slope}_{\max} \\
\text{null}, & \text{otherwise}
\end{cases}
$$

$$
\mathrm{aspectMask} =
\begin{cases}
1, & \mathrm{aspect}_{\min} \leq \mathrm{aspect} \leq \mathrm{aspect}_{\max} \\
\text{null}, & \text{otherwise}
\end{cases}
$$

If a roof-mask raster is available, the suitable-area raster is built as:

$$
\text{suitable} =
\begin{cases}
1, & \mathrm{roofMask}=1 \land \mathrm{slopeMask}=1 \land \mathrm{aspectMask}=1 \\
\text{null}, & \text{otherwise}
\end{cases}
$$

Otherwise, the roof-mask condition is omitted and only the slope/aspect masks are combined.

After vectorization, the effective suitable area is computed from the counted raster cells:

$$
\mathrm{effectiveArea}_{m^2} = \mathrm{slpNumber} \times \mathrm{cellArea}_{m^2}
$$

In other words:

- uploaded mode starts from user-supplied suitable polygons
- derived mode starts from earlier workflow outputs and calculates suitable polygons inside the backend

This stage is important because many later workflows depend on `suitable_area_id`.

### 3. Area Clear-Sky Means Parquet

Use:

```text
POST /solar/area-clearsky-means-parquet
```

Purpose:

- aggregate point-level clear-sky outputs into area-level parquet results

This stage is a direct prerequisite for the real-sky batch and co-simulation models described in [simulation_models.md](./simulation_models.md). The fast horizon-based branch does not use `area_clearsky_means`.

---

## Weather Data

### Weather Workflow Map

The weather stage can either fetch and persist a canonical weather parquet artifact or accept a user-provided weather file at simulation time.

```mermaid
flowchart LR
    A[Open-Meteo fetch] --> B[Weather parquet]
    C[User weather file] --> D[Simulation models]
    B --> D
```

*Figure 2: Weather-input paths used by the simulation workflows: persisted Open-Meteo artifacts and direct weather-file upload during simulation.*

### Two Main Paths

#### 1. Fetch Weather from Open-Meteo

Use:

```text
POST /solar/weather/open-meteo
```

Main inputs:

- `start_date`
- `end_date`
- `timezone`
- optional `latitude`
- optional `longitude`

If latitude and longitude are omitted, the backend tries to resolve them from suitable areas.

Output:

- a persisted weather parquet artifact
- metadata such as row count and query source

#### 2. Upload a Weather File During Real-Sky Batch

Use:

```text
POST /solar/real-sky-batch
```

with a supported weather file.

The uploaded file can be:

- a file prepared by the user
- or a parquet file previously generated by the backend through the Open-Meteo workflow

Supported file types:

- `.csv`
- `.parquet`
- `.pq`
- `.epw`

The endpoint also requires `weather_columns_json`, which maps your file columns to the names expected by the backend.

Required logical fields:

- `dni`
- `dhi`
- `temp_air`
- `wind_speed`
- `time_col_real`

Example mapping:

```json
{
  "dni": "DNI",
  "dhi": "DHI",
  "temp_air": "Tair",
  "wind_speed": "Wspd",
  "time_col_real": "time"
}
```

The same uploaded-weather pattern is also used by:

```text
POST /solar/fast-horizon-pv-simulation
```

For the fast horizon-based branch, weather can be resolved in two ways:

- uploaded weather file plus `weather_columns_json`
- existing weather artifact or previously staged weather dataset resolved from workflow context when no upload is provided

### Timestamps

Weather timestamps can be handled in two ways:

- let the backend auto-detect them
- provide `timestamp_format` explicitly

Examples of explicit formats:

```text
%Y-%m-%d %H:%M:%S
%Y-%m-%d %H:%M
%d/%m/%Y %H:%M
%m/%d/%Y %H:%M
%Y%m%d:%H%M
```

For naive timestamps, the backend also needs a timezone such as:

- `UTC`
- `Europe/Rome`

---

## Output Layout

The simulation-inputs pipeline writes its canonical host-side outputs under `FILE_STORAGE_ROOT`.

Canonical output layout:

```text
FILE_STORAGE_ROOT/
  points_clearsky/
    <location>/
      <mapset>/
        points_static.parquet
        points_clearsky_timeseries.parquet/

  suitable_areas/
    <location>/
      <mapset>/
        suitable_areas.parquet

  area_clearsky_means/
    <location>/
      <mapset>/
        _cache/
          point_to_area_mapping.parquet
        suitable_area_id-<id>/
          part-000001.parquet

  weather_openmeteo/
    <location>/
      <mapset>/
        openmeteo_<start>_<end>_<freq>.parquet

  fast_horizon_pv_results/
    <location>/
      <mapset>/
        suitable_area_id-<id>.parquet
```

Meaning of these paths:

- `points_clearsky/<location>/<mapset>/points_static.parquet`
  point-level static rooftop artifact derived from terrain, slope/aspect, and horizon context
- `points_clearsky/<location>/<mapset>/points_clearsky_timeseries.parquet/`
  partitioned point-level clear-sky timeseries dataset generated from the `r.sun` outputs
- `suitable_areas/<location>/<mapset>/suitable_areas.parquet`
  canonical suitable-areas polygon artifact used by later batch and co-simulation steps
- `area_clearsky_means/<location>/<mapset>/_cache/point_to_area_mapping.parquet`
  cached point-to-area mapping reused by the area-aggregation workflow
- `area_clearsky_means/<location>/<mapset>/suitable_area_id-<id>/part-000001.parquet`
  finalized clear-sky means output for one suitable-area id
- `weather_openmeteo/<location>/<mapset>/openmeteo_<start>_<end>_<freq>.parquet`
  persisted weather artifact created by the Open-Meteo fetch path
- `fast_horizon_pv_results/<location>/<mapset>/suitable_area_id-<id>.parquet`
  final per-area timeseries output written by the fast horizon-based PV simulation branch

Artifact convention note:

- `points_static.horizon_angles` stores horizon values in the original GRASS `r.horizon` directional order. This field is not a pvlib/PVGIS azimuth profile unless the corresponding directions are converted with `pvlib_azimuth = (90 - grass_angle) % 360`.
- Fast horizon PV output columns such as `solar_azimuth` and `horizon_elevation` are produced after the local horizon profile has been converted to the pvlib/PVGIS azimuth convention.

Temporary host-side staging paths are also used during processing:

```text
WORKSPACE_TMP_HOST_ROOT/
  session_<session_id>/
    points_static.csv
    timeseries/
    suitable_areas/
      suitable_areas_tmp.geojson
```

These temporary paths are intermediate working files. The canonical artifacts for downstream workflows are the files and directories written under `FILE_STORAGE_ROOT`.

---

## Common Problems

### `suitable_areas_not_found`

Cause:

- the suitable areas stage was not completed in the same session

### `area_clearsky_means_not_found`

Cause:

- the area clear-sky means stage was not completed in the same session

### `unsupported_weather_file_type`

Cause:

- the uploaded file was not one of the supported extensions

### `invalid_weather_columns_json`

Cause:

- the mapping string was not valid JSON or not a JSON object

### `weather_columns_missing`

Cause:

- one or more required logical fields were missing from the mapping

### Missing Prerequisite Datasets

Cause:

- a later stage was run before an earlier stage produced the required inputs

For the fast horizon-based simulation branch, the most important prerequisites are:

- suitable areas
- horizon-analysis outputs
- weather input, either uploaded or resolvable from workflow context
