Skip to content

Commit dc3b662

Browse files
authored
Update Readme
Removed detailed path component descriptions and examples from the README.
1 parent 74e2493 commit dc3b662

File tree

1 file changed

+6
-46
lines changed

1 file changed

+6
-46
lines changed

README.md

Lines changed: 6 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -11,49 +11,11 @@ The repository uses a structured directory path format to organize downloaded, d
1111
{base_path}/{product_type}/{dataset}/{temporal_resolution}/{interpolation}/{variable}/
1212
```
1313

14-
**Path Components:**
15-
- `product_type`: Type of data processing
16-
- `raw`: Data downloaded directly from CDS
17-
- `derived`: Variables calculated from raw data (e.g., wind speed from u/v components) or interpolated to different grids
18-
- `dataset`: CDS dataset name (e.g., `reanalysis-era5-single-levels`, `reanalysis-cerra-single-levels`)
19-
- `temporal_resolution`: Time frequency of the data
20-
- `hourly`: Hourly data (e.g., ERA5 with all 24 hours)
21-
- `daily`: Daily data or daily statistics
22-
- `3hourly`: 3-hourly data (e.g., CERRA)
23-
- `6hourly`: 6-hourly data
24-
- `monthly`: Monthly data
25-
- `interpolation`: Grid specification
26-
- `native`: Original grid (no interpolation)
27-
- `gr006`: 0.0625 degree grid (6.25 km)
28-
- Other grid specifications as needed
29-
- `variable`: Variable name (e.g., `u10`, `v10`, `sfcwind`, `t2m`)
30-
3114
**Examples:**
3215

3316
1. **Raw ERA5 hourly wind components:**
3417
```
3518
/lustre/.../raw/reanalysis-era5-single-levels/hourly/native/u10/
36-
/lustre/.../raw/reanalysis-era5-single-levels/hourly/native/v10/
37-
```
38-
39-
2. **Derived daily wind speed from ERA5 (calculated variable):**
40-
```
41-
/lustre/.../derived/reanalysis-era5-single-levels/daily/native/sfcwind/
42-
```
43-
44-
3. **Raw CERRA 3-hourly data:**
45-
```
46-
/lustre/.../raw/reanalysis-cerra-single-levels/3hourly/native/t2m/
47-
```
48-
49-
4. **Derived CERRA data interpolated to 0.0625 degree grid:**
50-
```
51-
/lustre/.../derived/reanalysis-cerra-single-levels/3hourly/gr006/t2m/
52-
```
53-
54-
5. **ERA5 daily statistics:**
55-
```
56-
/lustre/.../raw/derived-era5-single-levels-daily-statistics/daily/native/t2m/
5719
```
5820

5921
**Note:** Interpolated data is stored under `derived` with the `interpolation` field indicating the target grid (e.g., `gr006`). This distinguishes it from calculated variables which use `interpolation=native`.
@@ -62,7 +24,7 @@ The repository uses a structured directory path format to organize downloaded, d
6224
Format of the files is "{var}\_{dataset}\_{date}.nc"
6325
With date:
6426
- "{year}{month}" for big datasets like CERRA saved month by month (download is faster this way).
65-
- "{year}" for the other datasets that are saved year by year.
27+
- "{year}" for the other datasets the data is saved year by year.
6628

6729
## Creating Directory Structure
6830

@@ -79,20 +41,18 @@ python scripts/create_folder_structure.py
7941
The script reads all CSV files in the `requests/` directory and creates the directory structure according to the format:
8042
`{base_path}/{product_type}/{dataset}/{temporal_resolution}/{interpolation}/{variable}/`
8143

82-
This is useful for:
83-
- Setting up the directory structure before starting downloads
84-
- Verifying the paths that will be used
85-
- Preparing storage locations in advance
8644

8745
## Contents
8846

8947
| Directory | Contents |
9048
| :-------- | :------- |
9149
| [requests](https://github.com/SantanderMetGroup/c3s-cds/tree/main/requests) | Contains one CSV file per CDS catalogue, listing the requested variables, temporal resolution, interpolation method, the target save directory, and whether the variable is raw or requires post-processing to be standardized.
92-
| [scripts](https://github.com/SantanderMetGroup/c3s-cds/tree/main/scripts) | Python scripts to download data from the CDS.
9350
| [provenance](https://github.com/SantanderMetGroup/c3s-cds/tree/main/provenance) | Contains one JSON file per catalogue describing the provenance and definitions of each variable.
94-
| [standardization](https://github.com/SantanderMetGroup/c3s-cds/tree/main/standardization) | Python recipes to standardize the variables.
95-
| [derived](https://github.com/SantanderMetGroup/c3s-cds/tree/main/derived) | Python recipes to calculate derived products from the variables.
51+
| [scripts/download](https://github.com/SantanderMetGroup/c3s-cds/tree/main/scripts/download) | Python scripts to download data from the CDS.
52+
| [scripts/standardization](https://github.com/SantanderMetGroup/c3s-cds/tree/main/scripts/standardization) | Python recipes to standardize the variables.
53+
| [scripts/derived](https://github.com/SantanderMetGroup/c3s-cds/tree/main/scripts/derived) | Python recipes to calculate derived products from the variables.
54+
| [scripts/interpolation](https://github.com/SantanderMetGroup/c3s-cds/tree/main/scripts/interpolation) | Python recipes to interpolate data.
55+
| [scripts/catalogue](https://github.com/SantanderMetGroup/c3s-cds/tree/main/scripts/catalogue) | Python recipes to produce the catalogues of downloaded data.
9656
| [catalogues](https://github.com/SantanderMetGroup/c3s-cds/tree/main/catalogues) | CSV catalogues of datasets consolidated in Lustre or GPFS. The catalogues are updated through a nightly CI job.
9757

9858

0 commit comments

Comments
 (0)