.. CACHE-A documentation master file, created by
   sphinx-quickstart on Mon Jul 15 21:35:50 2024.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to Data Cache Management Service documentation!
=======================================================

.. toctree::
   :maxdepth: 2
   :caption: Contents:


Cache-B
=======

Credentials and tools configuration
-----------------------------------

In order to access the datasets in CacheB cache you need to obtain CacheB credentials and instruct your tools to use them when downloading the data.

With your DestinE platform credentials at hand, download the `cacheb-authentication.py script <https://github.com/SercoSPA/DESP-UserWorkflowService-Templates/blob/main/cacheb/cacheb-authentication.py>`_.

You first need a Python interpreter (>= 3.9) and the installation of some dependencies:

.. code-block:: bash

   pip install requests conflator lxml



Then run the script:

.. code-block:: bash

   python cacheb-authentication.py
   Username: <Enter your DestinE username here>
   Password: <Type your DestinE password here>


The output will be a configuration snippet which follows the `netrc format <https://everything.curl.dev/usingcurl/netrc.html#the-netrc-file-format>`_.

It will be similar to this:

.. code-block:: none

   machine cacheb.dcms.destine.eu
       login anonymous
       password <…a-very-long-string…>



Copy the configuration in the `.netrc` file inside your home directory (`~/.netrc` on Linux/MacOS, on Windows the file must be named `_netrc`). Create the file if doesn't exists yet.

Now you should be able to access CacheB datasets with Xarray / Zarr.

.. note::
   CacheB credentials are valid for a limited period of time and need to be regenerated and reconfigured periodically by running the cacheb-authentication.py script.


Access the Data
---------------

After setting up your CacheB credentials, you can access CacheB datasets with Xarray / Zarr.

Here is a list of available datasets:

.. code-block:: python
   
   datasets = [
    "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-IFS-NEMO-0001-high-sfc-v0.zarr",
    "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-IFS-NEMO-0001-high-o2d-v0.zarr",
    "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-IFS-NEMO-0001-high-pl-v0.zarr",
    "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-ICON-0001-high-sfc-v0.zarr",
    "https://cacheb.dcms.destine.eu/era5/reanalysis-era5-land-no-antartica-v0.zarr",
    "https://cacheb.dcms.destine.eu/era5/reanalysis-era5-single-levels-v0.zarr",
   ]


Each dataset can be reached via its dedicated URL.

You can use the following code snippet to access the data:

.. code-block:: python
   
   # pip install xarray zarr fsspec requests aiohttp dask
   import xarray as xr
   data = xr.open_dataset(
       "https://cacheb.dcms.destine.eu/d1-climate-dt/ScenarioMIP-SSP3-7.0-IFS-NEMO-0001-high-sfc-v0.zarr",
       engine="zarr",
       storage_options={"client_kwargs": {"trust_env": "true"}},
       chunks={},
   )


Quota limits
---------------

**Request Limit**: 500000 requests per user per month

**Authentication**: Required for all data access

**Recommended Approach**: we recommend carefully planning your data retrieval strategy, as downloading entire datasets can quickly consume your quota. Our tutorials provide guidance on how to efficiently access and work with large datasets while managing your request limits effectively.