Dataset server capabilities¶

Summary¶

Dataset servers can be run on various hardware configurations, and PyCTOH cluster can be highly heterogeneous.

Optimizing DAP server is a matter of carefully choosing which dataset server will be requested.

A capability negociation system permits DAP servers to select dataset servers that will best answer the request.

Capability that can be advertised by dataset servers:

processing power: cores, bogomips, memory, ...
location relative to data: rack-id, network bandwidth, swift-proxy address, ...
CUDA informations: cores, memory, ...
supported checksums for byzantine processes checks (cf Byzantine Dataset servers)