The following is an excerpt from an National Renewable Energy Lab (NREL) press release. NREL Releases Major Update to Wind Energy Dataset May 8, 2018 A massive amount of wind data was recently made accessible online, greatly expanding the amount of information available on wind flow across the continental United States. The data from the […]
What is the Highly Scalable Data Service (HSDS)? As many organizations face the challenges of moving their data to the cloud, for many, major code changes and cost increases are top concerns. HSDS is a REST-based solution for reading and writing complex binary data formats within object-based storage environments such as the Cloud. Developed to
When using HDF5 or HSDS you’ve likely benefited (even if you weren’t aware of it) caching features built into the software that can drastically improve performance. HSDS and h5pyd utilize caching to improve performance for service-based applications. In this post, we’ll do a quick review of how HDF5 library caching works and then dive into HSDS and h5pyd caching (with a brief discussion of web caching).
The HDF Group hosted, Loïc Huder of ESRF for a webinar on Vscode-h5web: A VSCode extension to explore and visualize HDF5 files on August 31, 2022. H5Web is a web viewer for HDF5 files. It can be used to browse HDF5 files and display datasets with performant WebGL-based visualisations. While such files can be requested
Highly Scalable Data Service principal architect John Readey covers an update to the Highly Scalable Data Service. The max request size limit per HTTP request no longer applies with the latest HSDS update. In the new version large requests are streamed back to the client as the bytes are fetched from storage. Regardless of the size of the read request, the amount of memory used by the service is limited and clients will start to see bytes coming back while the server is still processing the tail chunks in the selection. The same applies for write operations—the service will fetch some bytes from the connection, update the storage, and fetch more bytes until the entire request is complete. Learn more about this update, plus check out John’s benchmark results using a couple of different MacBook Pros and his new DevOne laptop.
The HDF Group will be hosting a webinar presented by Loïc Huder of ESRF on Wednesday, August 31 at 9:00 a.m. Central US time, 4:00 p.m. Central European Summer Time. H5Web is a web viewer for HDF5 files. It can be used to browse HDF5 files and display datasets with performant WebGL-based visualisations. While such
HSDS (Highly Scalable Data Service) is described as a “containerized” service, but how are these containers organized to create the service?
HSDS (Highly Scalable Data Service) is a REST based service for HDF data, part of HDF Cloud, our set of solutions for cloud deployments. In the recent blog about the latest release of HSDS we discussed many of the new features in the 0.6 release including support for Azure.
Suren Byna, Elena Pourmal, Lori Cooper Overview HDF5 has been a widely used tool to simplify management and access to scientific and engineering data with ubiquitous data solutions. With rapidly growing data across all domains of science and industry, HDF5 developers have been building technologies that provide rapid, easy, and permanent access to complex data.
HSDS (Highly Scalable Data Service) is a REST-based service for reading and writing HDF data. Initially developed as a NASA Access 2015 project, the HDF Group has continued to invest in the project, and as we’ll see, the latest version has a bevy of new and interesting features.
