Release of HDF5 1.13.3 (Newsletter #188)
We are very pleased to announce the release of HDF5 1.13.3, which can now be obtained from the HDF5 Download page.
The HDF Group appoints Dana Robinson as Director of Engineering
Dana Robinson has been appointed as the new Director of Engineering at The HDF Group. Dana started at The HDF Group in 2009 as a software engineer until stepping into the role of interim Director of Engineering in April 2022. As the Director of Engineering, Dana will lead the team of software engineers and shape the
Improve HDF5 performance using caching
When using HDF5 or HSDS you’ve likely benefited (even if you weren’t aware of it) caching features built into the software that can drastically improve performance. HSDS and h5pyd utilize caching to improve performance for service-based applications. In this post, we’ll do a quick review of how HDF5 library caching works and then dive into HSDS and h5pyd caching (with a brief discussion of web caching).
Webinar Announcement: Tablite: 9BN rows/sec + HDF5 Support for all Python Datatypes
Tablite is an open source project which can be used for incremental data processing. Tablite uses HDF5 as a backend with strong abstraction, so that copy/append/repetition of data is handled in pages (this allows us to slice 9,000,000,000 rows in less than a second on localhost.
The HDF Group appoints Neil Fortner as Chief HDF5 Software Architect
We are excited to announce the appointment of Neil Fortner as the new Chief HDF5 Software Architect. Neil has worked for The HDF Group as a software engineer since 2008. While at The HDF Group, he focused his talents in storage and HPC on improving performance, expanding the features, and improving the maintainability of the
Webinar Follow-up: Subfiling and Multiple dataset APIs: An introduction to two new features in HDF5 version 1.14
On September 30, Neil Fortner, Jordan Henderson, and Scot Breitenfeld of The HDF Group presented Subfiling and Multiple dataset APIs: An introduction to two new features in HDF5 version 1.14.
New Employee Announcement: Luke Logan
Luke Logan has joined The HDF Group as a software engineer and ambassador of the Scalable Computing Systems Lab (SCS).
Webinar Announcement: Subfiling and Multiple dataset APIs: An introduction to two new features in HDF5 version 1.14
On September 30th, The HDF Group will present a webinar on subfiling and HDF5 multiple dataset APIs.
Webinar Followup: Vscode-h5web: A VSCode extension to explore and visualize HDF5 files
The HDF Group hosted, Loïc Huder of ESRF for a webinar on Vscode-h5web: A VSCode extension to explore and visualize HDF5 files on August 31, 2022. H5Web is a web viewer for HDF5 files. It can be used to browse HDF5 files and display datasets with performant WebGL-based visualisations. While such files can be requested
HSDS Streaming
Highly Scalable Data Service principal architect John Readey covers an update to the Highly Scalable Data Service. The max request size limit per HTTP request no longer applies with the latest HSDS update. In the new version large requests are streamed back to the client as the bytes are fetched from storage. Regardless of the size of the read request, the amount of memory used by the service is limited and clients will start to see bytes coming back while the server is still processing the tail chunks in the selection. The same applies for write operations—the service will fetch some bytes from the connection, update the storage, and fetch more bytes until the entire request is complete. Learn more about this update, plus check out John’s benchmark results using a couple of different MacBook Pros and his new DevOne laptop.