HDF at the 2015 Oil & Gas High Performance Computing Workshop
Quincey Koziol, The HDF Group
Photo from nasa.gov
Perhaps the original producers of “big data,” the oil & gas (O&G) industry held its eighth annual High-Performance Computing (HPC) workshop in early March. Hosted by Rice University, the workshop brings in attendees from both the HPC and petroleum industries.
Jan Odegard, the workshop organizer, invited me to the workshop to give a tutorial and short update on HDF5.
The workshop (#oghpc) has grown a great deal during the last few years and now has more than 500 people attending, with preliminary attendance numbers for this year’s workshop over 575 people (even in a “down” year for the industry). In fact, Jan’s pushing it to a “conference” next year, saying, “any workshop with more attendees than Congress is really a conference.” But it’s still a small enough crowd and venue that most people know each other well, both on the Oil & Gas and HPC sides.
The workshop program had two main tracks, one on HPC-oriented technologies that support the industry, and one on oil & gas technologies and how they can leverage HPC. The HPC track is interesting, but mostly “practical” and not research-oriented, unlike, for example, the SC technical track. The oil & gas track seems more research-focused, in ways that can enable the industry to be more productive.
I gave an hour and a half tutorial on developing and tuning parallel HDF5 applications, which was nicely received. There were about 40 people in attendance, most of whom were on the “system” side of the HPC centers, not developers. Nevertheless, the audience was nicely interactive and asked many questions, leading to a good session. As usual, many people are interested in performance tuning, and it would be good to have more information on our documentation that helps in this regard.
I also gave a short talk about the status of HDF5 and our plan, which was well received too.
Both talks, as well as walking around the vendor area and poster session generated multiple follow-up conversations with new and old friends from the Australian National Computational Infrastructure, Omnibond, Penguin Computing, R-Systems, Bright Computing, iRODS, and the Fraunhofer HPC center. Many people in this industry are using HDF5 and are interested in staying up on what we are doing.
Overall, for an industry-focused workshop, I was surprised by the collegial atmosphere and how many O&G companies were willing to share details about how their HPC center was built and how it’s run. Take a look at my slides and see how HDF5 can help you too!