Finding One in a Trillion
HDF5: Maximum I/O. Maximum Storage. Maximum Flexibility.
Flexible technical capabilities — the kind that spur discovery — are possible when powerful I/O is performing for you.
Streams of charged particles called plasma continually boil from the surface of the sun and bombard the magnetic field that surrounds Earth like a protective shield. Most are deflected safely away. But others are pulled in towards Earth’s magnetic poles, and when conditions are right, accelerate downwards along magnetic field lines and collide with atoms and molecules in the upper atmosphere.
The energy emitted in each collision bursts across the polar skies as light in brilliant auroras. The same mechanism causes solar flares and can fracture Earth’s magnetic shield, wreaking havoc on electronics, power grids, and space satellites.
The question the scientists want to answer is why some particles are accelerated to very high energy and others are not. It’s a trillion particle question. To model the process, scientists have to simulate underlying physics on scales that range from the tiny motions of electrons we can’t see to 100 times the radii of the Earth, all in three dimensions.
The first successful simulation of this scale was conducted at the Lawrence Berkeley National Laboratory (LBNL) in 2013 and was powered by HDF5.
“We can’t save all the data for all the particles over the lifetime of the simulation, so we did the next best thing,” says Homa Karimabadi, a physicist at the University of California, San Diego, and one of the lead scientists. “We ran the simulation and stored the particle data at multiple timesteps and then used visualization tools to focus on the time and regions where acceleration was occurring.”
The key, says Karimabadi, was finding the small number of particles that mattered— the one hundred, the thousand, or maybe one million in a trillion.
In total, 10 separate trillion-particle datasets, each ranging between 30 and 42 terabytes in size, were written as HDF5 files at rates reaching 90 percent of maximum and a sustained rate of 27 out of a possible 35 gigabytes per second. Larger simulations are in the works. Learn more.
http://www.mendeley.com/catalog/parallel-io-analysis-visualization-trillion-particle-simulation/