Interestingly enough, in addition to being known as the place to go for BBQ and live music, Austin, Texas is a major hub of Python development. Each year, Austin is host to the annual confab of Python developers known as the SciPy Conference. Enthought, a local Python-based company, was the major sponsor of the conference and did a great job of organizing the event. By the way, Enthought is active in Python-based training, and I thought the tutorial sessions I attended were very well done. If you would like to get some expert training on various aspects of Python, check outtheir offerings.
As a first-time conference attendee, I found attending the talks and tutorials very informative and entertaining. The conference’s focus is the set of packages that form the core of the SciPy ecosystem (SciPy, iPython, NumPy, Pandas, Matplotlib, and SymPy) and the ever-increasing number of specialized packages around this core. Continue reading →
I first heard of HDF during the “Data Format Wars” of the 1990’s. These “battles” centered on the selection of a format for the emerging NASA Earth Observing System archives, and there were a number of contenders. HDF won that battle in the end because of the inherent flexibility of the format and the tools for reading and writing it.
Now, twenty years later, HDF has emerged as the foundation format for an incredibly diverse and growing selection of scientific and commercial disciplines.
Is it the inherent flexibility of the format that has led to this success? Maybe, but I would pick information integration as the killer HDF feature. Continue reading →
“A strong foundation is being built for sharing data and information to create community knowledge and wisdom. This foundation includes HDF5 as the data layer with community conventions and ISO metadata facilitating use and understanding.”
We have experienced so many monumental technological shifts during the last several decades that, like the diurnal cycle of light and dark, the technology life cycle (shown below) is becoming instinctual.
It starts with a new idea, (usually aimed at new customers), that destroys existing organizational expertise and threatens the continued existence of established processes and organizations. These disruptions raise a variety of difficult questions and initiate an Era of Ferment during which established enterprises gauge the impact of the disruption in their worlds and try to adjust. The ferment creates uncertainty, high risk, considerable wasted resources, and no interoperability.
The ferment is ended when the community agrees on a dominant design and works together to make the design work. Instead of deciding what they are going to do, they work to make what they are going to do better. Continue reading →