Why should I use international standards?
The post below is a guest post from Will Poynter, lead engineer at CLOSER Discovery, based in the UK.
Space is big. Really big. You just won’t believe how vastly, hugely, mind-bogglingly big it is. – Douglas Adams
And now data is big too. Lots of you will have heard, or read, that in the past two years we have created more data than all previous years combined. And there appears to be no indication or reason to think that this rate of growth is going to slow down anytime soon.
In a previous post, I shared my thoughts on the big data issue, and why, in order to correctly utilize the new quantity of data, we need new tools and infrastructure. But currently, there is a huge impediment in the way of achieving these new tools and infrastructure. Most data is not held using international standards.
Let me jump back a little. I am an alumnus of the University of Manchester, where an impressive number of buildings, roads and parks are named after Joseph Whitworth. Whitworth was a pivotal figure, during the 19th century, for his creation of a set of standards covering screws, nuts, bolts and tools. This may sound simple, but before Whitworth’s work all kinds of machines, including trains, were made with nuts and bolts of any size and quality. This caused huge issues when a train broke down hundreds of miles from the yard where it was maintained. After train manufacturers began using Whitworth’s standards, engineers from all of Britain, using their own tools could now fix the trains.
Now, moving back to the 21st century, we are still in the early days of the data revolution and just like the early days of the industrial revolution, we are being limited by our inability to move data between environments. Most of us will have experienced this with a poorly formatted CSV file, but organizations that store and manipulate data with their own homegrown solutions are causing the same problem to happen with just about every type of data you can imagine, on a colossal scale.
Not using international standards for data can be tempting for a number of reasons, for example, to make initial development quicker, or to more easily allow custom features and functions. But, I assure you, the short term gains will be canceled out over the years to come. Firstly, if you use an international standard it is easier to recruit a new member of staff that will understand your data quickly. Secondly, you will eventually have to migrate your data to a new environment, either because you want to use a new software provider, or maybe you need to supplement your current data analytics, and using international standards allows companies to use off-the-shelf solutions to these issues.
To summarize, your data, my data, everyone’s data, will grow exponentially for the foreseeable future and will need constant development and new solutions to handle. By not using international standards for your data and software, you will greatly limit your flexibility in the years to come.
30 May 2017, Will Poynter