I have been having a miserable time this week reading SEGY files. This is data from the largest seismic acquisition company in the world whose client is the 7th largest oil company in the world. So if anyone could get it right you would think it would be these two.
The SEGY standard for storing seismic data is used throughout the oil industry but is showing it's age quite badly. Problems include:
- non standard header information
- the textual header can be in ASCII or EBCDIC
- crazy machinations to save a few bytes
- binary data stored in IEEE or IBM floating point
- etc.
Some of the most glaring deficiencies are being patched up with a v2 release in the works. Details are here: http://www.seg.org/documents/51956/6062543/SEGY+2.0+draft+February+2014.
This is not a standard you can implement just by reading the spec and this new version won't help fix that. The continual abuse of the standard means that it takes calendar years to 'debug into existence' a library that can withstand real world data files. This is hugely frustrating for both the software developers and the geoscientists who have to work with it.
There was a discussion on this excellent blog recently where HDF5 was mentioned
- http://www.agilegeoscience.com/journal/2014/3/27/how-to-load-seg-y-data.html
- http://www.agilegeoscience.com/journal/2014/3/26/what-is-seg-y.html
I would prefer to see a new standard that is less complicated than SEGY and with no dependencies on third party libraries. So while I think technologies like HDF5 (and JavaSeis etc.) may have a role within a company they are not a replacement for SEGY. Even a technology like XML is going to be outlived by the data files we will have to read in the future. So I think the format will need to be extremely simple and conservative.
Some features of a format that could replace SEGY might include:
- one character encoding (ASCII or UTF-8)
- all headers in human readable form – key-value pairs?
- mandatory header information, e.g. shotpoints, x/y locations etc.
- standard unit definitions, map projections etc.
- traces may be binary but only IEEE
- data should be in one file (SEGY got this right)
- no data compression
So now for the question – what do we have to do to move beyond the extremely frustrating SEGY standard? Is there anything out there already we should just start evangelizing? If not, what do you think a replacement format needs to look like?
Thanks for reading. I feel much better after this rant and will get back to debugging why byte number 3603 is wrong.
EDIT: After chilling out over the weekend here are some more thoughts. SEGY is complicated for a number of reasons:
- inherent complexity
- accidental complexity
- different expectations for the format from users
So SEGY is hard to use because the seismic problem domain is an inherently difficult one to model. We are lucky to have the benefit of the wisdom of those who came up with the format.
Even so, a lot of cruft has accumulated in the format over the years. This creates a non-trivial cognitive burden that I could do without but will realistically just have to deal with.
The last point though is interesting. We already have SEGD for storing the raw field data. This format is a lot more challenging than SEGY but is also a good place to bury a lot of the inherent complexity of real world seismic data.
When people suggest HDF as a replacement I guess that (maybe) they are viewing SEGY as a 'processing' format?
Last week I was thinking of SEGY as a vehicle for sharing final seismic volumes with interpreters. I read at the weekend that 80% of SEGY headers are not populated in real world data. So, this is speculation on my part, but perhaps what we need is
- SEGD for raw field data
- something else for actually processing
- and something else again for sharing data between processors and interpreters
To accurately, and reproducibly, load seismic data into interpretation systems we don't need much meta-data but it absolutely does need to be present. The hand-off from processors to interpreters via SEGY is where I think we could use a new approach.