bpch2nc in GEOS-Chem v12.9.1

[chen-yuy]

I noticed that the files tracerinfo.dat and diaginfo.dat created by GEOS-Chem had been adjusted, which made old version bpch2nc conversion error. I want to know if there was a new version of the bpch2nc python module could solve this problem, and if it is possible send it to me cyy1446767286@gmail.com.
Thank you so much.

[yantosca]

Thanks for writing. There is an example script in our https://github.com/geoschem/gcpy package: gcpy/examples/bpch_to_nc/bpch2nc.py. You can try that.

Also -- I am not sure which version of GEOS-Chem you are using, but most of the bpch diagnostics have been superseded by the GEOS-Chem History diagnostics, which write output to netCDF format. Unless you are using one of the diagnostics that has not been ported to netCDF yet, we recommend using the History diagnostics instead of the bpch diagnostics.

[chen-yuy]

Thank you for replying.

Actually I am using GEOS-Chem V12.9.1, and I want to apply the bpch2nc module for ND51 Satellite files.
Here is the problem I met, when I use old version(GEOS Chem V12.0.0) of diaginfo.dat and tracerinfo.dat in which tracers not listed in alphabetical order, everything is ok except for the output netCDF file in wrong orders.
And if I used new diaginfo.dat and tracerinfo.dat produced by GEOS-Chem V12.9.1, it didn't work. I also tried the python file in gcpy package and the function open_bpchdataset didn't work when applying new diaginfo.dat and tracerinfo.dat.
In addition, I also try the combinations of old tracerinfo.dat plus new diaginfor.dat, and new tracerinfo.dat plus old diaginfor.dat, neither of them work and I am quite confused.

Your help is highly appreciated, if you need more error information please let me know.

[chen-yuy]

Dear Bob, Thank you for replying in Github.Actually I am using GEOS-Chem V12.9.1, and I want to apply the bpch2nc module for ND51 Satellite files.Here is the problem I met, when I use old version(GEOS Chem V12.0.0) of diaginfo.dat and tracerinfo.dat in which tracers not listed in alphabetical order, everything is ok except for the output netCDF file in wrong orders.And if I used new diaginfo.dat and tracerinfo.dat produced by GEOS-Chem V12.9.1, it didn't work. I also tried the python file in gcpy package you offered, but the function open_bpchdataset didn't work when applying new diaginfo.dat and tracerinfo.dat.In addition, I also try the combinations of old tracerinfo.dat plus new diaginfor.dat, and new tracerinfo.dat plus old diaginfor.dat, neither of them work and I am quite confused.Your help is highly appreciated

[yantosca]

Can you post here the steps that you did and the ouptut that you got? That would help us diagnose the issue.

Also, in our GCPy package, there is an example script to convert bpch to netcdf. Clone the gcpy repo and then look in examples/diagnostics/bpch_to_nc/bpch2nc.py.

[lizziel]

HI @chen-yuy, I was not able to reproduce the problem with using the xbpch function open_bpchdataset with the ND51 satellite output files. What version of xbpch are you using? Can you include your output error message? Also, please include your diaginfo.dat and tracerinfo.dat files. You can do this by adding extension '.txt' to the filenames and drag and drop them into the GitHub comment box.

[chen-yuy]

Thank you for the reply. Here is my error message when I using the xbpch function.
@lizziel @yantosca

---

KeyError Traceback (most recent call last)
D:\Users\ding9\anaconda3\envs\bpch\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
2896 try:
-> 2897 return self._engine.get_loc(key)
2898 except KeyError:

pandas_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas_libs\index_class_helper.pxi in pandas._libs.index.Int64Engine._check_type()

KeyError: 'name'

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last)
in
1 import xbpch
2
----> 3 data = xbpch.open_bpchdataset("ts_satellite.20110420.bpch")

D:\Users\ding9\anaconda3\envs\bpch\lib\site-packages\xbpch\core.py in open_bpchdataset(filename, fields, categories, tracerinfo_file, diaginfo_file, endian, decode_cf, memmap, dask, return_store)
79 tracerinfo_file=tracerinfo_file,
80 diaginfo_file=diaginfo_file, endian=endian,
---> 81 use_mmap=memmap, dask_delayed=dask
82 )
83 ds = xr.Dataset.load_store(store)

D:\Users\ding9\anaconda3\envs\bpch\lib\site-packages\xbpch\core.py in init(self, filename, fields, categories, fix_cf, mode, endian, diaginfo_file, tracerinfo_file, use_mmap, dask_delayed)
278
279 # Parse the binary file and prepare to add variables to the DataStore
--> 280 self._bpch._read_var_data()
281
282 # Create storage dicts for variables and attributes, to be used later

D:\Users\ding9\anaconda3\envs\bpch\lib\site-packages\xbpch\bpch.py in read_var_data(self)
312 var_attr['unit'] = unit
313
--> 314 vname = diag['name']
315 fullname = category_name.strip() + "" + vname
316

D:\Users\ding9\anaconda3\envs\bpch\lib\site-packages\pandas\core\frame.py in getitem(self, key)
2978 if self.columns.nlevels > 1:
2979 return self._getitem_multilevel(key)
-> 2980 indexer = self.columns.get_loc(key)
2981 if is_integer(indexer):
2982 indexer = [indexer]

D:\Users\ding9\anaconda3\envs\bpch\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
2897 return self._engine.get_loc(key)
2898 except KeyError:
-> 2899 return self._engine.get_loc(self._maybe_cast_indexer(key))
2900 indexer = self.get_indexer([key], method=method, tolerance=tolerance)
2901 if indexer.ndim > 1 or indexer.size > 1:

pandas_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas_libs\index_class_helper.pxi in pandas._libs.index.Int64Engine._check_type()

KeyError: 'name'

Here are my diaginfo.dat and tracerinfo.dat of GEOS-ChemV12.9.1
diaginfo.dat.txt
tracerinfo.dat.txt

[lizziel]

Hi @chen-yuy , I think this might be this issue that I reported recently: darothen/xbpch#25.
Check what version of xbpch and pandas you are using and try reverting to xbpch 0.25.1. If you still get the same issue with a different set of versions please add to the existing xbpch issue or create a new one on the xbpch GitHub.