python-xarray：open_dataarray HPC上的分割错误

  • Z时代
  • 2024-01-10
  • 分类：问答

我在努力理解为什么在我使用的HPC中使用open_dataarray的NetCDF文件中的读取产生Segmentation fault (core dumped)。但是，当我在我的Mac上使用open_dataarray在文件中读取时，它工作正常。python-xarray：open_dataarray HPC上的分割错误

从进一步调查这是我创建似乎有问题的NPac文件（小节）。下面是我把生成文件的步骤：

  • WaveWatchIII输出文件

  • 提取七月：$ncks -d time,0,30 in.nc out.nc能在HPC打开out.nc

  • 提取变量：$ncks -v hs in.nc out.nc可以在HPC
    • 打开out.nc

  • 提取域名：$ncks -d longitude,100.0,290.0 -d latitude,0.0,65.0 in.nc out.nc无法在HPC上打开out.nc，但可以在Mac上打开out.nc。

这是我第一次看到这个问题，我相信这是由于NetCDF文件域的属性。我猜它可能与版本有关呢？我在HPC上完成了大部分工作，并使用我的Mac进行测试和理解，因此在HPC上进行这项工作会很好。

的NetCDF文件可以下载here

ncdump -h ww3.Hs.July.NPac.nc 
netcdf ww3.Hs.July.NPac { 
dimensions: 
    time = UNLIMITED ; // (31 currently) 
    latitude = 66 ; 
    longitude = 191 ; 
variables: 
    short hs(time, latitude, longitude) ; 
      hs:long_name = "significant height of wind and swell waves" ; 
      hs:standard_name = "sea_surface_wave_significant_height" ; 
      hs:globwave_name = "significant_wave_height" ; 
      hs:units = "m" ; 
      hs:_FillValue = -32767s ; 
      hs:scale_factor = 0.002f ; 
      hs:add_offset = 0.f ; 
      hs:valid_min = 0 ; 
      hs:valid_max = 32000 ; 
    float latitude(latitude) ; 
      latitude:units = "degree_north" ; 
      latitude:long_name = "latitude" ; 
      latitude:standard_name = "latitude" ; 
      latitude:valid_min = -90.f ; 
      latitude:valid_max = 90.f ; 
      latitude:axis = "Y" ; 
    float longitude(longitude) ; 
      longitude:units = "degree_east" ; 
      longitude:long_name = "longitude" ; 
      longitude:standard_name = "longitude" ; 
      longitude:valid_min = -180.f ; 
      longitude:valid_max = 180.f ; 
      longitude:axis = "X" ; 
    double time(time) ; 
      time:long_name = "julian day (UT)" ; 
      time:standard_name = "time" ; 
      time:units = "days since 1850-01-01T00:00:00Z" ; 
      time:conventions = "relative julian days with decimal part (as parts of the day)" ; 
      time:axis = "T" ; 
// global attributes: 
      :WAVEWATCH_III_version_number = "4.18b" ; 
      :WAVEWATCH_III_switches = "NC4 F90 NOGRB NOPA LRB4 SHRD PR3 UQ FLX0 LN1 ST4 STAB0 NL1 BT1 DB1 MLIM TR0 BS0 IC0 REF0 XX0 WNT1 WNX1 CRT1 CRX1 O0 O1 O2 O3 O4 O5 O6 O7 O11 O14 TRKNC" ; 
      :SDS4\ namelist\ parameter\ WHITECAPWIDTH = 0.3f ; 
      :product_name = "ww3.199307.nc" ; 
      :area = "Indian Ocean Pacfic 1 degree" ; 
      :latitude_resolution = " 1.0000000" ; 
      :longitude_resolution = " 1.0000000" ; 
      :southernmost_latitude = "-70.0000000" ; 
      :northernmost_latitude = "65.0000000" ; 
      :westernmost_longitude = "20.0000000" ; 
      :easternmost_longitude = "295.0000000" ; 
      :minimum_altitude = "-12000 m" ; 
      :maximum_altitude = "9000 m" ; 
      :altitude_resolution = "n/a" ; 
      :start_date = "1993-07-01T00:00:00Z" ; 
      :stop_date = "1993-07-31T00:00:00Z" ; 
      :history = "Sat Dec 2 17:53:06 2017: ncks -O -d longitude,100.0,290.0 -d latitude,0.0,65.0 /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.Hs.July.nc /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.Hs.July.NPac.nc\nSat Dec 2 17:53:06 2017: ncks -O -v hs /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.July.nc /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.Hs.July.nc\nSat Dec 2 17:53:05 2017: ncks -O -d time,0,30 /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.19930701_19940630.nc /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.July.nc\nFri Nov 4 13:50:57 2016: ncrcat -O -o tmp.nc ww3.199307.nc ww3.199308.nc ww3.199309.nc ww3.199310.nc ww3.199311.nc ww3.199312.nc ww3.199401.nc ww3.199402.nc ww3.199403.nc ww3.199404.nc ww3.199405.nc ww3.199406.nc" ; 
      :nco_openmp_thread_number = 1 ; 
      :NCO = "4.3.7" ;

在HPC：

$ nc-config --version 
netCDF 4.2.1.1 
>>> xr.show_versions() 
INSTALLED VERSIONS 
------------------ 
commit: None 
python: 3.6.3.final.0 
python-bits: 64 
OS: Linux 
OS-release: 2.6.32-431.el6.x86_64 
machine: x86_64 
processor: x86_64 
byteorder: little 
LC_ALL: None 
LANG: en_US.UTF-8 
LOCALE: en_US.UTF-8 
xarray: 0.10.0 
pandas: 0.20.2 
numpy: 1.13.1 
scipy: 0.19.1 
netCDF4: 1.2.4 
h5netcdf: None 
Nio: None 
bottleneck: 1.2.1 
cyordereddict: None 
dask: 0.16.0 
matplotlib: 2.0.2 
cartopy: 0.15.1 
seaborn: None 
setuptools: 27.2.0 
pip: 9.0.1 
conda: 4.3.30 
pytest: None 
IPython: None 
sphinx: None 
>>> xr.open_dataarray('ww3.Hs.July.NPac.nc') 
Segmentation fault (core dumped)

我的Mac：

$ nc-config --version 
netCDF 4.4.1 
xr.show_versions() 
INSTALLED VERSIONS 
------------------ 
commit: None 
python: 3.6.3.final.0 
python-bits: 64 
OS: Darwin 
OS-release: 17.2.0 
machine: x86_64 
processor: i386 
byteorder: little 
LC_ALL: en_US.UTF-8 
LANG: en_US.UTF-8 
LOCALE: en_US.UTF-8 
xarray: 0.10.0 
pandas: 0.20.1 
numpy: 1.12.1 
scipy: 1.0.0 
netCDF4: 1.2.4 
h5netcdf: None 
Nio: None 
bottleneck: 1.2.1 
cyordereddict: None 
dask: 0.16.0 
matplotlib: 2.1.0 
cartopy: 0.15.1 
seaborn: 0.8.1 
setuptools: 36.2.7 
pip: 9.0.1 
conda: 4.3.29 
pytest: 3.2.0 
IPython: 6.2.1 
sphinx: 1.5.6 
In [10]:xr.open_dataarray('ww3.Hs.July.NPac.nc') 
Out[10]: 
<xarray.DataArray 'hs' (time: 31, latitude: 66, longitude: 191)> 
[390786 values with dtype=float64] 
Coordinates: 
    * latitude (latitude) float32 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 ... 
    * longitude (longitude) float32 100.0 101.0 102.0 103.0 104.0 105.0 106.0 ... 
    * time  (time) datetime64[ns] 2013-07-01 2013-07-02 2013-07-03 ... 
Attributes: 
    long_name:  significant height of wind and swell waves 
    standard_name: sea_surface_wave_significant_height 
    globwave_name: significant_wave_height 
    units:   m 
    valid_min:  0 
    valid_max:  32000

回答：

道歉。它看起来像我的ncks命令没有完成财产。与xarray没有任何关系，只是奇怪我的Mac没有抛出一个错误。更奇怪的是，给予out.nc不同的文件名似乎解决它

$ncks -O -d longitude,100.0,290.0 -d latitude,0.0,65.0 ww3.Hs.July.nc ww3.Hs.July.NPac.nc 
$python 
>>>import xarray as xr 
>>>xr.open_dataarray('ww3.Hs.July.NPac.nc') 
Segmentation fault (core dumped) 
$ncks -O -d longitude,100.0,290.0 -d latitude,0.0,65.0 ww3.Hs.July.nc ww3.Hs.July.NPac1.nc 
$python 
>>>import xarray as xr 
>>>xr.open_dataarray('ww3.Hs.July.NPac1.nc') 
<xarray.DataArray 'hs' (time: 31, latitude: 66, longitude: 191)> 
[390786 values with dtype=float64] 
Coordinates: 
    * latitude (latitude) float32 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 ... 
    * longitude (longitude) float32 100.0 101.0 102.0 103.0 104.0 105.0 106.0 ... 
    * time  (time) datetime64[ns] 1993-07-01 1993-07-02 1993-07-03 ... 
Attributes: 
    long_name:  significant height of wind and swell waves 
    standard_name: sea_surface_wave_significant_height 
    globwave_name: significant_wave_height 
    units:   m 
    valid_min:  0 
    valid_max:  32000

以上是 python-xarray：open_dataarray HPC上的分割错误 的全部内容， 来源链接： utcz.com/qa/259465.html

回到顶部