Skip to content

Writing index file twice causes ecCodes assertion failed: field->file'` #18

@acowlikeobject

Description

@acowlikeobject
  • Ubuntu 18.04.3 LTS
  • eccodes 2.16.0
  • conda install -c conda-forge eccodes and pip install eccodes-python

If I try to write an index file off the same grib file twice:

import eccodes

grib_name = 'era5-levels-members.grib'

for i in range(2):
    index_name = grib_name + f'.idx{i}'
    with eccodes.GribIndex(grib_name, ['level']) as index:
        index.write(index_name)

    with eccodes.GribIndex(file_index=index_name) as index:
        msgs = index.select({'level': 500})  # Fails here on second run of loop.
        print(f'Using {index_name}: {len(msgs)} messages')

I get an ecCodes assertion error:

Using era5-levels-members.grib.idx0: 192 messages
ecCodes assertion failed: `field->file' in /home/conda/feedstock_root/build_artifacts/eccodes_1579074340882/work/src/grib_index.c:1470
Aborted (core dumped)

The two index files written are not identical:

cmp era5-levels-members.grib.idx0 era5-levels-members.grib.idx1
era5-levels-members.grib.idx0 era5-levels-members.grib.idx1 differ: byte 36, line 1

However, if I replace index.write with a command line call to grib_index_build, it works:

import subprocess
import eccodes

grib_name = 'era5-levels-members.grib'

for i in range(2):
    index_name = grib_name + f'.idx{i}'
    subprocess.call(f'grib_index_build -o {index_name} -k level {grib_name}'.split(' '))

    with eccodes.GribIndex(file_index=index_name) as index:
        msgs = index.select({'level': 500})  # Fails here on second run of loop.
        print(f'Using {index_name}: {len(msgs)} messages')

Output:

--- grib_index_build: processing era5-levels-members.grib
--- grib_index_build: keys included in the index file era5-levels-members.grib.idx0:
--- level
--- level = { 500, 850 }
--- 160 messages indexed
Using era5-levels-members.grib.idx0: 192 messages
--- grib_index_build: processing era5-levels-members.grib
--- grib_index_build: keys included in the index file era5-levels-members.grib.idx1:
--- level
--- level = { 500, 850 }
--- 160 messages indexed
Using era5-levels-members.grib.idx1: 192 messages

Seems like some state is being retained incorrectly after the first call?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions