"Bad CRC-32" issue

Hi Folks,
i have a strange issue on ODM - when i try deal with 5k pics (urban/forest scenario - nothing wird, like my others successful done job)

i get an error:

Traceback (most recent call last):
  File "/code/SuperBuild/install/bin/opensfm/bin/opensfm_main.py", line 25, in <module>
    commands.command_runner(
  File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/command_runner.py", line 37, in command_runner
    command.run(data, args)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/command.py", line 12, in run
    self.run_impl(data, args)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/match_features.py", line 11, in run_impl
    match_features.run_dataset(dataset)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/actions/match_features.py", line 14, in run_dataset
    pairs_matches, preport = matching.match_images(data, {}, images, images)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/matching.py", line 56, in match_images
    match_images_with_pairs(data, config_override, exifs, pairs),
  File "/code/SuperBuild/install/bin/opensfm/opensfm/matching.py", line 80, in match_images_with_pairs
    matches = context.parallel_map(match_unwrap_args, args, processes, jobs_per_process)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/context.py", line 56, in parallel_map
    res = Parallel(batch_size=batch_size)(delayed(func)(arg) for arg in args)
  File "/usr/local/lib/python3.9/dist-packages/joblib/parallel.py", line 1061, in __call__
    self.retrieve()
  File "/usr/local/lib/python3.9/dist-packages/joblib/parallel.py", line 940, in retrieve
    self._output.extend(job.get(timeout=self.timeout))
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/lib/python3.9/dist-packages/joblib/_parallel_backends.py", line 595, in __call__
    return self.func(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/joblib/parallel.py", line 262, in __call__
    return [func(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/joblib/parallel.py", line 262, in <listcomp>
    return [func(*args, **kwargs)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/matching.py", line 210, in match_unwrap_args
    matches = match(im1, im2, camera1, camera2, data, config_override, pose)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/matching.py", line 579, in match
    p1, p2, matches, matcher_type = _match_descriptors_impl(
  File "/code/SuperBuild/install/bin/opensfm/opensfm/matching.py", line 353, in _match_descriptors_impl
    features_data1 = feature_loader.instance.load_all_data(
  File "/code/SuperBuild/install/bin/opensfm/opensfm/feature_loading.py", line 109, in load_all_data
    features_data = self._load_all_data_masked(data, image)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/feature_loading.py", line 160, in _load_all_data_masked
    features_data = self._load_all_data_unmasked(data, image)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/feature_loading.py", line 154, in _load_all_data_unmasked
    return self._load_features_nocache(data, image)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/feature_loading.py", line 201, in _load_features_nocache
    features_data = data.load_features(image)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/dataset.py", line 314, in load_features
    return features.FeaturesData.from_file(f, self.config)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/features.py", line 142, in from_file
    return getattr(cls, "_from_file_v%d" % version)(s, config)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/features.py", line 200, in _from_file_v2
    descriptors = data["descriptors"]
  File "/usr/local/lib/python3.9/dist-packages/numpy/lib/npyio.py", line 254, in __getitem__
    return format.read_array(bytes,
  File "/usr/local/lib/python3.9/dist-packages/numpy/lib/format.py", line 779, in read_array
    data = _read_bytes(fp, read_size, "array data")
  File "/usr/local/lib/python3.9/dist-packages/numpy/lib/format.py", line 908, in _read_bytes
    r = fp.read(size - len(data))
  File "/usr/lib/python3.9/zipfile.py", line 922, in read
    data = self._read1(n)
  File "/usr/lib/python3.9/zipfile.py", line 1012, in _read1
    self._update_crc(data)
  File "/usr/lib/python3.9/zipfile.py", line 940, in _update_crc
    raise BadZipFile("Bad CRC-32 for file %r" % self.name)
zipfile.BadZipFile: Bad CRC-32 for file 'descriptors.npy'
terminate called without an active exception
/code/SuperBuild/install/bin/opensfm/bin/opensfm: line 12:   101 Aborted                 (core dumped) "$PYTHON" "$DIR"/opensfm_main.py "$@"

===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 134
Traceback (most recent call last):
  File "/code/stages/odm_app.py", line 94, in execute
    self.first_stage.run()
  File "/code/opendm/types.py", line 346, in run
    self.next_stage.run(outputs)
  File "/code/opendm/types.py", line 346, in run
    self.next_stage.run(outputs)
  File "/code/opendm/types.py", line 346, in run
    self.next_stage.run(outputs)
  File "/code/opendm/types.py", line 327, in run
    self.process(self.args, outputs)
  File "/code/stages/run_opensfm.py", line 35, in process
    octx.feature_matching(self.rerun())
  File "/code/opendm/osfm.py", line 326, in feature_matching
    self.run('match_features')
  File "/code/opendm/osfm.py", line 34, in run
    system.run('%s %s "%s"' %
  File "/code/opendm/system.py", line 106, in run
    raise SubprocessException("Child returned {}".format(retcode), retcode)
opendm.system.SubprocessException: Child returned 134

maybe someone have an idea?

1 Like

Hmm :thinking:

Can you verify that the disk is healthy using SMART, and run a filesystem repair on it before re-running the task?

I’m gathering that this happens every time?

possibly is the hardware issue? last time i changed mobo in my pc… hm. i will deal with cpu/ram clock and we will see. the ssd are freshly formatted so should by fine

1 Like

If it keeps recurring and you’re sure everything is good hardware-wise, would this be a dataset you can share?

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.