Cannot write log.json: [Errno 28] No space left on device

with latest docker image 88860a6a7c09 ODM stop with the following error
‘’‘[Cannot write log.json: [Errno 28] No space left on device’‘’

I am running ODM with Singularity on a grid computer. With more or less the same data, I have this error for maybe 30% of the different runs. Maybe the docker/singularity image needs some space to run somewhere… 70% of other runs do not failed with exactly the same parameters and same number of pictures. I am not running of space on my disk

023-03-27 16:14:15,771 DEBUG: Found 145488 points in 16.967890977859497s
2023-03-27 16:14:29,972 DEBUG: Found 135873 points in 16.014127731323242s
2023-03-27 16:14:30,022 DEBUG: Found 133640 points in 16.09254240989685s
2023-03-27 16:14:30,146 DEBUG: Found 136841 points in 16.33139395713806s
2023-03-27 16:14:30,607 DEBUG: Found 140882 points in 16.61655282974243s
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x14e7a915d1f0>
Traceback (most recent call last):
  File "/usr/lib/python3.9/zipfile.py", line 1807, in __del__
    self.close()
  File "/usr/lib/python3.9/zipfile.py", line 1824, in close
    self.fp.seek(self.start_dir)
ValueError: seek of closed file
Traceback (most recent call last):
  File "/code/SuperBuild/install/bin/opensfm/bin/opensfm_main.py", line 25, in <module>
    commands.command_runner(
  File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/command_runner.py", line 38, in command_runner
    command.run(data, args)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/command.py", line 13, in run
    self.run_impl(data, args)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/commands/detect_features.py", line 13, in run_impl
    detect_features.run_dataset(dataset)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/actions/detect_features.py", line 15, in run_dataset
    features_processing.run_features_processing(data, data.images(), False)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/features_processing.py", line 79, in run_features_processing
    parallel_map(process, arguments, processes, 1)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/context.py", line 59, in parallel_map
    res = Parallel(batch_size=batch_size)(delayed(func)(arg) for arg in args)
  File "/usr/local/lib/python3.9/dist-packages/joblib/parallel.py", line 1056, in __call__
    self.retrieve()
  File "/usr/local/lib/python3.9/dist-packages/joblib/parallel.py", line 935, in retrieve
    self._output.extend(job.get(timeout=self.timeout))
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/lib/python3.9/dist-packages/joblib/_parallel_backends.py", line 595, in __call__
    return self.func(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/joblib/parallel.py", line 262, in __call__
    return [func(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/joblib/parallel.py", line 262, in <listcomp>
    return [func(*args, **kwargs)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/features_processing.py", line 146, in process
    read_images(queue, data, images, counter, expected, force)
  File "/code/SuperBuild/install/bin/opensfm/opensfm/features_processing.py", line 170, in read_images
    queue.put(args, block=True, timeout=full_queue_timeout)
  File "/usr/lib/python3.9/queue.py", line 148, in put
    raise Full
queue.Full

===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 1
Traceback (most recent call last):
  File "/code/stages/odm_app.py", line 81, in execute
    self.first_stage.run()
  File "/code/opendm/types.py", line 398, in run
    self.next_stage.run(outputs)
  File "/code/opendm/types.py", line 398, in run
    self.next_stage.run(outputs)
  File "/code/opendm/types.py", line 398, in run
    self.next_stage.run(outputs)
  File "/code/opendm/types.py", line 377, in run
    self.process(self.args, outputs)
  File "/code/stages/run_opensfm.py", line 35, in process
    octx.feature_matching(self.rerun())
  File "/code/opendm/osfm.py", line 408, in feature_matching
    raise e
  File "/code/opendm/osfm.py", line 396, in feature_matching
    self.run('detect_features')
  File "/code/opendm/osfm.py", line 34, in run
    system.run('"%s" %s "%s"' %
  File "/code/opendm/system.py", line 110, in run
    raise SubprocessException("Child returned {}".format(retcode), retcode)
opendm.system.SubprocessException: Child returned 1

===== Done, human-readable information to follow... =====

e[91m[ERROR]   Uh oh! Processing stopped because of strange values in the reconstruction. This is often a sign that the input data has some issues or the software cannot deal with it. Have you followed best practices for data acquisition? See https://docs.opendronemap.org/flying/e[0m
Cannot write log.json: [Errno 28] No space left on device
```
1 Like

Does the volume hosting your containers have space? How are containers handled in Singularity? Are they transparent and can fill the entire volume like with Docker/PodMan for Linux, or are they a FS image like with Docker for Windows and MacOS and you need to tweak the max size it can grow to?

Does the volume hosting your containers have space?
=> Yes, There is 400Go free
How are containers handled in Singularity?
=> I really don’t know !
Are they transparent and can fill the entire volume like with Docker/PodMan for Linux, or are they a FS image like with Docker for Windows and MacOS and you need to tweak the max size it can grow to?
=> I really don’t know !

Singularity is still like a black box for me…

1 Like

I have never used it, so I don’t know either.

Have you begun to look through their documentation yet? It might have the answers we need.

I think for OpenDroneMap with singularity, I had to build it in a sandbox to avoid such issues, but I’m no expert on singularity.

1 Like

Do you remember how to do or use the sandbox ?
Thanks

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.